# Dealing with "Too many samples were discarded" **URL:** \ **Category:** StrainPhlAn\ **Created:** [March 20, 2023, 2:53pm UTC](https://forum.biobakery.org/t/dealing-with-too-many-samples-were-discarded/4994 "2023-03-20T14:53:44Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![kescobo](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.biobakery.org/kescobo/32/47_2.png) [@kescobo](https://forum.biobakery.org/u/kescobo)\ **Post date:** [March 20, 2023, 2:53pm UTC](https://forum.biobakery.org/t/dealing-with-too-many-samples-were-discarded/4994/1 "2023-03-20T14:53:44Z")
I’m attempting to do a strainphlan analysis of _Blautia wexlerae_, but nothing I’m doing is working. I think the problem might be the reference genomes that I’m using, but I’m using the 2 primary genomes [from NCBI](https://www.ncbi.nlm.nih.gov/data-hub/taxonomy/1121115/). Is there another place to get correctly formatted references? ## Error Here’s the primary command I was trying, along with the output: ```auto ❯ strainphlan -s consenseus_markers/*.pkl -m db_markers/s __Blautia_wexlerae.fna -r ../genomes/Blautia_wexlerae/GCF_025148125.1_ASM2514812v1_genomic.fna ../genomes/Blautia_wexlerae/GCA_025148125.1_ASM2514812v1_genomic.fna -o output -n 16 -c s__ Blautia_wexlerae --mutation_rate Mon Mar 20 10:23:57 2023: Start StrainPhlAn 3.1.0 execution Mon Mar 20 10:23:57 2023: Creating temporary directory... Mon Mar 20 10:23:57 2023: Done. Mon Mar 20 10:23:57 2023: Getting markers from main sample files... Mon Mar 20 10:23:57 2023: Done. Mon Mar 20 10:23:57 2023: Getting markers from main reference files...Warning: [blastn] Examining 5 or more matches is recommended Warning: [blastn] Examining 5 or more matches is recommended Mon Mar 20 10:23:58 2023: Done. Mon Mar 20 10:23:58 2023: Removing bad markers / samples... [e] Phylogeny can not be inferred. Too many samples were discarded Mon Mar 20 10:23:58 2023: Stop StrainPhlAn 3.0 execution. ``` ## Samples I have 371 stool samples from kids that have _B wexlerae_ according to metaphlan, with the following abundance / read depth distributions (this is all samples, not just those with _B. wexlerae_): ![image](https://canada1.discourse-cdn.com/flex027/uploads/biobakery/original/2X/1/18542ef2e167f1834a775ca27eb978e3b628574e.png) ## Things I’ve tried - I tried downloading ~100 reference genomes from ENA. After fixing the identifier lines to make them play nicely with `makeblastdb`, I tried including them in the `-r` argument, and the only difference is that I get ~100 lines of `Warning: [blastn] Examining 5 or more matches is recommended`. - I tried restricting the samples I tested to those with \>1%, \>2%, \>5%, and \>10% abundance Same result each time. - I tried with ` --marker_in_n_samples 20 --sample_with_n_markers 20`, same result. - I tried running without reference genomes, sample problem ## Earlier commands ```auto ❯ sample2markers -i sams/*.sam.bz2 -o consensus_markers/ -n 16 Thu Mar 16 11:56:50 2023: Start samples to markers execution Thu Mar 16 11:56:50 2023: Decompressing samples... Thu Mar 16 11:57:17 2023: Done. Thu Mar 16 11:57:17 2023: Converting samples to BAM format... Thu Mar 16 11:57:42 2023: Done. Thu Mar 16 11:57:42 2023: Sorting BAM samples... Thu Mar 16 11:58:05 2023: Done. Thu Mar 16 11:58:05 2023: Getting consensus markers from samples... Thu Mar 16 11:58:05 2023: Processing sample: consensus_markers/tmpfor3b8wt/FG02304_S75.bam Thu Mar 16 12:02:04 2023: Done. Thu Mar 16 12:02:04 2023: Processing sample: consensus_markers/tmpfor3b8wt/FG02296_S59.bam Thu Mar 16 12:07:09 2023: Done. #... etc ``` I don’t have the output from `extract_markers`, but i ran `extract_markers -c s__Blautia_wexlerae -o db_markers/` and it proceeded without errors. EDIT: Just re-ran extract\_markers to see ```auto ❯ extract_markers -c s__Blautia_wexlerae -o db_markers/ Mon Mar 20 10:55:34 2023: Start extract markers execution Mon Mar 20 10:55:34 2023: Generating DB markers FASTA... Mon Mar 20 10:56:13 2023: Done. Mon Mar 20 10:56:13 2023: Loading MetaPhlan 3.1.0 databa se... Mon Mar 20 10:56:20 2023: Done. Mon Mar 20 10:56:20 2023: Number of markers for the clad e "s__Blautia_wexlerae": 150 Mon Mar 20 10:56:20 2023: Exporting markers... Mon Mar 20 10:56:31 2023: Done. Mon Mar 20 10:56:32 2023: Finish extract markers execution (57 .27 seconds): Results are stored at "db_markers/ ``` ## Final command: ```auto ❯ strainphlan -s (cat bwex_gt10_pkl.txt) \ # gets samples only > 10% abundance -m db_markers/s__Blautia_wexlerae.fna \ -r ../genomes/Blautia_wexlerae/GCA_025148125.1_ASM2514812v1_genomic.fna \ # can leave this out, same result -o output -n 16 -c s__Blautia_wexlerae --mutation_rate \ --marker_in_n_samples 20 --sample_with_n_markers 20 Mon Mar 20 10:57:23 2023: Start StrainPhlAn 3.1.0 execution Mon Mar 20 10:57:23 2023: Creating temporary directory... Mon Mar 20 10:57:23 2023: Done. Mon Mar 20 10:57:23 2023: Getting markers from main sample fil es... Mon Mar 20 10:57:24 2023: Done. Mon Mar 20 10:57:24 2023: Getting markers from main reference files...Warning: [blastn] Examining 5 or more matches is recom mended Mon Mar 20 10:57:25 2023: Done. Mon Mar 20 10:57:25 2023: Removing bad markers / samples... [e] Phylogeny can not be inferred. Too many samples were discarded Mon Mar 20 10:57:25 2023: Stop StrainPhlAn 3.0 execution. ``` ---
**Author:** ![aitor.blancomiguez](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.biobakery.org/aitor.blancomiguez/32/86_2.png) [@aitor.blancomiguez](https://forum.biobakery.org/u/aitor.blancomiguez)\ **Post date:** [March 23, 2023, 8:44am UTC](https://forum.biobakery.org/t/dealing-with-too-many-samples-were-discarded/4994/2 "2023-03-23T08:44:22Z")
Hi @kescobo I think the best thing to understand what is going on is to run strainphlan with the --debug option and if possible share the full content of the tmp folder created ---
**Author:** ![kescobo](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.biobakery.org/kescobo/32/47_2.png) [@kescobo](https://forum.biobakery.org/u/kescobo)\ **Post date:** [March 24, 2023, 4:20pm UTC](https://forum.biobakery.org/t/dealing-with-too-many-samples-were-discarded/4994/3 "2023-03-24T16:20:15Z")
Hmm - looks like the only thing in the directory is an `fna` file: ```auto exa --tree output/debugdir/ output/debugdir ├── blastn └── s__Blautia_wexlerae.fna ``` So nothing is even getting searched? ---
**Author:** ![aitor.blancomiguez](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.biobakery.org/aitor.blancomiguez/32/86_2.png) [@aitor.blancomiguez](https://forum.biobakery.org/u/aitor.blancomiguez)\ **Post date:** [March 30, 2023, 12:09pm UTC](https://forum.biobakery.org/t/dealing-with-too-many-samples-were-discarded/4994/4 "2023-03-30T12:09:36Z")
I see, so it might be a problem of having too high thresholds for the `--sample_with_n_markers` and` --marker_in_n_samples` parameters. You could try to reduce them a bit and try to see the results