The bioBakery help forum

Humann can not run with metaphlan together

hi,
I installed metaphlan whose database in /data/xmjd/miniconda2/envs/py37/lib/python3.7/site-packages/metaphlan/metaphlan_databases/. When I run humann like following:
(py37) [zhangwenping@localhost try_humann]$ humann --input /data/liying_metagenome/clean_data_liying/SRR5130527_paired.1.fastq --output try_SRR5130527_paired.1 --metaphlan /data/xmjd/miniconda2/envs/py37/lib/python3.7/site-packages/metaphlan
Output files will be written to: /data/try_humann/try_SRR5130527_paired.1
Removing spaces from identifiers in input file …

WARNING: Can not call software version for bowtie2

Running metaphlan …

CRITICAL ERROR: Error executing: /data/xmjd/miniconda2/envs/py37/bin/metaphlan /data/try_humann/try_SRR5130527_paired.1/SRR5130527_paired.1_humann_temp/tmpbs8jnhgf/tmpf8cqxgux -t rel_ab -o /data/try_humann/try_SRR5130527_paired.1/SRR5130527_paired.1_humann_temp/SRR5130527_paired.1_metaphlan_bugs_list.tsv --input_type fastq --bowtie2out /data/try_humann/try_SRR5130527_paired.1/SRR5130527_paired.1_humann_temp/SRR5130527_paired.1_metaphlan_bowtie2.txt

Error message returned from metaphlan :

Downloading https://www.dropbox.com/sh/7qze7m7g9fe2xjg/AAA4XDP85WHon_eHvztxkamTa/file_list.txt?dl=1

Warning: Unable to download https://www.dropbox.com/sh/7qze7m7g9fe2xjg/AAA4XDP85WHon_eHvztxkamTa/file_list.txt?dl=1
Traceback (most recent call last):
File “/data/xmjd/miniconda2/envs/py37/bin/metaphlan”, line 10, in
sys.exit(main())
File “/data/xmjd/miniconda2/envs/py37/lib/python3.7/site-packages/metaphlan/metaphlan.py”, line 916, in main
pars[‘index’] = check_and_install_database(pars[‘index’], pars[‘bowtie2db’], pars[‘bowtie2_build’], pars[‘nproc’], pars[‘force_download’])
File “/data/xmjd/miniconda2/envs/py37/lib/python3.7/site-packages/metaphlan/init.py”, line 269, in check_and_install_database
index = resolve_latest_database(bowtie2_db, ls_f[‘mpa_latest’], force_redownload_latest)
UnboundLocalError: local variable ‘ls_f’ referenced before assignment

Can you tell me which options for humann will show the database of metaphlan locate in /data/xmjd/miniconda2/envs/py37/lib/python3.7/site-packages/metaphlan/metaphlan_databases/?

Thank you very much!

wenping

Hi Wenping, Thanks for all the info! The MetaPhlAn databases are usually installed by default in a subfolder of the MetaPhlAn executable. Sorry we don’t have an option in HUMAnN to provide the location of the MetaPhlAn database. Based on your info I think the databases would be installed in the directory you listed /data/xmjd/miniconda2/envs/py37/lib/python3.7/site-packages/metaphlan/metaphlan_databases/ . It looks like the issue with installing the database might be resolved if you upgrade to a newer version of MetaPhlAn. Would you try upgrading the MetaPhlAn software and then rerun? If you continue to see errors please let us know.

Thanks!
Lauren

Hi Lauren,
Thank you for your reply. I upgraded the metaphlan and the version is MetaPhlAn version 3.0.1 (25 Jun 2020). However, it doesn’t work and metaphlan stiil want to download the database from dropbox that is not availble for us.

If Dropbox is not accessible you can try to download the database from Zenodo, here’s the link https://zenodo.org/record/3957592.
I’ve pushed a new release (3.0.2) in which the database automatically is retrieved from Zenodo if Dropbox is not reachable.

Hi Francesco,
Thank you very much for your reply. I downloaded the database of metaphlan and metaphlan can run succesfully when showing the pathway of the database. However, no options for running humann to show the pathway of database of metaphlan.

You can use the HUMAnN --metaphlan-options parameter to provide additional MetaPhlAn parameters for the internal exection (e.g. -x <dbname> --bowtie2db <db_path>)

Hi Francesco,
I run the following code, but it doesn’t work:

(py37) [zhangwenping@localhost try_humann]$ humann --input /data/liying_metagenome/clean_data_liying/SRR5130527_paired.1.fastq --output try_SRR5130527_paired.1 --metaphlan-options -x mpa_v30_CHOCOPhlAn_201901 --bowtie2db /data/xmjd/miniconda2/envs/py37/lib/python3.7/site-packages/metaphlan/metaphlan_databases/
usage: humann [-h] [–version] [-v] [-r] [–bypass-prescreen]
[–bypass-nucleotide-index] [–bypass-translated-search]
[–bypass-nucleotide-search] -i <input.fastq> -o
[–nucleotide-database <nucleotide_database>]
[–annotation-gene-index <3>]
[–protein-database <protein_database>] [–evalue <1.0>]
[–search-mode {uniref50,uniref90}] [–metaphlan ]
[–metaphlan-options <metaphlan_options>]
[–diamond-options <diamond_options>]
[–bowtie-options <bowtie_options>] [–o-log <sample.log>]
[–log-level {DEBUG,INFO,WARNING,ERROR,CRITICAL}]
[–remove-temp-output] [–threads <1>]
[–prescreen-threshold <0.01>]
[–nucleotide-identity-threshold <0.0>]
[–translated-identity-threshold <Automatically: 50.0 or 80.0, Custom: 0.0-100.0>]
[–translated-subject-coverage-threshold <50.0>]
[–nucleotide-subject-coverage-threshold <50.0>]
[–translated-query-coverage-threshold <90.0>]
[–nucleotide-query-coverage-threshold <90.0>]
[–bowtie2 ] [–usearch ]
[–rapsearch ] [–diamond ]
[–taxonomic-profile <taxonomic_profile.tsv>]
[–id-mapping <id_mapping.tsv>]
[–translated-alignment {usearch,rapsearch,diamond}]
[–xipe {on,off}] [–minpath {on,off}] [–pick-frames {on,off}]
[–gap-fill {on,off}] [–output-format {tsv,biom}]
[–output-max-decimals <10>] [–output-basename <sample_name>]
[–remove-stratified-output]
[–remove-column-description-output]
[–input-format {fastq,fastq.gz,fasta,fasta.gz,sam,bam,blastm8,genetable,biom}]
[–pathways-database <pathways_database.tsv>]
[–pathways {metacyc,unipathway}]
[–memory-use {minimum,maximum}]
humann: error: argument --metaphlan-options: expected one argument

You are missing the quotes, it should be --metaphlan-options "-x mpa_v30_CHOCOPhlAn_201901 --bowtie2db /data/xmjd/miniconda2/envs/py37/lib/python3.7/site-packages/metaphlan/metaphlan_databases/"

thank you very much. It works, but another error shows as following:

(py37) [zhangwenping@localhost try_humann]$ humann --input /data/liying_metagenome/clean_data_liying/SRR5130527_paired.1.fastq --output try_SRR5130527_paired.1 --metaphlan-options ‘-x mpa_v30_CHOCOPhlAn_201901 --bowtie2db /data/xmjd/miniconda2/envs/py37/lib/python3.7/site-packages/metaphlan/metaphlan_databases/’
Output files will be written to: /data/try_humann/try_SRR5130527_paired.1
Removing spaces from identifiers in input file …

WARNING: Can not call software version for bowtie2

Running metaphlan …

CRITICAL ERROR: Error executing: /data/xmjd/miniconda2/envs/py37/bin/metaphlan /data/try_humann/try_SRR5130527_paired.1/SRR5130527_paired.1_humann_temp/tmpkcc2_rom/tmpw83r1n2g -x mpa_v30_CHOCOPhlAn_201901 --bowtie2db /data/xmjd/miniconda2/envs/py37/lib/python3.7/site-packages/metaphlan/metaphlan_databases/ -o /data/try_humann/try_SRR5130527_paired.1/SRR5130527_paired.1_humann_temp/SRR5130527_paired.1_metaphlan_bugs_list.tsv --input_type fastq --bowtie2out /data/try_humann/try_SRR5130527_paired.1/SRR5130527_paired.1_humann_temp/SRR5130527_paired.1_metaphlan_bowtie2.txt

Error message returned from metaphlan :
Use of uninitialized value $bt2_args[2] in join or string at /data/xmjd/miniconda2/envs/py37/bin/bowtie2 line 423.
Use of uninitialized value bt2_args[3] in join or string at /data/xmjd/miniconda2/envs/py37/bin/bowtie2 line 423. Use of uninitialized value [2] in string eq at /data/xmjd/miniconda2/envs/py37/bin/bowtie2 line 360.
Use of uninitialized value $
[3] in string eq at /data/xmjd/miniconda2/envs/py37/bin/bowtie2 line 360.
Use of uninitialized value in exists at /data/xmjd/miniconda2/envs/py37/bin/bowtie2 line 81.
Use of uninitialized value in exists at /data/xmjd/miniconda2/envs/py37/bin/bowtie2 line 81.
Use of uninitialized value $bt2_args[2] in join or string at /data/xmjd/miniconda2/envs/py37/bin/bowtie2 line 459.
Use of uninitialized value $bt2_args[3] in join or string at /data/xmjd/miniconda2/envs/py37/bin/bowtie2 line 459.
Traceback (most recent call last):
File “/data/xmjd/miniconda2/envs/py37/bin/metaphlan”, line 10, in
sys.exit(main())
File “/data/xmjd/miniconda2/envs/py37/lib/python3.7/site-packages/metaphlan/metaphlan.py”, line 991, in main
min_alignment_len=pars[‘min_alignment_len’], read_min_len=pars[‘read_min_len’], min_mapq_val=pars[‘min_mapq_val’])
File “/data/xmjd/miniconda2/envs/py37/lib/python3.7/site-packages/metaphlan/metaphlan.py”, line 429, in run_bowtie2
nreads = int(read_fastx_stderr[0])
IndexError: list index out of range

Could you run read_fastx.py -l 70 <input file> > /dev/null on the input fastq and see if the total number of reads is printed on stderr?

I run the code and no message was shown as the following:

(py37) [zhangwenping@localhost clean_data_liying] read_fastx.py -l 70 SRR5130527_paired.1.fastq > /dev/null (py37) [zhangwenping@localhost clean_data_liying]

Are you sure the input file is not empty and well-formed?

The input file is as the following:
(py37) [zhangwenping@localhost clean_data_wuqi] read_fastx.py -l 70 CRR055205_paired.1.fastq > /dev/null (py37) [zhangwenping@localhost clean_data_wuqi] more CRR055205_paired.1.fastq
@HWI-ST531R:260:C4GKMACXX:3:1101:1954:1955 1:N:0:GTCTG
TAAAGTATACATTTATAATTAGTTAGGAGAAGAATTTATGTCGTTTTCAGC
+
CCCFFEDEHHHHGJJJIJJJJJHIJJJGGJIJIJJJJIIJHIJHIIJJIJI
@HWI-ST531R:260:C4GKMACXX:3:1101:2131:1942 1:N:0:GTCTG
TCANCAGTCATTATTGCTTCACCATGAACTTCAAAAAAATCACCCGTTG
+
@@@#4ADDHHHHHEIHEIJIIIJJJJJIJIJJIIJJIJJGBDFFHHAEH
@HWI-ST531R:260:C4GKMACXX:3:1101:2108:1942 1:N:0:GTCTG
CATNTTTATCATTAACTGGTCGAACAAGAATGTCAAATAATTGGGGATCGG
+
CCC#4ADFHHHHHJJJJJJHIJJJIJJJHIJJJJJJJJJJJJJJJIFGIIJ
@HWI-ST531R:260:C4GKMACXX:3:1101:2487:1994 1:N:0:GTCTG
GTCATAGATTAAGGCATGAAGAAAACTATTGTGAAGAAATTAATCTCAA
+
?@@DFDD?CFFFFHIIIGEHIIIIGEC@HICFDGHGEGHGHIICDG>DH
@HWI-ST531R:260:C4GKMACXX:3:1101:2647:1943 1:N:0:GTCTG
AGTNGTGTCGTGTGTTGAGGGTGTTATGGAACCTGTATTTATTGACATCTG
+
@@?#4=BDFFFFBGFFEEGII1CCFEGEIDFFIIFDEGIICGIIFFIEFII
@HWI-ST531R:260:C4GKMACXX:3:1101:2676:1948 1:N:0:GTCTG
CCGNTTGGTACAATAATTCATACAGGAGATTTTAAAATAGACTATACTCCA
+
@@C#4ADDBFHHHJJJJJIJDHJJJJIIFHGIJIIIJEEGHIHIJIJJIJJ
@HWI-ST531R:260:C4GKMACXX:3:1101:2694:1962 1:N:0:GTCTG
TTTAAGACAAATTCAAGATGGGATAGAGCTATTTATACTTGGACTAGGAAG
+
CCCFFFFFHHHHHJJIJIJJJJIJIJJJJJJJJJIGJJJJJJJJJIJJIII
@HWI-ST531R:260:C4GKMACXX:3:1101:2570:1982 1:N:0:GTCTG
CTATATCATTAAATCCATCCTTATTGAGAGTTTCTATGATTGCTGCAAAAACTTC
+
CCCFDFDFHHFHHIIHIIIGIJIHHJHIHIAHHGIGGIJIJIJJJJJIIHJJIJJ
(py37) [zhangwenping@localhost clean_data_wuqi]$

It looks like that all the reads may be filtered since the length is below 70bp. You should try adding --read_min_len 49 to --metaphlan-options

Hi, Francesco
Thank you very much, it works.