No MetaPhlAn BowTie2 database found (--index option)!

Hi Giacomo, Awesome! Thanks for the follow up and glad to hear you are all set! Please post again if you have other issues.

Thank you,
Lauren

Hi there!

I’m having trouble running the demo as well. For context, I’m on linux server and I’ve tried the exact commands listed on the humann3 site (link here). I’ve made sure that metaphlan is correct version, as well as humann. I’ve tried both the conda installation and the pip installation but consistently run into the same error when I try to run the demo. For fun, I tried running metaphlan --install but got the same error again. The error reads as follows:

###########################################
Downloading MetaPhlAn database
Please note due to the size this might take a few minutes

File /home/kmorten/bin/anaconda3/envs/biobakery3/lib/python3.7/site-packages/metaphlan/metaphlan_databases/mpa_v30_CHOCOPhlAn_201901.tar already present!

File /home/kmorten/bin/anaconda3/envs/biobakery3/lib/python3.7/site-packages/metaphlan/metaphlan_databases/mpa_v30_CHOCOPhlAn_201901.md5 already present!
Traceback (most recent call last):
File “/home/kmorten/bin/anaconda3/envs/biobakery3/bin/metaphlan”, line 10, in
sys.exit(main())
File “/home/kmorten/bin/anaconda3/envs/biobakery3/lib/python3.7/site-packages/metaphlan/metaphlan.py”, line 925, in main
pars[‘index’] = check_and_install_database(pars[‘index’], pars[‘bowtie2db’], pars[‘bowtie2_build’], pars[‘nproc’], pars[‘force_download’])
File “/home/kmorten/bin/anaconda3/envs/biobakery3/lib/python3.7/site-packages/metaphlan/init.py”, line 301, in check_and_install_database
download_unpack_tar(index, bowtie2_db, bowtie2_build, nproc, use_zenodo)
File “/home/kmorten/bin/anaconda3/envs/biobakery3/lib/python3.7/site-packages/metaphlan/init.py”, line 180, in download_unpack_tar
bt2_build_output = subp.check_output([bowtie2_build, ‘–usage’], stderr=subp.STDOUT)
File “/home/kmorten/bin/anaconda3/envs/biobakery3/lib/python3.7/subprocess.py”, line 411, in check_output
**kwargs).stdout
File “/home/kmorten/bin/anaconda3/envs/biobakery3/lib/python3.7/subprocess.py”, line 512, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command ‘[‘bowtie2-build’, ‘–usage’]’ returned non-zero exit status 127.
############################################

I’ve read a lot of the posts that seem to have traces of the issue I’m experiencing, but no luck. Any suggestions would be much appreciated!

Thank you!
Kate

I was getting this error when I was running HumanN3. I fixed it (and am now getting OP’s error) by running

conda install bowtie2

for some reason bowtie2 wasn’t installed.

Also I didn’t have tbb installed and had to do that as well

I’m getting this error and

metaphlan —install

does not solve it. I do have the latest versions of metaphlan and humann

I am also running into this error. Just installed today

It might be a fault of the new database.
I managed to trick metaphlan into downloading an earlier version, by editing the mpa_latest file in the databases folder. we’ll see how that goes

no luck at all

Like @Massoud_Maher, for me this error was due to the lack of a particular threading library for C++, called tbb2. I think Bowtie needs it. I couldn’t find tbb2 on Conda. I’m in an Ubuntu environment, so I installed it outside of conda with apt:

sudo apt-get install libtbb2

Hello biobakery community,
I used alpha version of humann3 and an older version of MetaPhlAn it worked.
However, I installed the newest version of both, installed the most up to date chocoplan, uniref databases, it shows the same error message.

Error message returned from metaphlan :
No MetaPhlAn BowTie2 database found (–index option)!
Expecting location bowtie2db

Is there a way to use the newer version and still be able to run?
Thanks!!
Yike

Hi!

Just wanted to chime in and say that im getting the same error as @YikeShen. Im using humann v3.0.1 and metaphlan 3.0.14.
Running metaphlan --install returns “The database is installed”. I checked the metaphlan_databases folder in my metaphlan installation on conda and the following files are there, so it looks ok:

README.txt
mpa_latest
mpa_v30_CHOCOPhlAn_201901.1.bt2.tmp
mpa_v30_CHOCOPhlAn_201901.2.bt2.tmp
mpa_v30_CHOCOPhlAn_201901.3.bt2.tmp
mpa_v30_CHOCOPhlAn_201901.4.bt2.tmp
mpa_v30_CHOCOPhlAn_201901.76.sa
mpa_v30_CHOCOPhlAn_201901.77.sa
mpa_v30_CHOCOPhlAn_201901.78.sa
mpa_v30_CHOCOPhlAn_201901.79.sa
mpa_v30_CHOCOPhlAn_201901.80.sa
mpa_v30_CHOCOPhlAn_201901.81.sa
mpa_v30_CHOCOPhlAn_201901.82.sa
mpa_v30_CHOCOPhlAn_201901.83.sa
mpa_v30_CHOCOPhlAn_201901.84.sa
mpa_v30_CHOCOPhlAn_201901.85.sa
mpa_v30_CHOCOPhlAn_201901.86.sa
mpa_v30_CHOCOPhlAn_201901.87.sa
mpa_v30_CHOCOPhlAn_201901.88.sa
mpa_v30_CHOCOPhlAn_201901.89.sa
mpa_v30_CHOCOPhlAn_201901.90.sa
mpa_v30_CHOCOPhlAn_201901.91.sa
mpa_v30_CHOCOPhlAn_201901.92.sa
mpa_v30_CHOCOPhlAn_201901.93.sa
mpa_v30_CHOCOPhlAn_201901.94.sa
mpa_v30_CHOCOPhlAn_201901.95.sa
mpa_v30_CHOCOPhlAn_201901.96.sa
mpa_v30_CHOCOPhlAn_201901.fna
mpa_v30_CHOCOPhlAn_201901.fna.bz2
mpa_v30_CHOCOPhlAn_201901.md5
mpa_v30_CHOCOPhlAn_201901.pkl
mpa_v30_CHOCOPhlAn_201901.tar

Ive tried manually specifying the --index and --bowtie2db paths in --metaphlan-options but it still returns the same error messsage:

Error message returned from metaphlan :
No MetaPhlAn BowTie2 database found (–index option)!
Expecting location bowtie2db

The exact call i used is the following:

humann --metaphlan-options "-t rel_ab --index mpa_v30_CHOCOPhlAn_201901 --bowtie2db /priv/c3-sysbio/sysbio/LeaversData_Omics/MetabogenMelanoma/ici_env/lib/python3.9/site-packages/metaphlan/metaphlan_databases" --input /priv/c3-sysbio/sysbio/LeaversData_Omics/MetabogenMelanoma/fastq_filt_cat/D9_FDPL210411324-1a_HWWWJDSX2_L3_cat.fq.gz --output /priv/c3-sysbio/sysbio/LeaversData_Omics/MetabogenMelanoma/test_out --threads 18

Best,
Angelo

I looked through the metaphlan.py source code now and it looks like the database files i have are incorrect. The first 4 should end with .bt2 and not .bt2.tmp, also the rev1.bt2 and rev2.bt2 are missing. Gonna see if i can re-download these somehow.

EDIT:
Managed to fix it now! simply re-downloaded the databases and re-built the bowtie2 index in a separate folder using metaphlan --install --bowtie2db /path/to/mpa_dbs

I managed to run the humann_test function a few days ago without any problems, using just the installation instructions on your website (humann3 – The Huttenhower Lab). Also, just runnning metaphlan --install returns that the databases have already been installed, even though in my case they still had the .tmp extension as well. Might be good to add an extra check in these functions in a future release.

For any of u having similar issues, maybe double check that the metaphlan_databases folder in your conda installation contains the following files:

mpa_v30_CHOCOPhlAn_201901.1.bt2
mpa_v30_CHOCOPhlAn_201901.2.bt2
mpa_v30_CHOCOPhlAn_201901.3.bt2
mpa_v30_CHOCOPhlAn_201901.4.bt2
mpa_v30_CHOCOPhlAn_201901.rev1.bt2
mpa_v30_CHOCOPhlAn_201901.rev2.bt2
mpa_v30_CHOCOPhlAn_201901.fna.bz2
mpa_v30_CHOCOPhlAn_201901.md5
mpa_v30_CHOCOPhlAn_201901.pkl
mpa_v30_CHOCOPhlAn_201901.tar

1 Like

Hi everyone,

I am running HUMAnN3 on an ssh client and dealing with the same error when I run the demo:
Error message returned from metaphlan :
No MetaPhlAn BowTie2 database found (–index option)!
Expecting location bowtie2db

I have updated or tried updating humann, metaphlan, bowtie2, and tbb, and the issue still occurs when I have the latest version of all of these packages.

I attempted @angelolimeta’s solution to re-download the databases and rebuild the bowtie2 index using metaphlan --install --bowtie2db mpa_dbs and that partially failed (therefore not resolving the issue), with this error message:
Building Bowtie2 indexes
Fatal error running ‘bowtie2-build --quiet --threads 4 -f mpa_dbs/mpa_v30_CHOCOPhlAn_201901.fna mpa_dbs/mpa_v30_CHOCOPhlAn_201901’
Error message: ‘Command ‘[‘bowtie2-build’, ‘–quiet’, ‘–threads’, ‘4’, ‘-f’, ‘mpa_dbs/mpa_v30_CHOCOPhlAn_201901.fna’, ‘mpa_dbs/mpa_v30_CHOCOPhlAn_201901’]’ returned non-zero exit status 247.’

Any assistance would be very appreciated!

Those two versions I installed in December 2020 works for me.

I have encountered the same error and find that every time I try to manually download the databases, I get different results. Sometimes the mpa_v30_CHOCOPhlAn_201901.rev.1.bt2/ mpa_v30_CHOCOPhlAn_201901.rev.2.bt2 files are just not included. Sometimes one or more of the file sizes are zero. I have followed another thread (Humann 3 bowtie error - #9 by nasnuvens) that suggested deleting the .bt2 files and manually rebuilding the index. This seemed to work for me but I am not sure if the files are complete.

For someone who has successfully used these databases: what are the final sizes of each file in the metaphlan_databases directory?

Hi @sabrinaarif
Here’s a screenshot of the file sizes in my metaphlan db folders

1 Like

Hello,

I had the same issue reported here. I have installed humann3 using a humann3.yml file. I git clone the humann3 from github and tryed to run the test samples within the ~/humann/examples folder. I used this command:

humann --input demo.fastq.gz --output demo_fastq --threads 16

and I got this output

Creating output directory: /panfs/roc/groups/11/noyes046/diazo005/humann/examples/demo_fastq
Output files will be written to: /panfs/roc/groups/11/noyes046/diazo005/humann/examples/demo_fastq
Decompressing gzipped file ...


Running metaphlan ........

CRITICAL ERROR: Error executing: /home/noyes046/diazo005/.conda/envs/humann3/bin/metaphlan /panfs/roc/groups/11/
noyes046/diazo005/humann/examples/demo_fastq/demo_humann_temp/tmpp_ptudn3/tmpzbt96zmj -t rel_ab -o /panfs/roc/gr
oups/11/noyes046/diazo005/humann/examples/demo_fastq/demo_humann_temp/demo_metaphlan_bugs_list.tsv --input_type
fastq --bowtie2out /panfs/roc/groups/11/noyes046/diazo005/humann/examples/demo_fastq/demo_humann_temp/demo_metap
hlan_bowtie2.txt --nproc 16

Error message returned from metaphlan :
No MetaPhlAn BowTie2 database found (--index option)!
Expecting location bowtie2db
Exiting...

Then I just looked in several forums and ended up doing a fresh installation of the new release of Humann3, as indicated in the release note: Announcing MetaPhlAn 3.1 and HUMAnN 3.1

The following steps are detailed here (humann3 – The Huttenhower Lab), briefly:

Create and activate a new conda environment called “biobakery3”

conda create --name biobakery3 python=3.7
conda activate biobakery3

Set conda channel priority

conda config --add channels defaults
conda config --add channels bioconda
conda config --add channels conda-forge
conda config --add channels biobakery

Install HUMAnN 3.1 software with demo databases:

conda install humann -c biobakery

Tested my INSTALLATION

humann_test

I moved to examples directory (within the already copied humann directory):

cd ~/humann/examples

I ran the following code in the “examples” folder

humann --input demo.fastq.gz --output demo_fastq --threads 16

Everything worked out and I got the following OUTPUT:

Creating output directory: /panfs/roc/groups/11/noyes046/diazo005/humann/examples/demo_fastq
Output files will be written to: /panfs/roc/groups/11/noyes046/diazo005/humann/examples/demo_fastq
Decompressing gzipped file ...


Running metaphlan ........

Found t__SGB1815 : 53.72% of mapped reads ( s__Bacteroides_dorei,s__Phocaeicola_vulgatus,s__Bacteroides_v$
lgatus,s__Bacteroides_sp_9_1_42FAA,s__Bacteroides_sp_3_1_33FAA,s__Bacteroides_sp_NMBE5,s__Phocaeicola_dor$
i,s__Bacteroidaceae_bacterium,g__Phocaeicola.s__Phocaeicola_vulgatus,g__Bacteroides.s__Bacteroides_sp_9_1$
42FAA,g__Bacteroides.s__Bacteroides_sp_3_1_33FAA,g__Bacteroides.s__Bacteroides_sp_NMBE5,g__Bacteroidaceae$
unclassified.s__Bacteroidaceae_bacterium )
Found t__SGB1814 : 46.28% of mapped reads ( s__Bacteroides_vulgatus,s__Bacteroides_dorei,s__Phocaeicola_d$
rei,s__Bacteroides_sp_3_1_33FAA,s__Bacteroides_sp_4_3_47FAA,s__Bacteroides_sp_3_1_40A,s__Bacteroides_sp_A$
18_9,s__Bacteroides_sp_AM23_18,s__Bacteroides_sp_AF39_10AT,s__Bacteroides_sp_AF32_15BH,s__Bacteroides_sp_$
F25_18,s__Bacteroides_sp_AF16_29,s__Bacteroides_sp_AM28_6,s__Bacteroides_sp_AM27_13,s__Bacteroides_sp_AM2$
_11,s__Bacteroides_sp_AF15_23LB,s__Bacteroides_sp_AF17_1,g__Phocaeicola.s__Phocaeicola_dorei,g__Bacteroid$
s.s__Bacteroides_sp_3_1_33FAA,g__Bacteroides.s__Bacteroides_sp_4_3_47FAA,g__Bacteroides.s__Bacteroides_sp$
3_1_40A,g__Bacteroides.s__Bacteroides_sp_AM18_9,g__Bacteroides.s__Bacteroides_sp_AM23_18,g__Bacteroides.s_
_Bacteroides_sp_AF39_10AT,g__Bacteroides.s__Bacteroides_sp_AF32_15BH,g__Bacteroides.s__Bacteroides_sp_AF25
_18,g__Bacteroides.s__Bacteroides_sp_AF16_29,g__Bacteroides.s__Bacteroides_sp_AM28_6,g__Bacteroides.s__Bac
teroides_sp_AM27_13,g__Bacteroides.s__Bacteroides_sp_AM26_11,g__Bacteroides.s__Bacteroides_sp_AF15_23LB,g_
_Bacteroides.s__Bacteroides_sp_AF17_1 )

Total species selected from prescreen: 47

Selected species explain 100.00% of predicted community composition


Creating custom ChocoPhlAn database ........

Running bowtie2-build ........                                                                  [870/1996]


Running bowtie2 ........

Total bugs from nucleotide alignment: 2
g__Bacteroides.s__Bacteroides_dorei: 1270 hits
g__Bacteroides.s__Bacteroides_vulgatus: 1335 hits

Total gene families from nucleotide alignment: 572

Unaligned reads after nucleotide alignment: 87.5952380952 %


Running diamond ........


Aligning to reference database: uniref90_demo_prots_v201901b.dmnd

Total bugs after translated alignment: 3
g__Bacteroides.s__Bacteroides_dorei: 1270 hits
g__Bacteroides.s__Bacteroides_vulgatus: 1335 hits
unclassified: 1017 hits

Total gene families after translated alignment: 615

Unaligned reads after translated alignment: 82.8857142857 %


Computing gene families ...

Computing pathways abundance and coverage ...

Output files created:
/panfs/roc/groups/11/noyes046/diazo005/humann/examples/demo_fastq/demo_genefamilies.tsv
/panfs/roc/groups/11/noyes046/diazo005/humann/examples/demo_fastq/demo_pathabundance.tsv
/panfs/roc/groups/11/noyes046/diazo005/humann/examples/demo_fastq/demo_pathcoverage.tsv


I obtained the 3 output files from the tutorial. So, suggestion: Download the new Humann 3.1 (with the new Metaphlan release).

Gerardo R Diaz
CVM-UMN

I got the same problem yesterday. I solved it by installing specific versions of metaphlan.
The version humann 3.0.0.alpha.3 & metaphlan 3.0.9 would work perfectly.

what’s the command of installing specific versions of metaphlan?

pip install metaphlan==[here is version string]
Such as, pip install metaphlan==3.0.9
>=, <=, vice versa.

I am also stuck on this step. I followed the recommended installation

conda create --name biobakery3 python=3.7
conda config --add channels defaults
conda config --add channels bioconda
conda config --add channels conda-forge
conda config --add channels biobakery
conda install humann -c biobakery```

I then downloaded all the databases:

humann_databases --download chocophlan full /scratch4/jsuez1/week12/chocophlan --update-config yes
humann_databases --download uniref uniref90_diamond/scratch4/jsuez1/week12/diamond --update-config yes
humann_databases --download utility_mapping full /scratch4/jsuez1/week12/diamond --update-config yes

Then when I try to run either a sample of mine or the demo.fastq.gz, i get this error:

Running metaphlan ........

CRITICAL ERROR: Error executing: /home/hhallow1/.conda/envs/biobakery3/bin/metaphlan /scratch4/jsuez1/week12_virome/sample_results/demo_humann_temp/tmpm6r5uqir/tmpe2u0gnbk -t rel_ab -o /scratch4/jsuez1/week12_virome/sample_results/demo_humann_temp/demo_metaphlan_bugs_list.tsv --input_type fastq --bowtie2out /scratch4/jsuez1/week12_virome/sample_results/demo_humann_temp/demo_metaphlan_bowtie2.txt

Error message returned from metaphlan :

No MetaPhlAn BowTie2 database found (--index option)!

Expecting location bowtie2db

Exiting...

Ive tried re-installing the databases, resinstalling humann completely, nothing is working. After some digging, it seems that when i download the database using

humann_databases --download chocophlan full /scratch4/jsuez1/week12/chocophlan --update-config yes

The file becomes unzipped. I tried manually combining all files into one .fna by doing this

cat chocophlan/* > mpa_v30_CHOCOPhlAn_201901.fna

then using bowtie-build to build my index

bowtie2-build mpa_v30_CHOCOPhlAn_201901.fna mpa_v30_CHOCOPhlAn_201901

I got about 1000 warnings from bowtie that each reference was empty, i.e contains all Ns rather than ATCG, before i killed the job

So my question is, why is the database unzipping when i download it? what is the correct why to cat these files together to be indexed?

I am running Metaphlan 4.0.2 and Humann 3.6

*I should note that i was able to run Humann before with no problem but lost my install because we started using a new cluster.

Thanks!

1 Like

Hello!
I used anaconda2 to install humann2 for analysis, but when running, I suddenly reported an error
(CRITICAL ERROR: Error executing: /home/hlli/anaconda2/envs/metagenome_env/bin/metaphlan2.py /home/hlli/AUC/temp/concat/ERR688551.fq -t rel_ab -o /home/hlli/AUC/temp/humann2/ERR688551_humann2_temp/ERR688551_metaphlan_bugs_list.tsv --input_type multifastq --bowtie2out /home/hlli/AUC/temp/humann2/ERR688551_humann2_temp/ERR688551_metaphlan_bowtie2.txt --nproc 8

Error message returned from metaphlan2.py :
No MetaPhlAn BowTie2 database found [–bowtie2db option]! (or wrong path provided).
Expecting location ${mpa_dir}/db_v20/map_v20_m200
Exiting…

I have downloaded related related databases and placed them in the specified path, but the same problem still occurs
微信截图_20230201200708
微信截图_20230201200719
I would like to ask the professor what is the best way to solve this problem

Can you try running the MetaPhlAn command external to HUMAnN? If you’re manually specifying an index path (or expecting to) HUMAnN won’t have that information. You can pass additional information to MetaPhlAn within the HUMAnN CLI using the --metaphlan-options flag.