No MetaPhlAn BowTie2 database found (--index option)!

Microbial community profiling HUMAnN
1

Hi Biobakery community!

I recently upgraded to HUMAnN v3.0 after having used HUMAnN2, but don’t seem to be able to run the demo because of a recurring error.
This text will be hidden
I installed it via conda in a contained environment, as suggested here: humann3 – The Huttenhower Lab

# create env 
conda create --name biobakery3 python=3.7
# activate env
conda activate biobakery3

# update channels
conda config --add channels defaults
conda config --add channels bioconda
conda config --add channels conda-forge
conda config --add channels biobakery

# install biobakery and metaphlan
conda install -c biobakery human
conda install -c bioconda metaphlan

# test installation
humann_test

All of the tests seemed to be fine. I then proceeded to run the demo:

humann -i demo.fastq -o demo_outputs

The first time I ran it, it took 4 hours, after which the job was killed on the cluster I was using.

After that, it returned the following errors:

CRITICAL ERROR: Error executing: /data/home/btx143/.conda/envs/biobakery3/bin/metaphlan /data/home/btx143/shotgun2/demo/data/demo.fastq -t rel_ab -o /data/home/btx143/shotgun2/demo/outputs/demo_output/demo_humann_temp/demo_metaphlan_bugs_list.tsv --input_type fastq --bowtie2out /data/home/btx143/shotgun2/demo/outputs/demo_output/demo_humann_temp/demo_metaphlan_bowtie2.txt

and

Error message returned from metaphlan :
No MetaPhlAn BowTie2 database found (–index option)!
Expecting location /data/home/btx143/.conda/envs/biobakery3/lib/python3.7/site-packages/metaphlan/metaphlan_databases/mpa_v30_CHOCOPhlAn_201901
Exiting…

The issue is that when I check in the directory where the database indices should be, then I find the following files:

mpa_latest
mpa_v30_CHOCOPhlAn_201901.1.bt2
mpa_v30_CHOCOPhlAn_201901.2.bt2
mpa_v30_CHOCOPhlAn_201901.3.bt2
mpa_v30_CHOCOPhlAn_201901.4.bt2
mpa_v30_CHOCOPhlAn_201901.fna
mpa_v30_CHOCOPhlAn_201901.fna.bz2
mpa_v30_CHOCOPhlAn_201901.md5
mpa_v30_CHOCOPhlAn_201901.pkl
mpa_v30_CHOCOPhlAn_201901.rev.1.bt2
mpa_v30_CHOCOPhlAn_201901.rev.2.bt2
mpa_v30_CHOCOPhlAn_201901.tar

And the expected “mpa_v30_CHOCOPhlAn_201901” is missing

I found a similar post in this forum and tried the following:

1. manually delete metaphlan/bowtie2 databases and re-run demo

From what I unnderstand, humann (through metaphlan and bowtie2) should automatically

2. manually delete metaphlan/bowtie2 databases, download the files and use bowtie2 to build the databases

tar -xvf mpa_v30_CHOCOPhlAn_201901.tar
bunzip2 *.bz2
bowtie2-build mpa_v30_CHOCOPhlAn_201901.fna mpa_v30_CHOCOPhlAn_201901

And even tried to run humann specifying the databases option:

humann -i demo.fastq -o sample_results --metaphlan-options “–bowtie2db ${condapath}/${dbpa} -x mpa_v30_CHOCOPhlAn_201901”

After this, I still get the same annoying error:

No MetaPhlAn BowTie2 database found (–index option)!
Expecting location /data/home/btx143/.conda/envs/biobakery3/lib/python3.7/site-packages/metaphlan/metaphlan_databases/mpa_v30_CHOCOPhlAn_201901
Exiting…

Did anyone run into the same or similar error?

Cheers,

Giacomo

2 Likes
2

Hello, Sorry you are having issues running the demo. Thank you for your detailed and organized post! I think you might have an older version of MetaPhlAn installed. Can you check the version number? If it is an older version if you update to the latest version and then run $ metaphlan --install it should download and install the database. Once you have the database installed, you should hopefully be set! If you still see errors double check you have the latest version of HUMAnN installed. Sometimes conda does not always install the latest version but instead it installs an older version that is compatible with the current conda environment.

Thank you,
Lauren

2 Likes
3

Hi Lauren,

Thank you for your reply, and sorry for late answer!
I did have metaphlan v3.0.7 installed already. All it took was to run:

metaphlan --install

(as you sugested)
and the correct databases were installed!

Thank you so much!

Giacomo

4

Hi Giacomo, Awesome! Thanks for the follow up and glad to hear you are all set! Please post again if you have other issues.

Thank you,
Lauren

5

Hi there!

I’m having trouble running the demo as well. For context, I’m on linux server and I’ve tried the exact commands listed on the humann3 site (link here). I’ve made sure that metaphlan is correct version, as well as humann. I’ve tried both the conda installation and the pip installation but consistently run into the same error when I try to run the demo. For fun, I tried running metaphlan --install but got the same error again. The error reads as follows:

###########################################
Downloading MetaPhlAn database
Please note due to the size this might take a few minutes

File /home/kmorten/bin/anaconda3/envs/biobakery3/lib/python3.7/site-packages/metaphlan/metaphlan_databases/mpa_v30_CHOCOPhlAn_201901.tar already present!

File /home/kmorten/bin/anaconda3/envs/biobakery3/lib/python3.7/site-packages/metaphlan/metaphlan_databases/mpa_v30_CHOCOPhlAn_201901.md5 already present!
Traceback (most recent call last):
File “/home/kmorten/bin/anaconda3/envs/biobakery3/bin/metaphlan”, line 10, in
sys.exit(main())
File “/home/kmorten/bin/anaconda3/envs/biobakery3/lib/python3.7/site-packages/metaphlan/metaphlan.py”, line 925, in main
pars[‘index’] = check_and_install_database(pars[‘index’], pars[‘bowtie2db’], pars[‘bowtie2_build’], pars[‘nproc’], pars[‘force_download’])
File “/home/kmorten/bin/anaconda3/envs/biobakery3/lib/python3.7/site-packages/metaphlan/init.py”, line 301, in check_and_install_database
download_unpack_tar(index, bowtie2_db, bowtie2_build, nproc, use_zenodo)
File “/home/kmorten/bin/anaconda3/envs/biobakery3/lib/python3.7/site-packages/metaphlan/init.py”, line 180, in download_unpack_tar
bt2_build_output = subp.check_output([bowtie2_build, ‘–usage’], stderr=subp.STDOUT)
File “/home/kmorten/bin/anaconda3/envs/biobakery3/lib/python3.7/subprocess.py”, line 411, in check_output
**kwargs).stdout
File “/home/kmorten/bin/anaconda3/envs/biobakery3/lib/python3.7/subprocess.py”, line 512, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command ‘[‘bowtie2-build’, ‘–usage’]’ returned non-zero exit status 127.
############################################

I’ve read a lot of the posts that seem to have traces of the issue I’m experiencing, but no luck. Any suggestions would be much appreciated!

Thank you!
Kate

6

I was getting this error when I was running HumanN3. I fixed it (and am now getting OP’s error) by running

conda install bowtie2

for some reason bowtie2 wasn’t installed.

Also I didn’t have tbb installed and had to do that as well

7

I’m getting this error and

metaphlan —install

does not solve it. I do have the latest versions of metaphlan and humann

8

I am also running into this error. Just installed today

It might be a fault of the new database.
I managed to trick metaphlan into downloading an earlier version, by editing the mpa_latest file in the databases folder. we’ll see how that goes

no luck at all

9

Like @Massoud_Maher, for me this error was due to the lack of a particular threading library for C++, called tbb2. I think Bowtie needs it. I couldn’t find tbb2 on Conda. I’m in an Ubuntu environment, so I installed it outside of conda with apt:

sudo apt-get install libtbb2

10

Hello biobakery community,
I used alpha version of humann3 and an older version of MetaPhlAn it worked.
However, I installed the newest version of both, installed the most up to date chocoplan, uniref databases, it shows the same error message.

image
image760×146 42.3 KB

Error message returned from metaphlan :
No MetaPhlAn BowTie2 database found (–index option)!
Expecting location bowtie2db

Is there a way to use the newer version and still be able to run?
Thanks!!
Yike

11

Hi!

Just wanted to chime in and say that im getting the same error as @YikeShen. Im using humann v3.0.1 and metaphlan 3.0.14.
Running metaphlan --install returns “The database is installed”. I checked the metaphlan_databases folder in my metaphlan installation on conda and the following files are there, so it looks ok:

README.txt
mpa_latest
mpa_v30_CHOCOPhlAn_201901.1.bt2.tmp
mpa_v30_CHOCOPhlAn_201901.2.bt2.tmp
mpa_v30_CHOCOPhlAn_201901.3.bt2.tmp
mpa_v30_CHOCOPhlAn_201901.4.bt2.tmp
mpa_v30_CHOCOPhlAn_201901.76.sa
mpa_v30_CHOCOPhlAn_201901.77.sa
mpa_v30_CHOCOPhlAn_201901.78.sa
mpa_v30_CHOCOPhlAn_201901.79.sa
mpa_v30_CHOCOPhlAn_201901.80.sa
mpa_v30_CHOCOPhlAn_201901.81.sa
mpa_v30_CHOCOPhlAn_201901.82.sa
mpa_v30_CHOCOPhlAn_201901.83.sa
mpa_v30_CHOCOPhlAn_201901.84.sa
mpa_v30_CHOCOPhlAn_201901.85.sa
mpa_v30_CHOCOPhlAn_201901.86.sa
mpa_v30_CHOCOPhlAn_201901.87.sa
mpa_v30_CHOCOPhlAn_201901.88.sa
mpa_v30_CHOCOPhlAn_201901.89.sa
mpa_v30_CHOCOPhlAn_201901.90.sa
mpa_v30_CHOCOPhlAn_201901.91.sa
mpa_v30_CHOCOPhlAn_201901.92.sa
mpa_v30_CHOCOPhlAn_201901.93.sa
mpa_v30_CHOCOPhlAn_201901.94.sa
mpa_v30_CHOCOPhlAn_201901.95.sa
mpa_v30_CHOCOPhlAn_201901.96.sa
mpa_v30_CHOCOPhlAn_201901.fna
mpa_v30_CHOCOPhlAn_201901.fna.bz2
mpa_v30_CHOCOPhlAn_201901.md5
mpa_v30_CHOCOPhlAn_201901.pkl
mpa_v30_CHOCOPhlAn_201901.tar

Ive tried manually specifying the --index and --bowtie2db paths in --metaphlan-options but it still returns the same error messsage:

Error message returned from metaphlan :
No MetaPhlAn BowTie2 database found (–index option)!
Expecting location bowtie2db

The exact call i used is the following:

humann --metaphlan-options "-t rel_ab --index mpa_v30_CHOCOPhlAn_201901 --bowtie2db /priv/c3-sysbio/sysbio/LeaversData_Omics/MetabogenMelanoma/ici_env/lib/python3.9/site-packages/metaphlan/metaphlan_databases" --input /priv/c3-sysbio/sysbio/LeaversData_Omics/MetabogenMelanoma/fastq_filt_cat/D9_FDPL210411324-1a_HWWWJDSX2_L3_cat.fq.gz --output /priv/c3-sysbio/sysbio/LeaversData_Omics/MetabogenMelanoma/test_out --threads 18

Best,
Angelo

12

I looked through the metaphlan.py source code now and it looks like the database files i have are incorrect. The first 4 should end with .bt2 and not .bt2.tmp, also the rev1.bt2 and rev2.bt2 are missing. Gonna see if i can re-download these somehow.

EDIT:
Managed to fix it now! simply re-downloaded the databases and re-built the bowtie2 index in a separate folder using metaphlan --install --bowtie2db /path/to/mpa_dbs

I managed to run the humann_test function a few days ago without any problems, using just the installation instructions on your website (humann3 – The Huttenhower Lab). Also, just runnning metaphlan --install returns that the databases have already been installed, even though in my case they still had the .tmp extension as well. Might be good to add an extra check in these functions in a future release.

For any of u having similar issues, maybe double check that the metaphlan_databases folder in your conda installation contains the following files:

mpa_v30_CHOCOPhlAn_201901.1.bt2
mpa_v30_CHOCOPhlAn_201901.2.bt2
mpa_v30_CHOCOPhlAn_201901.3.bt2
mpa_v30_CHOCOPhlAn_201901.4.bt2
mpa_v30_CHOCOPhlAn_201901.rev1.bt2
mpa_v30_CHOCOPhlAn_201901.rev2.bt2
mpa_v30_CHOCOPhlAn_201901.fna.bz2
mpa_v30_CHOCOPhlAn_201901.md5
mpa_v30_CHOCOPhlAn_201901.pkl
mpa_v30_CHOCOPhlAn_201901.tar

1 Like
13

Hi everyone,

I am running HUMAnN3 on an ssh client and dealing with the same error when I run the demo:
Error message returned from metaphlan :
No MetaPhlAn BowTie2 database found (–index option)!
Expecting location bowtie2db

I have updated or tried updating humann, metaphlan, bowtie2, and tbb, and the issue still occurs when I have the latest version of all of these packages.

I attempted @angelolimeta’s solution to re-download the databases and rebuild the bowtie2 index using metaphlan --install --bowtie2db mpa_dbs and that partially failed (therefore not resolving the issue), with this error message:
Building Bowtie2 indexes
Fatal error running ‘bowtie2-build --quiet --threads 4 -f mpa_dbs/mpa_v30_CHOCOPhlAn_201901.fna mpa_dbs/mpa_v30_CHOCOPhlAn_201901’
Error message: ‘Command ‘[‘bowtie2-build’, ‘–quiet’, ‘–threads’, ‘4’, ‘-f’, ‘mpa_dbs/mpa_v30_CHOCOPhlAn_201901.fna’, ‘mpa_dbs/mpa_v30_CHOCOPhlAn_201901’]’ returned non-zero exit status 247.’

Any assistance would be very appreciated!

14

Those two versions I installed in December 2020 works for me.

image
image726×174 51.4 KB

15

I have encountered the same error and find that every time I try to manually download the databases, I get different results. Sometimes the mpa_v30_CHOCOPhlAn_201901.rev.1.bt2/ mpa_v30_CHOCOPhlAn_201901.rev.2.bt2 files are just not included. Sometimes one or more of the file sizes are zero. I have followed another thread (Humann 3 bowtie error - #9 by nasnuvens) that suggested deleting the .bt2 files and manually rebuilding the index. This seemed to work for me but I am not sure if the files are complete.

For someone who has successfully used these databases: what are the final sizes of each file in the metaphlan_databases directory?

16

Hi @sabrinaarif
Here’s a screenshot of the file sizes in my metaphlan db folders

image
image1218×456 55.1 KB

1 Like
17

Hello,

I had the same issue reported here. I have installed humann3 using a humann3.yml file. I git clone the humann3 from github and tryed to run the test samples within the ~/humann/examples folder. I used this command:

humann --input demo.fastq.gz --output demo_fastq --threads 16

and I got this output

Creating output directory: /panfs/roc/groups/11/noyes046/diazo005/humann/examples/demo_fastq
Output files will be written to: /panfs/roc/groups/11/noyes046/diazo005/humann/examples/demo_fastq
Decompressing gzipped file ...


Running metaphlan ........

CRITICAL ERROR: Error executing: /home/noyes046/diazo005/.conda/envs/humann3/bin/metaphlan /panfs/roc/groups/11/
noyes046/diazo005/humann/examples/demo_fastq/demo_humann_temp/tmpp_ptudn3/tmpzbt96zmj -t rel_ab -o /panfs/roc/gr
oups/11/noyes046/diazo005/humann/examples/demo_fastq/demo_humann_temp/demo_metaphlan_bugs_list.tsv --input_type
fastq --bowtie2out /panfs/roc/groups/11/noyes046/diazo005/humann/examples/demo_fastq/demo_humann_temp/demo_metap
hlan_bowtie2.txt --nproc 16

Error message returned from metaphlan :
No MetaPhlAn BowTie2 database found (--index option)!
Expecting location bowtie2db
Exiting...

Then I just looked in several forums and ended up doing a fresh installation of the new release of Humann3, as indicated in the release note: Announcing MetaPhlAn 3.1 and HUMAnN 3.1

The following steps are detailed here (humann3 – The Huttenhower Lab), briefly:

Create and activate a new conda environment called “biobakery3”

conda create --name biobakery3 python=3.7
conda activate biobakery3

Set conda channel priority

conda config --add channels defaults
conda config --add channels bioconda
conda config --add channels conda-forge
conda config --add channels biobakery

Install HUMAnN 3.1 software with demo databases:

conda install humann -c biobakery

Tested my INSTALLATION

humann_test

I moved to examples directory (within the already copied humann directory):

cd ~/humann/examples

I ran the following code in the “examples” folder

humann --input demo.fastq.gz --output demo_fastq --threads 16

Everything worked out and I got the following OUTPUT:

Creating output directory: /panfs/roc/groups/11/noyes046/diazo005/humann/examples/demo_fastq
Output files will be written to: /panfs/roc/groups/11/noyes046/diazo005/humann/examples/demo_fastq
Decompressing gzipped file ...


Running metaphlan ........

Found t__SGB1815 : 53.72% of mapped reads ( s__Bacteroides_dorei,s__Phocaeicola_vulgatus,s__Bacteroides_v$
lgatus,s__Bacteroides_sp_9_1_42FAA,s__Bacteroides_sp_3_1_33FAA,s__Bacteroides_sp_NMBE5,s__Phocaeicola_dor$
i,s__Bacteroidaceae_bacterium,g__Phocaeicola.s__Phocaeicola_vulgatus,g__Bacteroides.s__Bacteroides_sp_9_1$
42FAA,g__Bacteroides.s__Bacteroides_sp_3_1_33FAA,g__Bacteroides.s__Bacteroides_sp_NMBE5,g__Bacteroidaceae$
unclassified.s__Bacteroidaceae_bacterium )
Found t__SGB1814 : 46.28% of mapped reads ( s__Bacteroides_vulgatus,s__Bacteroides_dorei,s__Phocaeicola_d$
rei,s__Bacteroides_sp_3_1_33FAA,s__Bacteroides_sp_4_3_47FAA,s__Bacteroides_sp_3_1_40A,s__Bacteroides_sp_A$
18_9,s__Bacteroides_sp_AM23_18,s__Bacteroides_sp_AF39_10AT,s__Bacteroides_sp_AF32_15BH,s__Bacteroides_sp_$
F25_18,s__Bacteroides_sp_AF16_29,s__Bacteroides_sp_AM28_6,s__Bacteroides_sp_AM27_13,s__Bacteroides_sp_AM2$
_11,s__Bacteroides_sp_AF15_23LB,s__Bacteroides_sp_AF17_1,g__Phocaeicola.s__Phocaeicola_dorei,g__Bacteroid$
s.s__Bacteroides_sp_3_1_33FAA,g__Bacteroides.s__Bacteroides_sp_4_3_47FAA,g__Bacteroides.s__Bacteroides_sp$
3_1_40A,g__Bacteroides.s__Bacteroides_sp_AM18_9,g__Bacteroides.s__Bacteroides_sp_AM23_18,g__Bacteroides.s_
_Bacteroides_sp_AF39_10AT,g__Bacteroides.s__Bacteroides_sp_AF32_15BH,g__Bacteroides.s__Bacteroides_sp_AF25
_18,g__Bacteroides.s__Bacteroides_sp_AF16_29,g__Bacteroides.s__Bacteroides_sp_AM28_6,g__Bacteroides.s__Bac
teroides_sp_AM27_13,g__Bacteroides.s__Bacteroides_sp_AM26_11,g__Bacteroides.s__Bacteroides_sp_AF15_23LB,g_
_Bacteroides.s__Bacteroides_sp_AF17_1 )

Total species selected from prescreen: 47

Selected species explain 100.00% of predicted community composition


Creating custom ChocoPhlAn database ........

Running bowtie2-build ........                                                                  [870/1996]


Running bowtie2 ........

Total bugs from nucleotide alignment: 2
g__Bacteroides.s__Bacteroides_dorei: 1270 hits
g__Bacteroides.s__Bacteroides_vulgatus: 1335 hits

Total gene families from nucleotide alignment: 572

Unaligned reads after nucleotide alignment: 87.5952380952 %


Running diamond ........


Aligning to reference database: uniref90_demo_prots_v201901b.dmnd

Total bugs after translated alignment: 3
g__Bacteroides.s__Bacteroides_dorei: 1270 hits
g__Bacteroides.s__Bacteroides_vulgatus: 1335 hits
unclassified: 1017 hits

Total gene families after translated alignment: 615

Unaligned reads after translated alignment: 82.8857142857 %


Computing gene families ...

Computing pathways abundance and coverage ...

Output files created:
/panfs/roc/groups/11/noyes046/diazo005/humann/examples/demo_fastq/demo_genefamilies.tsv
/panfs/roc/groups/11/noyes046/diazo005/humann/examples/demo_fastq/demo_pathabundance.tsv
/panfs/roc/groups/11/noyes046/diazo005/humann/examples/demo_fastq/demo_pathcoverage.tsv

I obtained the 3 output files from the tutorial. So, suggestion: Download the new Humann 3.1 (with the new Metaphlan release).

Gerardo R Diaz
CVM-UMN

18

I got the same problem yesterday. I solved it by installing specific versions of metaphlan.
The version humann 3.0.0.alpha.3 & metaphlan 3.0.9 would work perfectly.

19

what’s the command of installing specific versions of metaphlan?

20

pip install metaphlan==[here is version string]
Such as, pip install metaphlan==3.0.9
>=, <=, vice versa.