Hi Giacomo, Awesome! Thanks for the follow up and glad to hear you are all set! Please post again if you have other issues.
Thank you,
Lauren
Hi Giacomo, Awesome! Thanks for the follow up and glad to hear you are all set! Please post again if you have other issues.
Thank you,
Lauren
Hi there!
I’m having trouble running the demo as well. For context, I’m on linux server and I’ve tried the exact commands listed on the humann3 site (link here). I’ve made sure that metaphlan is correct version, as well as humann. I’ve tried both the conda installation and the pip installation but consistently run into the same error when I try to run the demo. For fun, I tried running metaphlan --install but got the same error again. The error reads as follows:
###########################################
Downloading MetaPhlAn database
Please note due to the size this might take a few minutes
File /home/kmorten/bin/anaconda3/envs/biobakery3/lib/python3.7/site-packages/metaphlan/metaphlan_databases/mpa_v30_CHOCOPhlAn_201901.tar already present!
File /home/kmorten/bin/anaconda3/envs/biobakery3/lib/python3.7/site-packages/metaphlan/metaphlan_databases/mpa_v30_CHOCOPhlAn_201901.md5 already present!
Traceback (most recent call last):
File “/home/kmorten/bin/anaconda3/envs/biobakery3/bin/metaphlan”, line 10, in
sys.exit(main())
File “/home/kmorten/bin/anaconda3/envs/biobakery3/lib/python3.7/site-packages/metaphlan/metaphlan.py”, line 925, in main
pars[‘index’] = check_and_install_database(pars[‘index’], pars[‘bowtie2db’], pars[‘bowtie2_build’], pars[‘nproc’], pars[‘force_download’])
File “/home/kmorten/bin/anaconda3/envs/biobakery3/lib/python3.7/site-packages/metaphlan/init.py”, line 301, in check_and_install_database
download_unpack_tar(index, bowtie2_db, bowtie2_build, nproc, use_zenodo)
File “/home/kmorten/bin/anaconda3/envs/biobakery3/lib/python3.7/site-packages/metaphlan/init.py”, line 180, in download_unpack_tar
bt2_build_output = subp.check_output([bowtie2_build, ‘–usage’], stderr=subp.STDOUT)
File “/home/kmorten/bin/anaconda3/envs/biobakery3/lib/python3.7/subprocess.py”, line 411, in check_output
**kwargs).stdout
File “/home/kmorten/bin/anaconda3/envs/biobakery3/lib/python3.7/subprocess.py”, line 512, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command ‘[‘bowtie2-build’, ‘–usage’]’ returned non-zero exit status 127.
############################################
I’ve read a lot of the posts that seem to have traces of the issue I’m experiencing, but no luck. Any suggestions would be much appreciated!
Thank you!
Kate
I was getting this error when I was running HumanN3. I fixed it (and am now getting OP’s error) by running
conda install bowtie2
for some reason bowtie2 wasn’t installed.
Also I didn’t have tbb installed and had to do that as well
I’m getting this error and
metaphlan —install
does not solve it. I do have the latest versions of metaphlan and humann
I am also running into this error. Just installed today
It might be a fault of the new database.
I managed to trick metaphlan into downloading an earlier version, by editing the mpa_latest file in the databases folder. we’ll see how that goes
no luck at all
Like @Massoud_Maher, for me this error was due to the lack of a particular threading library for C++, called tbb2. I think Bowtie needs it. I couldn’t find tbb2 on Conda. I’m in an Ubuntu environment, so I installed it outside of conda with apt:
sudo apt-get install libtbb2
Hello biobakery community,
I used alpha version of humann3 and an older version of MetaPhlAn it worked.
However, I installed the newest version of both, installed the most up to date chocoplan, uniref databases, it shows the same error message.
Error message returned from metaphlan :
No MetaPhlAn BowTie2 database found (–index option)!
Expecting location bowtie2db
Is there a way to use the newer version and still be able to run?
Thanks!!
Yike
Hi!
Just wanted to chime in and say that im getting the same error as @YikeShen. Im using humann v3.0.1 and metaphlan 3.0.14.
Running metaphlan --install returns “The database is installed”. I checked the metaphlan_databases folder in my metaphlan installation on conda and the following files are there, so it looks ok:
README.txt
mpa_latest
mpa_v30_CHOCOPhlAn_201901.1.bt2.tmp
mpa_v30_CHOCOPhlAn_201901.2.bt2.tmp
mpa_v30_CHOCOPhlAn_201901.3.bt2.tmp
mpa_v30_CHOCOPhlAn_201901.4.bt2.tmp
mpa_v30_CHOCOPhlAn_201901.76.sa
mpa_v30_CHOCOPhlAn_201901.77.sa
mpa_v30_CHOCOPhlAn_201901.78.sa
mpa_v30_CHOCOPhlAn_201901.79.sa
mpa_v30_CHOCOPhlAn_201901.80.sa
mpa_v30_CHOCOPhlAn_201901.81.sa
mpa_v30_CHOCOPhlAn_201901.82.sa
mpa_v30_CHOCOPhlAn_201901.83.sa
mpa_v30_CHOCOPhlAn_201901.84.sa
mpa_v30_CHOCOPhlAn_201901.85.sa
mpa_v30_CHOCOPhlAn_201901.86.sa
mpa_v30_CHOCOPhlAn_201901.87.sa
mpa_v30_CHOCOPhlAn_201901.88.sa
mpa_v30_CHOCOPhlAn_201901.89.sa
mpa_v30_CHOCOPhlAn_201901.90.sa
mpa_v30_CHOCOPhlAn_201901.91.sa
mpa_v30_CHOCOPhlAn_201901.92.sa
mpa_v30_CHOCOPhlAn_201901.93.sa
mpa_v30_CHOCOPhlAn_201901.94.sa
mpa_v30_CHOCOPhlAn_201901.95.sa
mpa_v30_CHOCOPhlAn_201901.96.sa
mpa_v30_CHOCOPhlAn_201901.fna
mpa_v30_CHOCOPhlAn_201901.fna.bz2
mpa_v30_CHOCOPhlAn_201901.md5
mpa_v30_CHOCOPhlAn_201901.pkl
mpa_v30_CHOCOPhlAn_201901.tar
Ive tried manually specifying the --index and --bowtie2db paths in --metaphlan-options but it still returns the same error messsage:
Error message returned from metaphlan :
No MetaPhlAn BowTie2 database found (–index option)!
Expecting location bowtie2db
The exact call i used is the following:
humann --metaphlan-options "-t rel_ab --index mpa_v30_CHOCOPhlAn_201901 --bowtie2db /priv/c3-sysbio/sysbio/LeaversData_Omics/MetabogenMelanoma/ici_env/lib/python3.9/site-packages/metaphlan/metaphlan_databases" --input /priv/c3-sysbio/sysbio/LeaversData_Omics/MetabogenMelanoma/fastq_filt_cat/D9_FDPL210411324-1a_HWWWJDSX2_L3_cat.fq.gz --output /priv/c3-sysbio/sysbio/LeaversData_Omics/MetabogenMelanoma/test_out --threads 18
Best,
Angelo
I looked through the metaphlan.py source code now and it looks like the database files i have are incorrect. The first 4 should end with .bt2 and not .bt2.tmp, also the rev1.bt2 and rev2.bt2 are missing. Gonna see if i can re-download these somehow.
EDIT:
Managed to fix it now! simply re-downloaded the databases and re-built the bowtie2 index in a separate folder using metaphlan --install --bowtie2db /path/to/mpa_dbs
I managed to run the humann_test function a few days ago without any problems, using just the installation instructions on your website (humann3 – The Huttenhower Lab). Also, just runnning metaphlan --install returns that the databases have already been installed, even though in my case they still had the .tmp extension as well. Might be good to add an extra check in these functions in a future release.
For any of u having similar issues, maybe double check that the metaphlan_databases folder in your conda installation contains the following files:
mpa_v30_CHOCOPhlAn_201901.1.bt2
mpa_v30_CHOCOPhlAn_201901.2.bt2
mpa_v30_CHOCOPhlAn_201901.3.bt2
mpa_v30_CHOCOPhlAn_201901.4.bt2
mpa_v30_CHOCOPhlAn_201901.rev1.bt2
mpa_v30_CHOCOPhlAn_201901.rev2.bt2
mpa_v30_CHOCOPhlAn_201901.fna.bz2
mpa_v30_CHOCOPhlAn_201901.md5
mpa_v30_CHOCOPhlAn_201901.pkl
mpa_v30_CHOCOPhlAn_201901.tar
Hi everyone,
I am running HUMAnN3 on an ssh client and dealing with the same error when I run the demo:
Error message returned from metaphlan :
No MetaPhlAn BowTie2 database found (–index option)!
Expecting location bowtie2db
I have updated or tried updating humann, metaphlan, bowtie2, and tbb, and the issue still occurs when I have the latest version of all of these packages.
I attempted @angelolimeta’s solution to re-download the databases and rebuild the bowtie2 index using metaphlan --install --bowtie2db mpa_dbs and that partially failed (therefore not resolving the issue), with this error message:
Building Bowtie2 indexes
Fatal error running ‘bowtie2-build --quiet --threads 4 -f mpa_dbs/mpa_v30_CHOCOPhlAn_201901.fna mpa_dbs/mpa_v30_CHOCOPhlAn_201901’
Error message: ‘Command ‘[‘bowtie2-build’, ‘–quiet’, ‘–threads’, ‘4’, ‘-f’, ‘mpa_dbs/mpa_v30_CHOCOPhlAn_201901.fna’, ‘mpa_dbs/mpa_v30_CHOCOPhlAn_201901’]’ returned non-zero exit status 247.’
Any assistance would be very appreciated!
I have encountered the same error and find that every time I try to manually download the databases, I get different results. Sometimes the mpa_v30_CHOCOPhlAn_201901.rev.1.bt2/ mpa_v30_CHOCOPhlAn_201901.rev.2.bt2 files are just not included. Sometimes one or more of the file sizes are zero. I have followed another thread (Humann 3 bowtie error - #9 by nasnuvens) that suggested deleting the .bt2 files and manually rebuilding the index. This seemed to work for me but I am not sure if the files are complete.
For someone who has successfully used these databases: what are the final sizes of each file in the metaphlan_databases directory?
Hello,
I had the same issue reported here. I have installed humann3 using a humann3.yml
file. I git clone the humann3 from github and tryed to run the test samples within the ~/humann/examples
folder. I used this command:
humann --input demo.fastq.gz --output demo_fastq --threads 16
and I got this output
Creating output directory: /panfs/roc/groups/11/noyes046/diazo005/humann/examples/demo_fastq
Output files will be written to: /panfs/roc/groups/11/noyes046/diazo005/humann/examples/demo_fastq
Decompressing gzipped file ...
Running metaphlan ........
CRITICAL ERROR: Error executing: /home/noyes046/diazo005/.conda/envs/humann3/bin/metaphlan /panfs/roc/groups/11/
noyes046/diazo005/humann/examples/demo_fastq/demo_humann_temp/tmpp_ptudn3/tmpzbt96zmj -t rel_ab -o /panfs/roc/gr
oups/11/noyes046/diazo005/humann/examples/demo_fastq/demo_humann_temp/demo_metaphlan_bugs_list.tsv --input_type
fastq --bowtie2out /panfs/roc/groups/11/noyes046/diazo005/humann/examples/demo_fastq/demo_humann_temp/demo_metap
hlan_bowtie2.txt --nproc 16
Error message returned from metaphlan :
No MetaPhlAn BowTie2 database found (--index option)!
Expecting location bowtie2db
Exiting...
Then I just looked in several forums and ended up doing a fresh installation of the new release of Humann3, as indicated in the release note: Announcing MetaPhlAn 3.1 and HUMAnN 3.1
The following steps are detailed here (humann3 – The Huttenhower Lab), briefly:
conda create --name biobakery3 python=3.7
conda activate biobakery3
conda config --add channels defaults
conda config --add channels bioconda
conda config --add channels conda-forge
conda config --add channels biobakery
conda install humann -c biobakery
humann_test
cd ~/humann/examples
humann --input demo.fastq.gz --output demo_fastq --threads 16
Creating output directory: /panfs/roc/groups/11/noyes046/diazo005/humann/examples/demo_fastq
Output files will be written to: /panfs/roc/groups/11/noyes046/diazo005/humann/examples/demo_fastq
Decompressing gzipped file ...
Running metaphlan ........
Found t__SGB1815 : 53.72% of mapped reads ( s__Bacteroides_dorei,s__Phocaeicola_vulgatus,s__Bacteroides_v$
lgatus,s__Bacteroides_sp_9_1_42FAA,s__Bacteroides_sp_3_1_33FAA,s__Bacteroides_sp_NMBE5,s__Phocaeicola_dor$
i,s__Bacteroidaceae_bacterium,g__Phocaeicola.s__Phocaeicola_vulgatus,g__Bacteroides.s__Bacteroides_sp_9_1$
42FAA,g__Bacteroides.s__Bacteroides_sp_3_1_33FAA,g__Bacteroides.s__Bacteroides_sp_NMBE5,g__Bacteroidaceae$
unclassified.s__Bacteroidaceae_bacterium )
Found t__SGB1814 : 46.28% of mapped reads ( s__Bacteroides_vulgatus,s__Bacteroides_dorei,s__Phocaeicola_d$
rei,s__Bacteroides_sp_3_1_33FAA,s__Bacteroides_sp_4_3_47FAA,s__Bacteroides_sp_3_1_40A,s__Bacteroides_sp_A$
18_9,s__Bacteroides_sp_AM23_18,s__Bacteroides_sp_AF39_10AT,s__Bacteroides_sp_AF32_15BH,s__Bacteroides_sp_$
F25_18,s__Bacteroides_sp_AF16_29,s__Bacteroides_sp_AM28_6,s__Bacteroides_sp_AM27_13,s__Bacteroides_sp_AM2$
_11,s__Bacteroides_sp_AF15_23LB,s__Bacteroides_sp_AF17_1,g__Phocaeicola.s__Phocaeicola_dorei,g__Bacteroid$
s.s__Bacteroides_sp_3_1_33FAA,g__Bacteroides.s__Bacteroides_sp_4_3_47FAA,g__Bacteroides.s__Bacteroides_sp$
3_1_40A,g__Bacteroides.s__Bacteroides_sp_AM18_9,g__Bacteroides.s__Bacteroides_sp_AM23_18,g__Bacteroides.s_
_Bacteroides_sp_AF39_10AT,g__Bacteroides.s__Bacteroides_sp_AF32_15BH,g__Bacteroides.s__Bacteroides_sp_AF25
_18,g__Bacteroides.s__Bacteroides_sp_AF16_29,g__Bacteroides.s__Bacteroides_sp_AM28_6,g__Bacteroides.s__Bac
teroides_sp_AM27_13,g__Bacteroides.s__Bacteroides_sp_AM26_11,g__Bacteroides.s__Bacteroides_sp_AF15_23LB,g_
_Bacteroides.s__Bacteroides_sp_AF17_1 )
Total species selected from prescreen: 47
Selected species explain 100.00% of predicted community composition
Creating custom ChocoPhlAn database ........
Running bowtie2-build ........ [870/1996]
Running bowtie2 ........
Total bugs from nucleotide alignment: 2
g__Bacteroides.s__Bacteroides_dorei: 1270 hits
g__Bacteroides.s__Bacteroides_vulgatus: 1335 hits
Total gene families from nucleotide alignment: 572
Unaligned reads after nucleotide alignment: 87.5952380952 %
Running diamond ........
Aligning to reference database: uniref90_demo_prots_v201901b.dmnd
Total bugs after translated alignment: 3
g__Bacteroides.s__Bacteroides_dorei: 1270 hits
g__Bacteroides.s__Bacteroides_vulgatus: 1335 hits
unclassified: 1017 hits
Total gene families after translated alignment: 615
Unaligned reads after translated alignment: 82.8857142857 %
Computing gene families ...
Computing pathways abundance and coverage ...
Output files created:
/panfs/roc/groups/11/noyes046/diazo005/humann/examples/demo_fastq/demo_genefamilies.tsv
/panfs/roc/groups/11/noyes046/diazo005/humann/examples/demo_fastq/demo_pathabundance.tsv
/panfs/roc/groups/11/noyes046/diazo005/humann/examples/demo_fastq/demo_pathcoverage.tsv
I obtained the 3 output files from the tutorial. So, suggestion: Download the new Humann 3.1 (with the new Metaphlan release).
Gerardo R Diaz
CVM-UMN
I got the same problem yesterday. I solved it by installing specific versions of metaphlan.
The version humann 3.0.0.alpha.3 & metaphlan 3.0.9 would work perfectly.
what’s the command of installing specific versions of metaphlan?
pip install metaphlan==[here is version string]
Such as, pip install metaphlan==3.0.9
>=, <=, vice versa.
I am also stuck on this step. I followed the recommended installation
conda create --name biobakery3 python=3.7
conda config --add channels defaults
conda config --add channels bioconda
conda config --add channels conda-forge
conda config --add channels biobakery
conda install humann -c biobakery```
I then downloaded all the databases:
humann_databases --download chocophlan full /scratch4/jsuez1/week12/chocophlan --update-config yes
humann_databases --download uniref uniref90_diamond/scratch4/jsuez1/week12/diamond --update-config yes
humann_databases --download utility_mapping full /scratch4/jsuez1/week12/diamond --update-config yes
Then when I try to run either a sample of mine or the demo.fastq.gz, i get this error:
Running metaphlan ........
CRITICAL ERROR: Error executing: /home/hhallow1/.conda/envs/biobakery3/bin/metaphlan /scratch4/jsuez1/week12_virome/sample_results/demo_humann_temp/tmpm6r5uqir/tmpe2u0gnbk -t rel_ab -o /scratch4/jsuez1/week12_virome/sample_results/demo_humann_temp/demo_metaphlan_bugs_list.tsv --input_type fastq --bowtie2out /scratch4/jsuez1/week12_virome/sample_results/demo_humann_temp/demo_metaphlan_bowtie2.txt
Error message returned from metaphlan :
No MetaPhlAn BowTie2 database found (--index option)!
Expecting location bowtie2db
Exiting...
Ive tried re-installing the databases, resinstalling humann completely, nothing is working. After some digging, it seems that when i download the database using
humann_databases --download chocophlan full /scratch4/jsuez1/week12/chocophlan --update-config yes
The file becomes unzipped. I tried manually combining all files into one .fna by doing this
cat chocophlan/* > mpa_v30_CHOCOPhlAn_201901.fna
then using bowtie-build to build my index
bowtie2-build mpa_v30_CHOCOPhlAn_201901.fna mpa_v30_CHOCOPhlAn_201901
I got about 1000 warnings from bowtie that each reference was empty, i.e contains all Ns rather than ATCG, before i killed the job
So my question is, why is the database unzipping when i download it? what is the correct why to cat these files together to be indexed?
I am running Metaphlan 4.0.2 and Humann 3.6
*I should note that i was able to run Humann before with no problem but lost my install because we started using a new cluster.
Thanks!
Hello!
I used anaconda2 to install humann2 for analysis, but when running, I suddenly reported an error
(CRITICAL ERROR: Error executing: /home/hlli/anaconda2/envs/metagenome_env/bin/metaphlan2.py /home/hlli/AUC/temp/concat/ERR688551.fq -t rel_ab -o /home/hlli/AUC/temp/humann2/ERR688551_humann2_temp/ERR688551_metaphlan_bugs_list.tsv --input_type multifastq --bowtie2out /home/hlli/AUC/temp/humann2/ERR688551_humann2_temp/ERR688551_metaphlan_bowtie2.txt --nproc 8
Error message returned from metaphlan2.py :
No MetaPhlAn BowTie2 database found [–bowtie2db option]! (or wrong path provided).
Expecting location ${mpa_dir}/db_v20/map_v20_m200
Exiting…
)
I have downloaded related related databases and placed them in the specified path, but the same problem still occurs
I would like to ask the professor what is the best way to solve this problem
Can you try running the MetaPhlAn command external to HUMAnN? If you’re manually specifying an index path (or expecting to) HUMAnN won’t have that information. You can pass additional information to MetaPhlAn within the HUMAnN CLI using the --metaphlan-options
flag.