Dear, I was trying to run metaphlan3.0 and humann3.0 within the biobakery workflows wmgx pipeline.
Using the following line of code:
biobakery_workflows wmgx --input /Projects1//humann_input_subset --output /Projects1//humann_output_subset6 --input-extension fastq --bypass-quality-control --taxonomic-profiling-options “–bowtie2db /media//biobakery_3/metaphlan3/ --stat_q 0.1" --functional-profiling-options "–nucleotide-subject-coverage-threshold 0 --translated-subject-coverage-threshold 0 --pathways-database /media//biobakery_workflows/wmgx/humann/pathways/metacyc_reactions_level4ec_only.uniref.bz2,/media/****/biobakery_workflows/wmgx/humann/pathways/metacyc_pathways_structured_filtered_v24” --remove-intermediate-output --bypass-strain-profiling --threads 20
Metaplan3.0 ran successfully.
However, I ran into an error when Humann3.0 started which stated the following: “The MetaPhlAn2 taxonomic profile provided was not generated with the database version v30 . Please update your version of MetaPhlAn2 to v3.0.” However, when looking at the log file, the used versions of both metaphlan and humann were both 3.0. I provided the full error and the log file of one of the files underneath.
The pipeline did ran successfully in the past, but now I am running into some errors. Do you maybe have an idea what the problem could be?
Many thanks in advance!
Best regards,
Britta
Error :
(biobakery3) :/Projects1///Biobakery/wmgx_/$ biobakery_workflows wmgx --input /Projects1///Biobakery/wmgx_/humann_input_subset --output /Projects1//humann_output_subset6 --input-extension fastq --bypass-quality-control --taxonomic-profiling-options "–bowtie2db /media//biobakery_3/metaphlan3/ --stat_q 0.1" --functional-profiling-options "–nucleotide-subject-coverage-threshold 0 --translated-subject-coverage-threshold 0 --pathways-database /media//biobakery_workflows/wmgx/humann/pathways/metacyc_reactions_level4ec_only.uniref.bz2,/media//biobakery_workflows/wmgx/humann/pathways/metacyc_pathways_structured_filtered_v24" --remove-intermediate-output --bypass-strain-profiling --threads 20
(Mar 28 11:49:15) [ 0/25 - 0.00%] **Ready ** Task 3: metaphlan____Sample1
(Mar 28 11:49:15) [ 0/25 - 0.00%] **Started ** Task 3: metaphlan____Sample1
(Mar 28 11:51:49) [ 1/25 - 4.00%] Completed Task 3: metaphlan____Sample1
(Mar 28 11:51:49) [ 1/25 - 4.00%] **Ready ** Task 9: humann____Sample1
(Mar 28 11:51:49) [ 1/25 - 4.00%] **Started ** Task 9: humann____Sample1
(Mar 28 11:52:18) [ 2/25 - 8.00%] **Failed ** Task 9: humann____Sample1
(Mar 28 11:52:18) [ 3/25 - 12.00%] **Failed ** Task 12: humann_regroup_UniRef2EC____Sample1
(Mar 28 11:52:18) [ 4/25 - 16.00%] **Failed ** Task 19: humann_renorm_ecs_relab____Sample1
(Mar 28 11:52:18) [ 5/25 - 20.00%] **Failed ** Task 17: humann_renorm_genes_relab____Sample1
(Mar 28 11:52:18) [ 6/25 - 24.00%] **Failed ** Task 21: humann_renorm_pathways_relab____Sample1
(Mar 28 11:52:18) [ 6/25 - 24.00%] **Ready ** Task 0: metaphlan____Sample2
(Mar 28 11:52:18) [ 6/25 - 24.00%] **Started ** Task 0: metaphlan____Sample2
(Mar 28 11:54:27) [ 7/25 - 28.00%] Completed Task 0: metaphlan____Sample2
(Mar 28 11:54:27) [ 7/25 - 28.00%] **Ready ** Task 5: metaphlan_join_taxonomic_profiles
(Mar 28 11:54:27) [ 7/25 - 28.00%] **Started ** Task 5: metaphlan_join_taxonomic_profiles
(Mar 28 11:54:27) [ 8/25 - 32.00%] Completed Task 5: metaphlan_join_taxonomic_profiles
(Mar 28 11:54:27) [ 8/25 - 32.00%] **Ready ** Task 6: metaphlan_count_species
(Mar 28 11:54:27) [ 8/25 - 32.00%] **Started ** Task 6: metaphlan_count_species
(Mar 28 11:54:27) [ 9/25 - 36.00%] Completed Task 6: metaphlan_count_species
(Mar 28 11:54:27) [ 9/25 - 36.00%] **Ready ** Task 7: humann____Sample2
(Mar 28 11:54:27) [ 9/25 - 36.00%] **Started ** Task 7: humann____Sample2
(Mar 28 11:54:56) [10/25 - 40.00%] **Failed ** Task 7: humann____Sample2
(Mar 28 11:54:56) [11/25 - 44.00%] **Failed ** Task 10: humann_count_alignments_species
(Mar 28 11:54:56) [12/25 - 48.00%] **Failed ** Task 11: humann_regroup_UniRef2EC____Sample2
(Mar 28 11:54:56) [13/25 - 52.00%] **Failed ** Task 14: humann_join_tables_ecs
(Mar 28 11:54:56) [14/25 - 56.00%] **Failed ** Task 18: humann_renorm_ecs_relab____Sample2
(Mar 28 11:54:56) [15/25 - 60.00%] **Failed ** Task 23: humann_join_tables_ecs_relab
(Mar 28 11:54:56) [16/25 - 64.00%] **Failed ** Task 26: humann_count_features_ecs
(Mar 28 11:54:56) [17/25 - 68.00%] **Failed ** Task 13: humann_join_tables_genefamilies
(Mar 28 11:54:56) [18/25 - 72.00%] **Failed ** Task 15: humann_join_tables_pathabundance
(Mar 28 11:54:56) [19/25 - 76.00%] **Failed ** Task 16: humann_renorm_genes_relab____Sample2
(Mar 28 11:54:56) [20/25 - 80.00%] **Failed ** Task 22: humann_join_tables_genes_relab
(Mar 28 11:54:56) [21/25 - 84.00%] **Failed ** Task 25: humann_count_features_genes
(Mar 28 11:54:56) [22/25 - 88.00%] **Failed ** Task 20: humann_renorm_pathways_relab____Sample2
(Mar 28 11:54:56) [23/25 - 92.00%] **Failed ** Task 24: humann_join_tables_pathways_relab
(Mar 28 11:54:56) [24/25 - 96.00%] **Failed ** Task 27: humann_count_features_pathways
(Mar 28 11:54:56) [25/25 - 100.00%] **Failed ** Task 28: humann_merge_feature_counts
Run Finished
Task 9 failed
Name: humann____Sample1
Original error:
Error executing action 0. Original Exception:
Traceback (most recent call last):
File “/usr/local/bin/miniconda3/envs/biobakery3/lib/python3.7/site-packages/anadama2/runners.py”, line 201, in _run_task_locally
action_func(task)
File “/usr/local/bin/miniconda3/envs/biobakery3/lib/python3.7/site-packages/anadama2/helpers.py”, line 89, in actually_sh
ret = _sh(s, **kwargs)
File “/usr/local/bin/miniconda3/envs/biobakery3/lib/python3.7/site-packages/anadama2/util/init.py”, line 320, in sh
raise ShellException(proc.returncode, msg.format(cmd, ret[0], ret[1]))
anadama2.util.ShellException: [Errno 1] Command `humann --input /Projects1//humann_input_subset/Sample1.fastq --output /Projects1//humann_output_subset6/humann/main --o-log /Projects1//humann_output_subset6/humann/main/Sample1.log --threads 20 --taxonomic-profile /Projects1//humann_output_subset6/metaphlan/main/Sample1_taxonomic_profile.tsv --remove-temp-output --nucleotide-subject-coverage-threshold 0 --translated-subject-coverage-threshold 0 --pathways-database /media//biobakery_workflows/wmgx/humann/pathways/metacyc_reactions_level4ec_only.uniref.bz2,/media//biobakery_workflows/wmgx/humann/pathways/metacyc_pathways_structured_filtered_v24 ’ failed.
Out: b’Output files will be written to: /Projects1//Biobakery/wmgx_****/humann_output_subset6/humann/main\nFound g__Cutibacterium.s__Cutibacterium_acnes : 78.67% of mapped reads\nFound g__Corynebacterium.s__Corynebacterium_otitidis : 12.31% of mapped reads\nFound g__Malassezia.s__Malassezia_globosa : 2.04% of mapped reads\nFound g__Cutibacterium.s__Propionibacterium_namnetense : 1.86% of mapped reads\nFound g__Staphylococcus.s__Staphylococcus_epidermidis : 0.77% of mapped reads\nFound g__Streptococcus.s__Streptococcus_mitis : 0.59% of mapped reads\nFound g__Morococcus.s__Morococcus_cerebrosus : 0.38% of mapped reads\nFound g__Cutibacterium.s__Cutibacterium_granulosum : 0.30% of mapped reads\nFound g__Streptococcus.s__Streptococcus_gordonii : 0.27% of mapped reads\nFound g__Lactococcus.s__Lactococcus_lactis : 0.27% of mapped reads\nFound g__Micrococcus.s__Micrococcus_luteus : 0.26% of mapped reads\nFound g__Dermacoccus.s__Dermacoccus_nishinomiyaensis : 0.24% of mapped reads\nFound g__Micrococcus.s__Micrococcus_aloeverae : 0.22% of mapped reads\nFound g__Actinomyces.s__Actinomyces_odontolyticus : 0.21% of mapped reads\nFound g__Streptococcus.s__Streptococcus_thermophilus : 0.18% of mapped reads\nFound g__Streptococcus.s__Streptococcus_oralis : 0.16% of mapped reads\nFound g__Veillonella.s__Veillonella_parvula : 0.16% of mapped reads\nFound g__Rothia.s__Rothia_dentocariosa : 0.15% of mapped reads\nFound g__Rothia.s__Rothia_mucilaginosa : 0.13% of mapped reads\nFound g__Abiotrophia.s__Abiotrophia_defectiva : 0.11% of mapped reads\nFound g__Staphylococcus.s__Staphylococcus_saccharolyticus : 0.10% of mapped reads\nFound g__Prevotella.s__Prevotella_intermedia : 0.08% of mapped reads\nFound g__Actinomyces.s__Actinomyces_oris : 0.08% of mapped reads\nFound g__Streptococcus.s__Streptococcus_sanguinis : 0.07% of mapped reads\nFound g__Corynebacterium.s__Corynebacterium_pseudogenitalium : 0.05% of mapped reads\nFound g__Haemophilus.s__Haemophilus_influenzae : 0.05% of mapped reads\nFound g__Neisseria.s__Neisseria_subflava : 0.04% of mapped reads\nFound g__Neisseria.s__Neisseria_macacae : 0.03% of mapped reads\nFound g__Streptococcus.s__Streptococcus_sp_A12 : 0.03% of mapped reads\nFound g__Neisseria.s__Neisseria_flavescens : 0.03% of mapped reads\nFound g__Haemophilus.s__Haemophilus_parainfluenzae : 0.03% of mapped reads\nFound g__Porphyromonas.s__Porphyromonas_somerae : 0.02% of mapped reads\nFound g__Moraxella.s__Moraxella_osloensis : 0.02% of mapped reads\nFound g__Neisseria.s__Neisseria_polysaccharea : 0.01% of mapped reads\nFound g__Bradyrhizobium.s__Bradyrhizobium_sp_MOS004 : 0.01% of mapped reads\nFound g__Actinomyces.s__Actinomyces_viscosus : 0.01% of mapped reads\nFound g__Bacteria_unclassified.s__bacterium_HR18 : 0.01% of mapped reads\nFound g__Abiotrophia.s__Abiotrophia_sp_HMSC24B09 : 0.01% of mapped reads\n’
Err: b’\n\nERROR: The MetaPhlAn2 taxonomic profile provided was not generated with the database version v30 . Please update your version of MetaPhlAn2 to v3.0.\n’
Task 12 failed
Name: humann_regroup_UniRef2EC____Sample1
Original error:
Task failed because parent task 9' failed Task 19 failed Name: humann_renorm_ecs_relab____Sample1 Original error: Task failed because parent task 12’ failed
Task 17 failed
Name: humann_renorm_genes_relab____Sample1
Original error:
Task failed because parent task 9' failed Task 21 failed Name: humann_renorm_pathways_relab____Sample1 Original error: Task failed because parent task 9’ failed
Task 7 failed
Name: humann____Sample2
Original error:
Error executing action 0. Original Exception:
Traceback (most recent call last):
File “/usr/local/bin/miniconda3/envs/biobakery3/lib/python3.7/site-packages/anadama2/runners.py”, line 201, in _run_task_locally
action_func(task)
File “/usr/local/bin/miniconda3/envs/biobakery3/lib/python3.7/site-packages/anadama2/helpers.py”, line 89, in actually_sh
ret = _sh(s, **kwargs)
File “/usr/local/bin/miniconda3/envs/biobakery3/lib/python3.7/site-packages/anadama2/util/init.py”, line 320, in sh
raise ShellException(proc.returncode, msg.format(cmd, ret[0], ret[1]))
anadama2.util.ShellException: [Errno 1] Command `humann --input /Projects1//humann_input_subset/Sample2.fastq --output /Projects1//humann_output_subset6/humann/main --o-log /Projects1//humann_output_subset6/humann/main/Sample2.log --threads 20 --taxonomic-profile /Projects1//humann_output_subset6/metaphlan/main/Sample2_taxonomic_profile.tsv --remove-temp-output --nucleotide-subject-coverage-threshold 0 --translated-subject-coverage-threshold 0 --pathways-database /media//biobakery_workflows/wmgx/humann/pathways/metacyc_reactions_level4ec_only.uniref.bz2,/media//biobakery_workflows/wmgx/humann/pathways/metacyc_pathways_structured_filtered_v24 ’ failed.
Out: b’Output files will be written to: /Projects1/****/humann_output_subset6/humann/main\nFound g__Cutibacterium.s__Propionibacterium_namnetense : 51.63% of mapped reads\nFound g__Cutibacterium.s__Cutibacterium_acnes : 43.49% of mapped reads\nFound g__Corynebacterium.s__Corynebacterium_otitidis : 3.03% of mapped reads\nFound g__Morococcus.s__Morococcus_cerebrosus : 1.28% of mapped reads\nFound g__Staphylococcus.s__Staphylococcus_epidermidis : 0.57% of mapped reads\n’
Err: b’\n\nERROR: The MetaPhlAn2 taxonomic profile provided was not generated with the database version v30 . Please update your version of MetaPhlAn2 to v3.0.\n’
Task 10 failed
Name: humann_count_alignments_species
Original error:
Task failed because parent task 9' failed Task 11 failed Name: humann_regroup_UniRef2EC____Sample2 Original error: Task failed because parent task 7’ failed
Task 14 failed
Name: humann_join_tables_ecs
Original error:
Task failed because parent task 11' failed Task 18 failed Name: humann_renorm_ecs_relab____Sample2 Original error: Task failed because parent task 11’ failed
Task 23 failed
Name: humann_join_tables_ecs_relab
Original error:
Task failed because parent task 18' failed Task 26 failed Name: humann_count_features_ecs Original error: Task failed because parent task 23’ failed
Task 13 failed
Name: humann_join_tables_genefamilies
Original error:
Task failed because parent task 9' failed Task 15 failed Name: humann_join_tables_pathabundance Original error: Task failed because parent task 9’ failed
Task 16 failed
Name: humann_renorm_genes_relab____Sample2
Original error:
Task failed because parent task 7' failed Task 22 failed Name: humann_join_tables_genes_relab Original error: Task failed because parent task 16’ failed
Task 25 failed
Name: humann_count_features_genes
Original error:
Task failed because parent task 22' failed Task 20 failed Name: humann_renorm_pathways_relab____Sample2 Original error: Task failed because parent task 7’ failed
Task 24 failed
Name: humann_join_tables_pathways_relab
Original error:
Task failed because parent task 20' failed Task 27 failed Name: humann_count_features_pathways Original error: Task failed because parent task 24’ failed
Task 28 failed
Name: humann_merge_feature_counts
Original error:
Task failed because parent task `25’ failed
Traceback (most recent call last):
File “/usr/local/bin/miniconda3/envs/biobakery3/bin/wmgx.py”, line 184, in
workflow.go()
File “/usr/local/bin/miniconda3/envs/biobakery3/lib/python3.7/site-packages/anadama2/workflow.py”, line 800, in go
self._handle_finished()
File “/usr/local/bin/miniconda3/envs/biobakery3/lib/python3.7/site-packages/anadama2/workflow.py”, line 832, in _handle_finished
raise RunFailed()
anadama2.workflow.RunFailed
Log file of Sample 2:
03/28/2023 11:54:27 AM - humann.humann - INFO: Running humann v3.0.1
03/28/2023 11:54:27 AM - humann.humann - INFO: Output files will be written to: /Projects1//humann_output_subset6/humann/main
03/28/2023 11:54:27 AM - humann.humann - INFO: Writing temp files to directory: /Projects1//humann_output_subset6/humann/main/Sample2_humann_temp_xc2ni_ed
03/28/2023 11:54:27 AM - humann.utilities - INFO: File ( /Projects1/****/humann_input_subset/Sample2.fastq ) is of format: fastq
03/28/2023 11:54:27 AM - humann.utilities - DEBUG: Check software, metaphlan, for required version, 3.0
03/28/2023 11:54:28 AM - humann.utilities - INFO: Using metaphlan version 3.0
03/28/2023 11:54:28 AM - humann.utilities - DEBUG: Check software, bowtie2, for required version, 2.2
03/28/2023 11:54:28 AM - humann.utilities - INFO: Using bowtie2 version 2.4
03/28/2023 11:54:28 AM - humann.humann - INFO: Search mode set to uniref90 because a uniref90 translated search database is selected
03/28/2023 11:54:28 AM - humann.utilities - DEBUG: Check software, diamond, for required version, 0.9.36
03/28/2023 11:54:28 AM - humann.utilities - INFO: Using diamond version 2.0.14
03/28/2023 11:54:28 AM - humann.config - INFO:
Run config settings:
DATABASE SETTINGS
nucleotide database folder = /media//biobakery_workflows/wmgx/humann/chocophlan
protein database folder = /media//biobakery_workflows/wmgx/humann/uniref
pathways database file 1 = /media//biobakery_workflows/wmgx/humann/pathways/metacyc_reactions_level4ec_only.uniref.bz2
pathways database file 2 = /media//biobakery_workflows/wmgx/humann/pathways/metacyc_pathways_structured_filtered_v24
utility mapping database folder = /media/****/biobakery_workflows/wmgx/humann/utility_mapping
RUN MODES
resume = False
verbose = False
bypass prescreen = False
bypass nucleotide index = False
bypass nucleotide search = False
bypass translated search = False
translated search = diamond
threads = 20
SEARCH MODE
search mode = uniref90
nucleotide identity threshold = 0.0
translated identity threshold = 80.0
ALIGNMENT SETTINGS
bowtie2 options = --very-sensitive
diamond options = --top 1 --outfmt 6
evalue threshold = 1.0
prescreen threshold = 0.01
translated subject coverage threshold = 0.0
translated query coverage threshold = 90.0
nucleotide subject coverage threshold = 0.0
nucleotide query coverage threshold = 90.0
PATHWAYS SETTINGS
minpath = on
xipe = off
gap fill = on
INPUT AND OUTPUT FORMATS
input file format = fastq
output file format = tsv
output max decimals = 10
remove stratified output = False
remove column description output = False
log level = DEBUG
03/28/2023 11:54:28 AM - humann.store - DEBUG: Initialize Alignments class instance to minimize memory use
03/28/2023 11:54:28 AM - humann.store - DEBUG: Initialize Reads class instance to minimize memory use
03/28/2023 11:54:55 AM - humann.humann - INFO: Load pathways database part 1: /media//biobakery_workflows/wmgx/humann/pathways/metacyc_reactions_level4ec_only.uniref.bz2
03/28/2023 11:54:55 AM - humann.humann - INFO: Load pathways database part 2: /media//biobakery_workflows/wmgx/humann/pathways/metacyc_pathways_structured_filtered_v24
03/28/2023 11:54:55 AM - humann.search.prescreen - INFO: Found g__Cutibacterium.s__Propionibacterium_namnetense : 51.63% of mapped reads
03/28/2023 11:54:55 AM - humann.search.prescreen - INFO: Found g__Cutibacterium.s__Cutibacterium_acnes : 43.49% of mapped reads
03/28/2023 11:54:55 AM - humann.search.prescreen - INFO: Found g__Corynebacterium.s__Corynebacterium_otitidis : 3.03% of mapped reads
03/28/2023 11:54:55 AM - humann.search.prescreen - INFO: Found g__Morococcus.s__Morococcus_cerebrosus : 1.28% of mapped reads
03/28/2023 11:54:55 AM - humann.search.prescreen - INFO: Found g__Staphylococcus.s__Staphylococcus_epidermidis : 0.57% of mapped reads
03/28/2023 11:54:55 AM - humann.search.prescreen - ERROR: The MetaPhlAn2 taxonomic profile provided was not generated with the database version v30 . Please update your version of MetaPhlAn2 to v3.0.
