MinPath12hmp.py in humann3.6 stops with CRITICAL: Error

Microbial community profiling HUMAnN
1

Dear developers,

Thank you for developing this great tool!

I am currently trying to run human version 3.6. After two days of computation for all samples:

################################

software version:

humann --version
humann v3.6

just in case, package versions for all dependencies:

module list

Currently Loaded Modules:

  1. bowtie2/2.4.2 6) py-bcbio-gff/0.6.9 11) py-matplotlib/3.5.2 16) openssl/1.1.1o 21) py-scipy/1.8.1 26) py-seaborn/0.11.2
  2. zlib/1.2.11 7) py-numpy/1.22.4 12) bcftools/1.14 17) curl/7.83.0 22) py-cmseq/1.0.4 27) py-phylophlan/3.0.3
  3. diamond/2.0.15 8) py-pandas/1.4.2 13) libiconv/1.16 18) python/3.9.12 23) py-dendropy/4.3.0 28) py-requests/2.26.0
  4. py-biom-format/2.1.10 9) htslib/1.14 14) libunistring/0.9.10 19) samtools/1.14 24) py-h5py/3.6.0 29) py-metaphlan/4.0.4
  5. py-biopython/1.78 10) perl/5.34.1 15) libidn2/2.3.0 20) py-pysam/0.18.0 25) py-hclust2/1.0.0 30) py-humann/3.6

######################

I am running humann 3.6 on a set of moderate size (2-4 Gb) .fastq.gz files with metagenome data.
I allocated 61440 memory using slurm:

#!/bin/bash

#SBATCH --nodes=1

#SBATCH --ntasks=1

#SBATCH --cpus-per-task=1

#SBATCH --time=72:00:00

#SBATCH --mem=61440

#SBATCH --job-name=h_10

#SBATCH --mail-type=END

#SBATCH --mail-type=FAIL

#SBATCH --mail-user=NA

module load py-humann/3.6

humann --input-format fastq.gz --remove-temp-output --metaphlan-options “-t rel_ab --bowtie2db /gpfs/space/home/bayazit1/MICROBIOTA/metaphlan4_DB” --nucleotide-database /gpfs/space/home/bayazit1/MICROBIOTA/humann3_6_DB/chocophlan --protein-database /gpfs/space/home/bayazit1/MICROBIOTA/humann3_6_DB/uniref --input /gpfs/space/home/bayazit1/MICROBIOTA/TB/input/K10_kneaddata_joined.fastq.gz --output /gpfs/space/home/bayazit1/MICROBIOTA/TB/humann3_output

#############

After two days of computation, I get only one out of three expected output files, i.e. ‘{filename}_genefamilies.tsv’

Here is a piece of relevant humann log:
…
02/26/2023 01:12:04 AM - humann.utilities - CRITICAL: Error executing: /gpfs/space/software/cluster_software/spack/linux-centos7-x86_64/gcc-9.2.0/python-3.9.12-

ykgigrafumn37bkekcxyn3nsqt7xjj33/bin/python3.9 /gpfs/space/software/cluster_software/spack/linux-centos7-x86_64/gcc-9.2.0/py-humann-3.6-p7n6co5ltoofttebo3tqzb57

b2zqsydt/lib/python3.9/site-packages/humann/quantify/MinPath12hmp.py -any /gpfs/space/home/bayazit1/MICROBIOTA/TB/humann3_output/K9_kneaddata_joined_humann_temp

_4rodco3m/tmpe_5u9oah/tmprh8qu7tn -map /gpfs/space/home/bayazit1/MICROBIOTA/TB/humann3_output/K9_kneaddata_joined_humann_temp_4rodco3m/tmpe_5u9oah/tmpsn5coily -

report /gpfs/space/home/bayazit1/MICROBIOTA/TB/humann3_output/K9_kneaddata_joined_humann_temp_4rodco3m/tmpe_5u9oah/tmpdxxi0s3h -details /gpfs/space/home/bayazit

1/MICROBIOTA/TB/humann3_output/K9_kneaddata_joined_humann_temp_4rodco3m/tmpe_5u9oah/tmppkpzm3cm -mps /gpfs/space/home/bayazit1/MICROBIOTA/TB/humann3_output/K9_k

neaddata_joined_humann_temp_4rodco3m/tmpe_5u9oah/tmpr4qsr0me

02/26/2023 01:12:04 AM - humann.utilities - CRITICAL: TRACEBACK:

Traceback (most recent call last):

File "/gpfs/space/software/cluster_software/spack/linux-centos7-x86_64/gcc-9.2.0/py-humann-3.6-p7n6co5ltoofttebo3tqzb57b2zqsydt/lib/python3.9/site-packages/hu

mann/utilities.py", line 759, in execute_command

p = subprocess.check_call(cmd, stdin=stdin, stdout=stdout, stderr=stderr)

File “/gpfs/space/software/cluster_software/spack/linux-centos7-x86_64/gcc-9.2.0/python-3.9.12-ykgigrafumn37bkekcxyn3nsqt7xjj33/lib/python3.9/subprocess.py”,

line 373, in check_call

raise CalledProcessError(retcode, cmd)

subprocess.CalledProcessError: Command ‘[’/gpfs/space/software/cluster_software/spack/linux-centos7-x86_64/gcc-9.2.0/python-3.9.12-ykgigrafumn37bkekcxyn3nsqt7xj

j33/bin/python3.9’, '/gpfs/space/software/cluster_software/spack/linux-centos7-x86_64/gcc-9.2.0/py-humann-3.6-p7n6co5ltoofttebo3tqzb57b2zqsydt/lib/python3.9/sit

e-packages/humann/quantify/MinPath12hmp.py’, ‘-any’, '/gpfs/space/home/bayazit1/MICROBIOTA/TB/humann3_output/K9_kneaddata_joined_humann_temp_4rodco3m/tmpe_5u9oa

h/tmprh8qu7tn’, ‘-map’, ‘/gpfs/space/home/bayazit1/MICROBIOTA/TB/humann3_output/K9_kneaddata_joined_humann_temp_4rodco3m/tmpe_5u9oah/tmpsn5coily’, ‘-report’, '/

gpfs/space/home/bayazit1/MICROBIOTA/TB/humann3_output/K9_kneaddata_joined_humann_temp_4rodco3m/tmpe_5u9oah/tmpdxxi0s3h’, ‘-details’, '/gpfs/space/home/bayazit1/

MICROBIOTA/TB/humann3_output/K9_kneaddata_joined_humann_temp_4rodco3m/tmpe_5u9oah/tmppkpzm3cm’, ‘-mps’, '/gpfs/space/home/bayazit1/MICROBIOTA/TB/humann3_output/

K9_kneaddata_joined_humann_temp_4rodco3m/tmpe_5u9oah/tmpr4qsr0me’]’ returned non-zero exit status 1.

…

I wonder if this error is caused by our installation or it is a software bug?

2

Hello, Thank you for the detailed post! It looks like the error is during the MinPath call which in turn calls glpsol (part of glpk). If you would try updating your version of glpk it might resolve the issue as I have seen errors with older versions of this tool. You can install HUMAnN with pip adding the option --no-binary :all: and it will install all of the dependencies it needs (including a newer version of glpk).

Thank you,
Lauren

3

Hello Lauren,

Many thanks for looking into this issue. It took a while to test the solution:

So I have asked our HPC people to reinstall, as suggested.
They rebuild the human 3.6 with ‘glpk 5.0’.

Then I rerun the human and just got my results.
I can see that ‘MinPath12hmp.py’ did not complain.

So once again, thank you for your help with this!

4

Great! Glad to hear it is working now!

Lauren