Error Message from Diamond in humann v3.8

Microbial community profiling HUMAnN
1

Hello,
Iam trying to execute humann v3.8 for particular sample with input around 34 million. The sample has went through kneadata and Metaphlan4 separately. So the input for humann v3.8 is the fastq file from Kneaddata and taxonomy profile from metaphlan4. Tried this approach for various samples separately with resources as provided below. But this sample is an exception. FYI Other samples input reads were varying between 15-25 million input reads
Resources:
Memory - 32 GB
Disk Space - 2TB
Threads - 8

Encountered the following message despite running for few hours

CRITICAL ERROR: Error executing: /home/ubuntu/miniconda3/envs/metagenome/bin/diamond blastx --query /home/ubuntu/metagenome/output/poyet/gut-microbiome-s3/metagenome/output/POYET_1/SRR9224129/SRR9224129_humann_temp/SRR9224129_bowtie2_unaligned.fa --evalue 1.0 --threads 8 --top 1 --outfmt 6 --db /home/ubuntu/miniconda3/envs/metagenome/lib/python3.10/site-packages/humann/data/uniref/uniref90_201901b_full --out /home/ubuntu/metagenome/output/poyet/gut-microbiome-s3/metagenome/output/POYET_1/SRR9224129/SRR9224129_humann_temp/tmpp_a47e24/diamond_m8_2ilk21vd --tmpdir /home/ubuntu/metagenome/output/poyet/gut-microbiome-s3/metagenome/output/POYET_1/SRR9224129/SRR9224129_humann_temp/tmpp_a47e24

Error message returned from diamond :
diamond v2.1.8.162 (C) Max Planck Society for the Advancement of Science, Benjamin Buchfink, University of Tuebingen
Documentation, support and updates available at http://www.diamondsearch.org
Please cite: Sensitive protein alignments at tree-of-life scale using DIAMOND | Nature Methods Nature Methods (2021)

Error Message from Diamond step without complete information.

Have attached the complete log of the reference. Any help on fixing this would be much appreciated.
SRR9224129.txt (44.2 KB)

Regards
Karthik

2

If it’s just this one largest sample that’s failing my guess is that you’re hitting either a memory or storage limit. Unfortunately the error message doesn’t tell us much about why it failed. If it’s a memory issue, you can pass additional arguments to DIAMOND to tell it to work longer within a smaller memory footprint, and that might help to resolve it. HUMAnN includes a command-line parameter that allows you to pass additional arguments to DIAMOND (replacing XYZ in the command below).

--diamond-options "--top 1 --outfmt 6 XYZ"

You would need to consult the DIAMOND manual for recommended settings for your environment (I believe the parameters relate to “block” and “chunk” sizes, but I don’t have a ton of experience tuning them).

1 Like
3

Same error. As Franzosa says, it is a matter of memory. When I submitted the job to the supercomputer, the problem was solved!

Error message returned from diamond :
diamond v2.1.8.162 (C) Max Planck Society for the Advancement of Science, Benjamin Buchfink, University of Tuebingen
Documentation, support and updates available at http://www.diamondsearch.org
Please cite: http://dx.doi.org/10.1038/s41592-021-01101-x Nature Methods (2021)
4

Glad you were able to figure it out!