Error: the version of diamond in humann3

Microbial community profiling HUMAnN
1

Hi Francesco,

I try the version of diamond from 0.9.23 to 0.9.32, but some error always showed:

(1) diamond version <= 0.9.23:

(py37) [zhangwenping@localhost try_humann]$ humann --input /data/liying_metagenome/clean_data_liying/SRR5130527_paired.1.fastq --output try_SRR5130527_paired.1 --diamond ‘/data/xmjd/biosoft/’ --metaphlan-options ‘-x mpa_v30_CHOCOPhlAn_201901 --bowtie2db /data/xmjd/miniconda2/envs/py37/lib/python3.7/site-packages/metaphlan/metaphlan_databases/ --read_min_len 49’
Output files will be written to: /data/try_humann/try_SRR5130527_paired.1
Removing spaces from identifiers in input file …

WARNING: Can not call software version for bowtie2

CRITICAL ERROR: Please update diamond from version 0.9.23 to version 0.9.24

(2): diamond version: 0.9.24

07/31/2020 01:48:57 PM - humann.utilities - CRITICAL: Error executing: /data/xmjd/miniconda2/envs/py37/bin/diamond blastx --query /data/try_humann/try_SRR5130527_paired.1/SRR5130527_paired.1_humann_temp/SRR5130527_paired.1_bowtie2_unaligned.fa --evalue 1.0 --threads 1 --top 1 --sensitive --outfmt 6 --db /data/xmjd/biobakery_workflows_databases/humann/uniref/uniref90_filtered_201901 --out /data/try_humann/try_SRR5130527_paired.1/SRR5130527_paired.1_humann_temp/tmpa6l35xe1/diamond_m8_85rjcnpu --tmpdir /data/try_humann/try_SRR5130527_paired.1/SRR5130527_paired.1_humann_temp/tmpa6l35xe1

Error message returned from diamond :
diamond v0.9.24.125 | by Benjamin Buchfink buchfink@gmail.com
Licensed under the GNU GPL https://www.gnu.org/licenses/gpl.txt
Check http://github.com/bbuchfink/diamond for updates.

#CPU threads: 1
Scoring parameters: (Matrix=BLOSUM62 Lambda=0.267 K=0.041 Penalties=11/1)
Temporary directory: /data/try_humann/try_SRR5130527_paired.1/SRR5130527_paired.1_humann_temp/tmpa6l35xe1
Opening the database… [0.000337s]
Error: Database was built with a different version of Diamond and is incompatible.

(3) diamond version from 0.9.25 to 0.9.30: same error like the following:

(py37) [zhangwenping@localhost try_humann]$ humann --input /data/liying_metagenome/clean_data_liying/SRR5130527_paired.1.fastq --output try_SRR5130527_paired.1 --diamond ‘/data/xmjd/biosoft/’ --metaphlan-options ‘-x mpa_v30_CHOCOPhlAn_201901 --bowtie2db /data/xmjd/miniconda2/envs/py37/lib/python3.7/site-packages/metaphlan/metaphlan_databases/ --read_min_len 49’
Output files will be written to: /data/try_humann/try_SRR5130527_paired.1
Removing spaces from identifiers in input file …

WARNING: Can not call software version for bowtie2

CRITICAL ERROR: Can not call software version for diamond

(py37) [zhangwenping@localhost try_humann]$ /data/xmjd/biosoft/diamond --version
diamond v0.9.27.128 | by Benjamin Buchfink buchfink@gmail.com
Licensed under the GNU GPL https://www.gnu.org/licenses/gpl.txt
Check http://github.com/bbuchfink/diamond for updates.

diamond version 0.9.27

(5) diamond version > = 0.9.31:

CRITICAL ERROR: Error executing: /data/xmjd/biosoft/diamond blastx --query /data/try_humann/try_SRR5130527_paired.1/SRR5130527_paired.1_humann_temp/SRR5130527_paired.1_bowtie2_unaligned.fa --evalue 1.0 --threads 1 --top 1 --sensitive --outfmt 6 --db /data/xmjd/biobakery_workflows_databases/humann/uniref/uniref90_filtered_201901 --out /data/try_humann/try_SRR5130527_paired.1/SRR5130527_paired.1_humann_temp/tmp2318ing3/diamond_m8_gezv43ru --tmpdir /data/try_humann/try_SRR5130527_paired.1/SRR5130527_paired.1_humann_temp/tmp2318ing3

Error message returned from diamond :
diamond v0.9.31.132 © Max Planck Society for the Advancement of Science
Documentation, support and updates available at http://www.diamondsearch.org

#CPU threads: 1
Scoring parameters: (Matrix=BLOSUM62 Lambda=0.267 K=0.041 Penalties=11/1)
Temporary directory: /data/try_humann/try_SRR5130527_paired.1/SRR5130527_paired.1_humann_temp/tmp2318ing3
Opening the database… [0s]
Error: Database was built with an older version of Diamond and is incompatible.

How can I fix the error?

wenping

1 Like
2

Can you inspect the database using diamond getseq -d /data/xmjd/biobakery_workflows_databases/humann/uniref/uniref90_filtered_201901 ? Have you by any chance renamed the database file?

3

Hi, Francesco,

(py37) [zhangwenping@localhost ~] diamond getseq -d /data/xmjd/biobakery_workflows_databases/humann/uniref/uniref90_filtered_201901 Error: Database was built with a different version of Diamond and is incompatible. (py37) [zhangwenping@localhost ~] diamond --version
diamond version 0.9.24
(py37) [zhangwenping@localhost ~] cd /data/xmjd/biobakery_workflows_databases/humann/uniref/ (py37) [zhangwenping@localhost uniref] ls -all
total 49587860
drwxrwxr-x 2 zhangwenping zhangwenping 156 Jul 20 14:52 .
drwxrwxr-x 5 zhangwenping zhangwenping 77 Jul 20 15:04 …
-rw-rw-r-- 1 zhangwenping zhangwenping 12405421206 Jun 12 2019 uniref50_201901.dmnd
-rw-rw-r-- 1 zhangwenping zhangwenping 462767146 Jun 19 2019 uniref50_filtered_201901.dmnd
-rw-rw-r-- 1 zhangwenping zhangwenping 36332007348 Jun 12 2019 uniref90_201901.dmnd
-rw-rw-r-- 1 zhangwenping zhangwenping 1577764898 Jun 19 2019 uniref90_filtered_201901.dmnd
(py37) [zhangwenping@localhost uniref]$

Can you tell me how to fix the error?
wenping

4

@lauren.j.mciver It seems that the UniRef90 EC filtered database was built using version 0.8. In fact,

diamond-0.8.38/diamond getseq -d uniref90_201901.dmnd | head -n1

>UniRef90_A0A1I2BX06|687 Cluster: CRISPR-associated exonuclease Cas4

MDGDEDYLPLSALQHYLYCPRQCALIHVEQQWAESRHTAEGRVLHEQADKPRGERRRGVRTITALPLAHAGLRLGGIADVVELHDGPDGEQAFPVEYKRGRPKAHRADEVQLCSQALCLEDMLDQPVPQGALFYGATRRRTDVVFDAELRRLTLDTIAATRAMLDAGITPTAEYSARRCDACSLIDLCQPRLLQRRTGVNAWLARQLKSDRNDNGNHEGGEERGTCDGN

@wenping in the meantime, you could extract all the db sequences using diamond 0.8.x and rebuild the database with version 0.9.24

5

Hi Francesco,

Thank you very much for your help. I found uniref90_filtered_201901.dmnd builted with diamond 0.8.38 and rebuilt the database with version 0.9.24. It works.

wenping

6

Hi @fbeghini, Thank you for the catch and ping. I can confirm the UniRef90 EC filtered database v201901 is formatted for diamond v0.8.22 (the prior diamond that we had been using, db version 0) and it does not work with the newer diamond v0.9.24 (db version 2). I checked the UniRef50 EC filtered database v201901 and it looks like it is compatible with diamond v0.9.24 (db version 2). I have rebuilt and posted the UniRef90 filtered database v201901 with diamond v0.9.24.

Hi @wenping, Thank you for your post and sorry for the confusion. Thank you for rebuilding the database on your own. The updated database (with diamond v0.9.24) has been posted so others should not come across the same error.

Thank you,
Lauren

7

Hi Lauren,

While running humann3 I am getting following error.
CRITICAL ERROR: Please update diamond from version 0.9.24 to version 0.9.36

I have checked the uniref database and it says database was built with different version of diamond
diamond getseq -d uniref90_201901b_full.dmnd

Error: Database was built with a different version of Diamond and is incompatible.

I have installed latest version of humann3
conda activate biobakery3
conda install humann -c biobakery

Can you please suggest how to update diamond version.

2 Likes
8

Edit: Solved the problem by conda install -c bioconda diamond=0.9.36

I also recently tried to run humann3 with the same error here after installing using these direction. Here is the log file:

06/10/2021 02:44:43 PM - humann.humann - INFO: Running humann v3.0.0
06/10/2021 02:44:43 PM - humann.humann - INFO: Output files will be written to: /dfs/ROOTS/Myrold_Lab/Chris/dirt/humann/NI-19-A
06/10/2021 02:44:43 PM - humann.humann - INFO: Writing temp files to directory: /dfs/ROOTS/Myrold_Lab/Chris/dirt/humann/NI-19-A/NI-19-A_filtered_humann_temp
06/10/2021 02:44:43 PM - humann.utilities - INFO: File ( /dfs/ROOTS/Myrold_Lab/Chris/dirt/filtered/NI-19-A_filtered.fastq.gz ) is of format:  fastq.gz
06/10/2021 02:44:43 PM - humann.utilities - INFO: Decompressing gzipped file ...
06/10/2021 02:56:32 PM - humann.humann - INFO: Removing spaces from identifiers in input file
06/10/2021 03:13:34 PM - humann.utilities - DEBUG: Check software, metaphlan, for required version, 3.0
06/10/2021 03:13:35 PM - humann.utilities - INFO: Using metaphlan version 3.0
06/10/2021 03:13:35 PM - humann.utilities - DEBUG: Check software, bowtie2, for required version, 2.2
06/10/2021 03:13:35 PM - humann.utilities - WARNING: Can not call software version for bowtie2
06/10/2021 03:13:35 PM - humann.utilities - INFO: Using bowtie2 version UNK
06/10/2021 03:13:35 PM - humann.humann - INFO: Search mode set to uniref90 because a uniref90 translated search database is selected
06/10/2021 03:13:35 PM - humann.utilities - DEBUG: Check software, diamond, for required version, 0.9.36
06/10/2021 03:13:35 PM - humann.utilities - CRITICAL: Please update diamond from version 0.9.24 to version 0.9.36
1 Like
9

Hi @cjburgess , Thank you for the post.

Hi @ashoks773 , Sorry for my slow response. Please post again if you are still having issues.

Thank you both,
Lauren

10

Thanks! It really helped

11

Hi,

I’m running into a similar problem.
I installed humann3 using
miniconda/bin/conda create -n humann -y -c bioconda humann

I used the bioconda channel bc the biobakery channel was not working on my HPC server.

humann and metaphlan installed fine, but the version of diamond automatically installed was 0.9.24.

I then installed diamond from source using:

tar -xvf diamond-linux64.tar.gz
./diamond help | head -n 1 
alias diamond='./diamond'

and exported it as an alias.

then I ran the test like this:

[ptran5@build4000 dir_978535]$ humann --input data/demo.fastq.gz --output demo_fastq --threads 5
Output files will be written to: /var/lib/condor/execute/slot2/dir_978535/demo_fastq
Decompressing gzipped file ...

CRITICAL ERROR: Please update diamond from version 0.9.24 to version 0.9.36

Also what do you make the the diamond v0.9.36 that says " WARNING: This version contains a serious bug that can cause incomplete results in blastx mode. Using it is not recommended." ?

Some details:

[ptran5@build4000 dir_978535]$ lscpu | egrep 'Model name|Socket|Thread|NUMA|CPU\(s\)'
CPU(s):              256
On-line CPU(s) list: 0-255
Thread(s) per core:  2
Socket(s):           2
NUMA node(s):        8
Model name:          AMD EPYC 7763 64-Core Processor
NUMA node0 CPU(s):   0-15,128-143
NUMA node1 CPU(s):   16-31,144-159
NUMA node2 CPU(s):   32-47,160-175
NUMA node3 CPU(s):   48-63,176-191
NUMA node4 CPU(s):   64-79,192-207
NUMA node5 CPU(s):   80-95,208-223
NUMA node6 CPU(s):   96-111,224-239
NUMA node7 CPU(s):   112-127,240-255
[ptran5@build4000 dir_978535]$ cat /etc/*release
CentOS Stream release 8
NAME="CentOS Stream"
VERSION="8"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="8"
PLATFORM_ID="platform:el8"
PRETTY_NAME="CentOS Stream 8"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:8"
HOME_URL="https://centos.org/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
CentOS Stream release 8
CentOS Stream release 8

Thank you!

12

I would install from the biobakery channel rather than bioconda. The error you note from DIAMOND was something we discovered in the development of HUMAnN 3. As of HUMAnN v3.6 we will pull and use a version of DIAMOND v2.x where that error has been corrected.