Hi developers,
For some reason diamond is working pretty weird. I have 60 samples and 40 succeeded with version 2.0.X and the remaining 20 kept failing. I made sure memory is enough, wall time is abundant (I ran on a bsub cluster), and disk space is sufficient. But the 19 of the 20 were only able to be finished with version 0.9.X the required minimum version, in several rounds.
And the last sample just couldn’t finish, the log file stops at
12/14/2020 01:51:39 AM - humann.utilities - INFO: Execute command: /home/daia1/anaconda3/envs/py37/bin/diamond blastx --query /home/daia1/my_workdir/samples/CART_275G_humann3_humann_temp_jjnkrikb/CART_275G_humann3_bowtie2_unaligned.fa --evalue 1.0 --threads 16 --top 1 --outfmt 6 --db /home/daia1/my_workdir/ref_db/uniref/uniref/uniref/uniref90_201901 --out /home/daia1/my_workdir/samples/CART_275G_humann3_humann_temp_jjnkrikb/tmpddg7kbe1/diamond_m8_3p62dlmd --tmpdir /home/daia1/my_workdir/samples/CART_275G_humann3_humann_temp_jjnkrikb/tmpddg7kbe1
Do you have any idea why is this?
I think I will switch to usearch just to finish this one
Anqi
Hmm, I’m surprised that any of the jobs were working with DIAMOND 2.0 since the databases we provide were indexed on 0.9, and I don’t think the two are compatible?
I am having the sample problem using diamond 0.9.24
03/31/2021 01:38:32 PM - humann.humann - INFO: Total bugs from nucleotide alignment: 3
03/31/2021 01:38:32 PM - humann.humann - INFO:
g__Bacteroides.s__Bacteroides_dorei: 1243 hits
g__Bacteroides.s__Bacteroides_vulgatus: 939 hits
g__Bacteroides.s__Bacteroides_vulgatus_CAG_6: 470 hits
03/31/2021 01:38:32 PM - humann.humann - INFO: Total gene families from nucleotide alignment: 558
03/31/2021 01:38:32 PM - humann.humann - INFO: Unaligned reads after nucleotide alignment: 87.3714285714 %
03/31/2021 01:38:32 PM - humann.search.translated - INFO: Running diamond …
03/31/2021 01:38:32 PM - humann.search.translated - INFO: Aligning to reference database: uniref90_201901.dmnd
03/31/2021 01:38:32 PM - humann.utilities - DEBUG: Remove file: /labs/Microbiome/shighlander/cyp8b1.humann/demo_humann_temp/tmpj28_k02y/diamond_m8_jj8t72nt
03/31/2021 01:38:32 PM - humann.utilities - DEBUG: Using software: /labs/Microbiome/shighlander/bin/anaconda3/envs/biobakery/bin/diamond
03/31/2021 01:38:32 PM - humann.utilities - INFO: Execute command: /labs/Microbiome/shighlander/bin/anaconda3/envs/biobakery/bin/diamond blastx --query /labs/Microbiome/shighlander/cyp8b1.humann/demo_humann_temp/demo_bowtie2_unaligned.fa --evalue 1.0 --threads 1 --top 1 --outfmt 6 --db /labs/Microbiome/shighlander/bin/anaconda3/envs/biobakery/biobakerydb/uniref/uniref90_201901 --out /labs/Microbiome/shighlander/cyp8b1.humann/demo_humann_temp/tmpj28_k02y/diamond_m8_jj8t72nt --tmpdir /labs/Microbiome/shighlander/cyp8b1.humann/demo_humann_temp/tmpj28_k02y
Hi, I have the same problem with you using diamond 0.9.24 and humann v3.0.0.alpha.3. The fastq files have 20~30 millions reads, but the computational time is more than 24 hours with the command in log file as listed below. It seems the diamond will never finish. Is there any solution? Thank you!
03/31/2021 12:57:56 PM - humann.utilities - INFO: Execute command: /data/home/quj_lab/xiaoyu/software/miniconda2/envs/py37/bin/diamond blastx --query /dellstorage01/quj_lab/xiaoyu/01_human_gut_metagenomics/04_humann_test/DP8400017160BR_L01_127_1_kneaddata_humann_temp/DP8400017160BR_L01_127_1_kneaddata_bowtie2_unaligned.fa --evalue 1.0 --threads 1 --top 1 --outfmt 6 --db /data/home/quj_lab/xiaoyu/software/humann/humann_databases/uniref/uniref90_201901 --out /dellstorage01/quj_lab/xiaoyu/01_human_gut_metagenomics/04_humann_test/DP8400017160BR_L01_127_1_kneaddata_humann_temp/tmpr35_iee9/diamond_m8_90djd7dz --tmpdir /dellstorage01/quj_lab/xiaoyu/01_human_gut_metagenomics/04_humann_test/DP8400017160BR_L01_127_1_kneaddata_humann_temp/tmpr35_iee9
I have found that my computational node was down. And there was no error when restarting it.
Hello, Sorry to hear you are having the same problem. Diamond v0.9.24 should work with the latest version of HUMAnN . Are you seeing any error messages from the diamond run? If so, if you could post those error messages it would be helpful.
Thank you,
Lauren
No error message. It just hangs.
Thank you for the additional information. Is it possible it is running out of memory? Depending on the size of your input file and which translated search database you are running with the diamond step might take a bit of memory. How much memory does the process have access to?
Thank you,
Lauren
No, I’m running it on a linux server and trying the demo.fastq file.
Thanks for the help.
Thank you for the information and sorry to hear that. I have not seen that before. Have you tried reinstalling diamond? Alternatively, have you tried running the diamond command directly (the command HUMAnN is running as a subprocess) to see if you can get more information about the issue?
Thank you,
Lauren
Yes, I have run the diamond command independently and it completed just fine. I wonder if it is something about my environment.
That is interesting. It might be. Is it possible there are multiple diamond versions installed and HUMAnN is picking up a different version (or install)? You can point HUMAnN to a specific diamond version (or install) using the option --diamond <path to folder>.
Thank you,
Lauren