I’m running KneadData on a HPC cluster and parallelizing with GNU Parallel. I’m getting an error at the discordant pairs step but the log does not contain any information about what’s causing this. I’ve already fixed the sequence identifiers.
The kneaddata_bowtie2_discordant_pairs step runs by itself and all input files look correct. Is this possibly a RAM issue? I’m running 10 cores with 50 gb RAM per core.
Here is the output from the log: 09/22/2021 03:01:28 PM - kneaddata.utilities - CRITICAL: Error executing: kneaddata_bowtie2_discordant_pairs --bowtie2 /storage/work/epb5360/miniconda3/envs/eukdetect/bin/bowtie2 --threads 1 -x /gpfs/group/evk5387/default/emily/refs/hg37dec_v0.1 --mode strict --bowtie2-options "--very-sensitive-local --phred33" -1 /storage/home/epb5360/scratch/kneaded/sun2018/SRR7721904_rename_1_kneaddata.trimmed.1.fastq -2 /storage/home/epb5360/scratch/kneaded/sun2018/SRR7721904_rename_1_kneaddata.trimmed.2.fastq --un-pair /storage/home/epb5360/scratch/kneaded/sun2018/SRR7721904_rename_1_kneaddata_hg37dec_v0.1_bowtie2_paired_clean_%.fastq --al-pair /storage/home/epb5360/scratch/kneaded/sun2018/SRR7721904_rename_1_kneaddata_hg37dec_v0.1_bowtie2_paired_contam_%.fastq -U /storage/home/epb5360/scratch/kneaded/sun2018/SRR7721904_rename_1_kneaddata.trimmed.single.1.fastq,/storage/home/epb5360/scratch/kneaded/sun2018/SRR7721904_rename_1_kneaddata.trimmed.single.2.fastq --un-single /storage/home/epb5360/scratch/kneaded/sun2018/SRR7721904_rename_1_kneaddata_hg37dec_v0.1_bowtie2_unmatched_%_clean.fastq --al-single /storage/home/epb5360/scratch/kneaded/sun2018/SRR7721904_rename_1_kneaddata_hg37dec_v0.1_bowtie2_unmatched_%_contam.fastq -S /dev/null
Error message returned from bowtie2 :
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = “en_IL:en”,
LC_ALL = (unset),
LC_CTYPE = “C.UTF-8”,
LANG = “en_IL”
are supported and installed on your system.
perl: warning: Falling back to the standard locale (“C”).
/home/mybiotics/miniconda3/bin/bowtie2-align-s: error while loading shared libraries: libtbb.so.2: cannot open shared object file: No such file or directory
(ERR): Description of arguments failed!
Exiting now …
Unable to run bowtie2:
Hi,
EmilyB,
Have you solved the error executing: kneaddata_bowtie2_discordant_pairs???
I got the same error, and error.log was attached.
Look forward to your reply. yan-kneaddata-error.log.txt (19.8 KB)
Hi everyone
I ran into the same error, has anyone solved this ?
I am decontaminating two genomes (human + a plant). The log shows the human decontamination worked, but the error happens afterwards, with the plant genome. I supplied the --very-sensitive-local option to bowtie2 via kneaddata. The end of the log (after the error message similar to @EmilyB 's is :
Error message returned from bowtie2 :
87015042 reads; of these:
87015042 (100.00%) were unpaired; of these:
86283017 (99.16%) aligned 0 times
415099 (0.48%) aligned exactly 1 time
316926 (0.36%) aligned >1 times
0.84% overall alignment rate
Traceback (most recent call last):
File “/project/def-ilafores/common/kneaddata/bin/kneaddata_bowtie2_discordant_pairs”, line 8, in
sys.exit(main())
File “/project/def-ilafores/common/kneaddata/lib/python3.7/site-packages/kneaddata/bowtie2_discordant_pairs.py”, line 271, in main
process_alignments(args.sam,args.al_pair,args.un_pair,args.al_single,args.un_single,args.mode)
File “/project/def-ilafores/common/kneaddata/lib/python3.7/site-packages/kneaddata/bowtie2_discordant_pairs.py”, line 244, in process_alignments
organize_alignments_single(sam,open_files,counts,mode)
File “/project/def-ilafores/common/kneaddata/lib/python3.7/site-packages/kneaddata/bowtie2_discordant_pairs.py”, line 164, in organize_alignments_single
for line in file_handle:
OSError: [Errno 5] Input/output error
Hello @Joao-Gatica , Sorry for the slow response. Thank you for posting the detailed error. This error looks like an issue with the perl installation. If you add the shared library “libtbb.so.2” it should resolve the error.
Hi @EmilyB , Sorry for the slow response. That looks like a good amount of memory. Is it possible you are running out of disk space? If not, can you run the kneaddata_bowtie2_discordant_pairs command directly and see if it provides additional error information?
In my case, the path of bowtie2 was set to different directory with kneaddata installed directory. I fixed with the option --bowtie2 $kneaddata_bowtie2_dir(bin/). It worked well.