Metaphlan4 BrokenPipeError

Microbial community profiling MetaPhlAn
1

dragon@ser7:~/data/myfold$ metaphlan --bowtie2db /home/dragon/data/ sample.fasta --bowtie2out 4_bowtie2.bz2 --nproc 5 --input_type fasta --unclassified_estimation -o 4.txt
Killed
(ERR): bowtie2-align exited with value 137
Traceback (most recent call last):
File “/home/dragon/soft/anaconda3/envs/metaphlan/bin/read_fastx.py”, line 10, in
sys.exit(main())
File “/home/dragon/soft/anaconda3/envs/metaphlan/lib/python3.7/site-packages/metaphlan/utils/read_fastx.py”, line 168, in main
f_nreads, f_avg_read_length = read_and_write_raw(f, opened=False, min_len=min_len, prefix_id=prefix_id)
File “/home/dragon/soft/anaconda3/envs/metaphlan/lib/python3.7/site-packages/metaphlan/utils/read_fastx.py”, line 130, in read_and_write_raw
nreads, avg_read_length = read_and_write_raw_int(inf, min_len=min_len, prefix_id=prefix_id)
File “/home/dragon/soft/anaconda3/envs/metaphlan/lib/python3.7/site-packages/metaphlan/utils/read_fastx.py”, line 109, in read_and_write_raw_int
print_record(description + “__{}{}{}”.format(prefix_id, ‘.’ if prefix_id else ‘’, idx), sequence, qual, fmt))
BrokenPipeError: [Errno 32] Broken pipe

2

Hi @rooobben
It seems the error you are experimenting is due to lack of memory in your machine.

3

Hi @aitor.blancomiguez
I konw, but I’m confused that other samples can run with the same code.

4

I seem to be getting the same error:

(ERR): bowtie2-align died with signal 9 (KILL) 
Traceback (most recent call last):
  File "/usr/local/bin/read_fastx.py", line 10, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.10/site-packages/metaphlan/utils/read_fastx.py", line 168, in main
    f_nreads, f_avg_read_length = read_and_write_raw(f, opened=False, min_len=min_len, prefix_id=prefix_id)
  File "/usr/local/lib/python3.10/site-packages/metaphlan/utils/read_fastx.py", line 130, in read_and_write_raw
    nreads, avg_read_length = read_and_write_raw_int(inf, min_len=min_len, prefix_id=prefix_id)
  File "/usr/local/lib/python3.10/site-packages/metaphlan/utils/read_fastx.py", line 108, in read_and_write_raw_int
    _ = sys.stdout.write(
BrokenPipeError: [Errno 32] Broken pipe

…but I’m using a Github codespaces machine with 32 Gb of RAM, and the job dies after only ~7 Gb are used.

My specific nextflow process script:

    metaphlan --nproc ${task.cpus} --force \\
      --unclassified_estimation \\
      --nreads \$NREADS \\
      --output_file profile.txt \\
      --samout ${sample_id}.sam \\
      --bowtie2out ${sample_id}.bowtie2.txt \\
      --bowtie2db ${bowtie2_db} \\
      --input_type fastq \\
      --sample_id ${sample_id} \\
      read1.fq,read2.fq

I’m using quay.io/biocontainers/metaphlan:4.0.3--pyhca03a8a_0

At least for running bowtie2 in the same environment with the same metaphlan4 database & input read files, the bowtie2 job completes successfully. My job:

bowtie2 -p 16 --very-sensitive -x 4.0.3/mpa_vJan21_CHOCOPhlAnSGB_202103 -1 read1.fq -2 read2.fq > out.sam
5

The issue was due to the docker settings for memory that were automatically set by nextflow (only set to 6 Gb of RAM)