Broken pipe with 48 threads and 98 Gb of RAM

Microbial community profiling HUMAnN
1

I executed the following code and end up with the following error, i am running this using a machine with 48 threads and 98 Gb of RAM, i used conda to install humann and also tried source code and end up with same error ?!!

humann --input Ex1.fastq --output Ex1 --threads 48

Output files will be written to: /home/ubuntu/nanopore/WGS/Ex1
Removing spaces from identifiers in input file …

Running metaphlan …

CRITICAL ERROR: Error executing: /home/ubuntu/miniconda3/bin/metaphlan /home/ubuntu/nanopore/WGS/Ex1/Ex1_humann_temp/tmpqqtrpzcu/tmp2bwbmnrt -t rel_ab -o /home/ubuntu/nanopore/WGS/Ex1/Ex1_humann_temp/Ex1_metaphlan_bugs_list.tsv --input_type fastq --bowtie2out /home/ubuntu/nanopore/WGS/Ex1/Ex1_humann_temp/Ex1_metaphlan_bowtie2.txt --nproc 48

Error message returned from metaphlan :
Killed
(ERR): bowtie2-align exited with value 137
Traceback (most recent call last):
File “/home/ubuntu/miniconda3/bin/read_fastx.py”, line 10, in
sys.exit(main())
^^^^^^
File “/home/ubuntu/miniconda3/lib/python3.12/site-packages/metaphlan/utils/read_fastx.py”, line 168, in main
f_nreads, f_avg_read_length = read_and_write_raw(f, opened=False, min_len=min_len, prefix_id=prefix_id)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/home/ubuntu/miniconda3/lib/python3.12/site-packages/metaphlan/utils/read_fastx.py”, line 130, in read_and_write_raw
nreads, avg_read_length = read_and_write_raw_int(inf, min_len=min_len, prefix_id=prefix_id)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/home/ubuntu/miniconda3/lib/python3.12/site-packages/metaphlan/utils/read_fastx.py”, line 108, in read_and_write_raw_int
_ = sys.stdout.write(
^^^^^^^^^^^^^^^^^
BrokenPipeError: [Errno 32] Broken pipe

2

Do you still see this error if running with fewer threads (or a single thread)? 48 is a lot for some processes and you might be encountering this error because of it. I typically use 8 threads when running MetaPhlAn + HUMAnN. In my experience there are diminishing returns for accelerated read mapping (the bottleneck in these methods) beyond that many threads.

3

I still get same error, even if i use fewer number of threads