Humann 3 bowtie error

Microbial community profiling HUMAnN
1

Hi,
I am trying the biobakery tools for the first time. I have installed humann3 using pip install humann --user and tried to run with demo data demo.fastq. its returns the the below error.
bowtie2 version is =bowtie2-2.3.5.1; MetaPhlAn version 3.0 (20 Mar 2020)

Any clue what I am doing wrong here.?

$ humann -i demo.fastq -o sample_results --nucleotide-database humann3_database_downloads/ chocophlan/ --protein-database humann3_database_downloads/uniref/
Output files will be written to: /local_scratch/sajan86/Oslo/sample_results

Running metaphlan …

CRITICAL ERROR: Error executing: /users/sajan86/.local/bin/metaphlan /local_scratch/sajan86/Oslo/humann2-2.8.2/examples/demo.fastq -t rel_ab -o /local_scratch/sajan86/Oslo/sample_results/demo_humann_temp/demo_metaphlan_bugs_list.tsv --input_type fastq --bowtie2out /local_scratch/sajan86/Oslo/sample_results/demo_humann_temp/demo_metaphlan_bowtie2.txt

Error message returned from metaphlan :
Error reading _ebwt[] array: no more data
Error: Encountered internal Bowtie 2 exception (#1)
Command: /appl/soft/bio/bowtie2/bowtie2-2.3.5.1-linux-x86_64/bowtie2-align-s --wrapper basic-0 --quiet --very-sensitive -x /users/sajan86/.local/lib/python3.7/site-packages/metaphlan/metaphlan_databases/mpa_v30_CHOCOPhlAn_201901 -p 4 --passthrough -U -
(ERR): bowtie2-align exited with value 1
Traceback (most recent call last):
File “/users/sajan86/.local/bin/read_fastx.py”, line 8, in
sys.exit(main())
File “/users/sajan86/.local/lib/python3.7/site-packages/metaphlan/utils/read_fastx.py”, line 155, in main
nreads += read_and_write_raw(f, opened=False, min_len=min_len)
File “/users/sajan86/.local/lib/python3.7/site-packages/metaphlan/utils/read_fastx.py”, line 119, in read_and_write_raw
nreads = read_and_write_raw_int(inf, min_len=min_len)
File “/users/sajan86/.local/lib/python3.7/site-packages/metaphlan/utils/read_fastx.py”, line 98, in read_and_write_raw_int
_ = sys.stdout.write(print_record(description, sequence, qual, fmt))
BrokenPipeError: [Errno 32] Broken pipe

2

This appears to be an error thrown by bowtie2 during the initial MetaPhlAn profiling step within HUMAnN. Are you able to use bowtie2 to complete other tasks? It seems like there might be something wrong with bowtie2 specifically.

1 Like
3

You should check if /users/sajan86/.local/lib/python3.7/site-packages/metaphlan/metaphlan_databases/ contains 6 .bt2 files with non zero size.
The error indicates that the index may be corrupted, deleting all the .bt2 files and running again metaphlan --install may resolve this.

1 Like
4

Hi, Thank you both.
The index files in the local folder were shown below

localbt2
localbt21224×323 11.6 KB

Then, I have deleted all the bt2 files and installed it again.
then I got this result.

> [sajan86@xxxx-login2 metaphlan_databases]$ ll
> total 3.3G
> -rw-rw----. 1 sajan86 pepr_sajan86 3.5K Jun 19 12:14 file_list.txt
> -rw-rw----. 1 sajan86 pepr_sajan86   26 Jun 19 12:14 mpa_latest
> -rw-rw----. 1 sajan86 pepr_sajan86 1.4G Jun 19 12:17 mpa_v30_CHOCOPhlAn_201901.fna
> -rw-rw-r--. 1 sajan86 pepr_sajan86 342M May 12 14:25 mpa_v30_CHOCOPhlAn_201901.fna.bz2
> -rw-rw----. 1 sajan86 pepr_sajan86   63 Jun 19 12:16 mpa_v30_CHOCOPhlAn_201901.md5
> -rw-rw-r--. 1 sajan86 pepr_sajan86  25M May 12 14:31 mpa_v30_CHOCOPhlAn_201901.pkl
> -rw-rw----. 1 sajan86 pepr_sajan86 367M Jun 19 12:15 mpa_v30_CHOCOPhlAn_201901.tar
> -rw-rw----. 1 sajan86 pepr_sajan86 601M Jun 26 11:42 mpa_v30_CHOCOPhlAn_201901.1.bt2
> -rw-rw----. 1 sajan86 pepr_sajan86 286M Jun 26 11:42 mpa_v30_CHOCOPhlAn_201901.2.bt2
> -rw-rw----. 1 sajan86 pepr_sajan86 9.9M Jun 26 11:33 mpa_v30_CHOCOPhlAn_201901.3.bt2
> -rw-rw----. 1 sajan86 pepr_sajan86 286M Jun 26 11:33 mpa_v30_CHOCOPhlAn_201901.4.bt2
> -rw-rw----. 1 sajan86 pepr_sajan86  73M Jun 26 11:46 mpa_v30_CHOCOPhlAn_201901.rev.1.bt2
> -rw-rw----. 1 sajan86 pepr_sajan86    0 Jun 26 11:45 mpa_v30_CHOCOPhlAn_201901.rev.2.bt2

Here it seems one file is zero size.

Can I download those files from somewhere (any ftp) and place it this folder?

5

Are you sure the process was finished when you did the ll command above? Another option would be to delete the .bt2 files again and manually build the index from mpa_v30_CHOCOPhlAn_201901.fna. I believe the command would be…

$ bowtie2-build -f mpa_v30_CHOCOPhlAn_201901.fna mpa_v30_CHOCOPhlAn_201901

At a minimum this might reveal an error that is occurring silently during the automated build process.

6

Thank you. I tried to build the index as you mentioned. here is what I got. Seems like storage issue?

Returning block of 1197454432 for bucket 1
Exited Ebwt loop
fchr[A]: 0
fchr[C]: 308078315
fchr[G]: 602122540
fchr[T]: 916414569
fchr[$]: 1197454431
Exiting Ebwt::buildToDisk()
An error occurred writing the index to disk.  Please check if the disk is full.
Total time for backward call to driver() for mirror index: 00:10:12
Error: Encountered internal Bowtie 2 exception (#1)
Command: /appl/soft/bio/bowtie2/bowtie2-2.3.5.1-linux-x86_64/bowtie2-build-s --wrapper basic-0 -f mpa_v30_CHOCOPhlAn_201901.fna mpa_v30_CHOCOPhlAn_201901
Deleting "mpa_v30_CHOCOPhlAn_201901.3.bt2" file written during aborted indexing attempt.
Deleting "mpa_v30_CHOCOPhlAn_201901.4.bt2" file written during aborted indexing attempt.
Deleting "mpa_v30_CHOCOPhlAn_201901.1.bt2" file written during aborted indexing attempt.
Deleting "mpa_v30_CHOCOPhlAn_201901.2.bt2" file written during aborted indexing attempt.
Deleting "mpa_v30_CHOCOPhlAn_201901.rev.1.bt2" file written during aborted indexing attempt.
Deleting "mpa_v30_CHOCOPhlAn_201901.rev.2.bt2" file written during aborted indexing attempt.
[sajan86@puhti-login1 metaphlan_databases]$

Should I try to install it in some other folder?

7

Or remove some data from that disk? If the disk is almost full you are likely to encounter future problems.

8

I’m having the same issue, where the first 4 bt2 files build but files 5 and 6 seem to be missing. I’ve tried re-building the bowtie index with bowtie2-build and reinstalling humann with no success.

9

I’m also having this same error message, but only when I include in my database long sequences (> 200000 bp). You can find attached a file which I got an error message after using the command:

humann --input seqError2.fasta.txt --output /vol/volume2/outputTest

However, I have no error message if the long sequences is removed from the file, or if it is put at the beginning of the file (it is a bit strange that I can solve the problem by changing the position of the long sequence).

seqError2.fasta.txt (2.5 MB)