Fatal Error: not enough memory, please set -M option greater than 1240

Microbial community profiling ShortBRED
1

Hi,

I received the cd-hit error below while running shortbred_identify.py. My command is shortbred_identify.py --goi CAZyDB.07312020.fa --ref uniref90.fasta --markers CAZyDB_uniref90.faa --tmp CAZyDB_ShortBRED --threads 16, with 16 GB memmory per each core (16 cores). Any idea how to resolve this?

Fatal Error:
not enough memory, please set -M option greater than 1240

Program halted !!

Traceback (most recent call last):
  File "/home/amirza/.conda/envs/ShortBRED/bin/shortbred_identify.py", line 258, in <module>
    "-c", str(args.dClustID), "-b", "10","-g", "1"])
  File "/home/amirza/.conda/envs/ShortBRED/lib/python2.7/subprocess.py", line 190, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cd-hit', '-i', 'CAZyDB.07312020.fa', '-o', 'CAZyDB_ShortBRED/clust/clust.faa', '-d', '0', '-c', '0.85', '-b', '10', '-g', '1']' returned non-zero exit status 1

Memory info from cd-hit:

total seq: 1716042
longest and shortest : 39677 and 11
Total letters: 813254855
Sequences have been sorted


Approximated minimal memory consumption:
Sequence        : 1038M
Buffer          : 1 X 36M = 36M
Table           : 1 X 92M = 92M
Miscellaneous   : 22M
Total           : 1190M
2

I performed a dirty trick to temporarily solve this problem. I hope you can create a permanent solution soon.

“-M” is the memory limit (in MB) for the cd-hit program, default 800; 0 for unlimited. Edit shortbred_identify.py to increase the memory limit for cd-hit above the number the error suggested above. Add "-M", "3800", to line 258. It currently reads as "-o", strClustFile, "-d", "0", , change it to "-o", strClustFile, "-d", "0", "-M", "3800", .

location the file for conda users is .conda/envs/ShortBRED/bin/shortbred_identify.py

Creating an alias such as alias='cd-hit -M 2800' will not work.

1 Like