Biobakery workflows computational speed

Infrastructure and utilities bioBakery workflows
1

Hi

I have just installed and tested biobakery workflows with tutorial files and it works for me. It was a bi difficult to install so I created a conda environment file to save my configuration.

Now I am trying to us biobakery workflows with my samples, 32 human microbiota shotgun metagenome samples (64 paired files of 0.7-2GB each).

I used this code:

biobakery_workflows wmgx --input /mnt/d/DATOSSECUENCIACION/MVT-prj10_rendimiento_deportivo/BIKE/ --output /mnt/e/workflows_output --local-jobs 1 --threads 8

For now it takes 12 hours to perform 60/292 tasks, 9 samples.

I am running it in this PC:
Intel(R) Core™ i7-9700K CPU @ 3.60GHz
CPU(s): 8
CPU MHz: 3600.000
CPU MHz máx.: 4900,0000
CPU MHz mín.: 800,0000
Memory: 32026

Do you know any method to improve my computation speed. Could it be a code problem at --local-jobs and --threads arguments?

Thanks

2

Hello, Thank you for the detailed post. I think your settings based on your compute environment are spot on. The only thing you could possibly try is to run with --local-jobs 2 --threads 4 (2 tasks at once, each with 4 threads) to see if this might speed it up a bit.

Thanks!
Lauren

3

Thank you for your help. It works

4

That is great to hear! Thanks!
Lauren