Hi, I’m trying to run the wmgx workflow using the following command:
biobakery_workflows wmgx --input $INPUT --output $OUTPUT --grid-jobs 20 --grid sge --threads 20 --grid-partition long.q`
While it will submit a grid job, they will all fail. I have anadama2 V.0.10.0 and python V.3.10.11.
Here is the error:
(May 20 16:40:00) [ 0/460 - 0.00%] **GridJob ** Task 34: kneaddata____DFS1089_S121 <Grid JobId 1180424 : Submitted>
(May 20 16:40:05) [ 0/460 - 0.00%] **GridJob ** Task 37: kneaddata____DFS1093_S119 <Grid JobId 1180426 : Submitted>
(May 20 16:40:10) [ 0/460 - 0.00%] **GridJob ** Task 16: kneaddata____DFS1039_S9_L004 <Grid JobId 1180428 : Submitted>
(May 20 16:40:15) [ 0/460 - 0.00%] **GridJob ** Task 19: kneaddata____DFS1043_S10_L004 <Grid JobId 1180430 : Submitted>
(May 20 16:40:20) [ 0/460 - 0.00%] **GridJob ** Task 22: kneaddata____DFS1046_S11_L004 <Grid JobId 1180431 : Submitted>
(May 20 16:40:25) [ 0/460 - 0.00%] **GridJob ** Task 52: kneaddata____DFS1124_S189 <Grid JobId 1180432 : Submitted>
(May 20 16:40:30) [ 0/460 - 0.00%] **GridJob ** Task 28: kneaddata____DFS1067_S118 <Grid JobId 1180433 : Submitted>
(May 20 16:40:35) [ 0/460 - 0.00%] **GridJob ** Task 13: kneaddata____DFS1034_S8_L004 <Grid JobId 1180434 : Submitted>
(May 20 16:40:49) [ 0/460 - 0.00%] **GridJob ** Task 31: kneaddata____DFS1070_S117 <Grid JobId 1180435 : Submitted>
(May 20 16:40:54) [ 0/460 - 0.00%] **GridJob ** Task 43: kneaddata____DFS1117_S2_L004 <Grid JobId 1180436 : Submitted>
(May 20 16:40:59) [ 0/460 - 0.00%] **GridJob ** Task 49: kneaddata____DFS1120_S4_L004 <Grid JobId 1180437 : Submitted>
(May 20 16:41:04) [ 0/460 - 0.00%] **GridJob ** Task 55: kneaddata____DFS1126_S5_L004 <Grid JobId 1180438 : Submitted>
(May 20 16:41:09) [ 0/460 - 0.00%] **GridJob ** Task 46: kneaddata____DFS1119_S120 <Grid JobId 1180439 : Submitted>
(May 20 16:43:56) [ 0/460 - 0.00%] **GridJob ** Task 0: kneaddata____DFS1008_S3_L004 <Grid JobId 1180410 : FAILED>
(May 20 16:43:56) [ 0/460 - 0.00%] **GridJob ** Task 4: kneaddata____DFS1013_S1_L004 <Grid JobId 1180411 : FAILED>
(May 20 16:43:56) [ 0/460 - 0.00%] **GridJob ** Task 7: kneaddata____DFS1020_S6_L004 <Grid JobId 1180412 : FAILED>
(May 20 16:43:56) [ 0/460 - 0.00%] **GridJob ** Task 10: kneaddata____DFS1032_S7_L004 <Grid JobId 1180413 : FAILED>
(May 20 16:43:56) [ 0/460 - 0.00%] **GridJob ** Task 40: kneaddata____DFS1109_S136 <Grid JobId 1180414 : FAILED>
(May 20 16:43:56) [ 0/460 - 0.00%] **GridJob ** Task 25: kneaddata____DFS1062_S12_L004 <Grid JobId 1180415 : FAILED>
(May 20 16:43:56) [ 0/460 - 0.00%] **GridJob ** Task 58: kneaddata____DFS1132_S122 <Grid JobId 1180418 : FAILED>
(May 20 16:43:56) [ 0/460 - 0.00%] **GridJob ** Task 34: kneaddata____DFS1089_S121 <Grid JobId 1180424 : FAILED>
(May 20 16:43:56) [ 0/460 - 0.00%] **GridJob ** Task 37: kneaddata____DFS1093_S119 <Grid JobId 1180426 : FAILED>
(May 20 16:43:56) [ 0/460 - 0.00%] **GridJob ** Task 16: kneaddata____DFS1039_S9_L004 <Grid JobId 1180428 : FAILED>
(May 20 16:43:56) [ 0/460 - 0.00%] **GridJob ** Task 19: kneaddata____DFS1043_S10_L004 <Grid JobId 1180430 : FAILED>
(May 20 16:43:56) [ 0/460 - 0.00%] **GridJob ** Task 22: kneaddata____DFS1046_S11_L004 <Grid JobId 1180431 : FAILED>
(May 20 16:43:56) [ 0/460 - 0.00%] **GridJob ** Task 52: kneaddata____DFS1124_S189 <Grid JobId 1180432 : FAILED>
(May 20 16:43:56) [ 0/460 - 0.00%] **GridJob ** Task 28: kneaddata____DFS1067_S118 <Grid JobId 1180433 : FAILED>
(May 20 16:43:56) [ 0/460 - 0.00%] **GridJob ** Task 13: kneaddata____DFS1034_S8_L004 <Grid JobId 1180434 : FAILED>
(May 20 16:43:56) [ 0/460 - 0.00%] **GridJob ** Task 31: kneaddata____DFS1070_S117 <Grid JobId 1180435 : FAILED>
(May 20 16:43:56) [ 0/460 - 0.00%] **GridJob ** Task 49: kneaddata____DFS1120_S4_L004 <Grid JobId 1180437 : FAILED>
(May 20 16:43:56) [ 0/460 - 0.00%] **GridJob ** Task 43: kneaddata____DFS1117_S2_L004 <Grid JobId 1180436 : FAILED>
(May 20 16:43:56) [ 0/460 - 0.00%] **GridJob ** Task 55: kneaddata____DFS1126_S5_L004 <Grid JobId 1180438 : FAILED>
(May 20 16:43:56) [ 0/460 - 0.00%] **GridJob ** Task 46: kneaddata____DFS1119_S120 <Grid JobId 1180439 : FAILED>
Exception in thread Thread-17:
Exception in thread Thread-2:
Traceback (most recent call last):
Exception in thread Thread-9:
Traceback (most recent call last):
File "/miniconda3/envs/biobakery/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
File "/miniconda3/envs/biobakery/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
Exception in thread Thread-7:
Traceback (most recent call last):
File "/miniconda3/envs/biobakery/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
Exception in thread Thread-11:
Traceback (most recent call last):
Traceback (most recent call last):
Exception in thread Thread-13:
Exception in thread Thread-4:
Traceback (most recent call last):
Exception in thread Thread-6:
Traceback (most recent call last):
Exception in thread Thread-1:
Exception in thread Thread-5:
Traceback (most recent call last):
Exception in thread Thread-8:
Traceback (most recent call last):
self.run()
Exception in thread Thread-14:
Exception in thread Thread-12:
Exception in thread Thread-3:
File "/miniconda3/envs/biobakery/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
Exception in thread Thread-15:
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 485, in run
Traceback (most recent call last):
File "/miniconda3/envs/biobakery/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
Traceback (most recent call last):
Traceback (most recent call last):
Traceback (most recent call last):
Traceback (most recent call last):
Exception in thread Thread-19:
Traceback (most recent call last):
File "/miniconda3/envs/biobakery/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
Traceback (most recent call last):
File "/miniconda3/envs/biobakery/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
File "/miniconda3/envs/biobakery/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
File "/miniconda3/envs/biobakery/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
File "/miniconda3/envs/biobakery/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
Exception in thread Thread-10:
File "/miniconda3/envs/biobakery/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
File "/miniconda3/envs/biobakery/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
File "/miniconda3/envs/biobakery/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
File "/miniconda3/envs/biobakery/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
self.run()
Traceback (most recent call last):
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 485, in run
File "/miniconda3/envs/biobakery/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
File "/miniconda3/envs/biobakery/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
File "/miniconda3/envs/biobakery/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
Exception in thread Thread-18:
Traceback (most recent call last):
File "/miniconda3/envs/biobakery/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
Exception in thread Thread-16:
Exception in thread Thread-20:
return runners.worker_run_loop(self.work_q, self.result_q, self.run_task_by_type,
Traceback (most recent call last):
return runners.worker_run_loop(self.work_q, self.result_q, self.run_task_by_type,
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/runners.py", line 184, in worker_run_loop
File "/miniconda3/envs/biobakery/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
self.run()
self.run()
self.run()
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/runners.py", line 184, in worker_run_loop
self.run()
self.run()
self.run()
Traceback (most recent call last):
self.run()
self.run()
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 485, in run
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 485, in run
self.run()
self.run()
self.run()
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 485, in run
self.run()
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 485, in run
self.run()
self.run()
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 485, in run
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 485, in run
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 485, in run
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 485, in run
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 485, in run
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 485, in run
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 485, in run
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 485, in run
self.run()
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 485, in run
File "/miniconda3/envs/biobakery/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 485, in run
self.run()
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 485, in run
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 485, in run
result = run_task(task, extra)
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 494, in run_task_by_type
result = run_task(task, extra)
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 494, in run_task_by_type
self.run()
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 485, in run
self.run()
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 485, in run
return runners.worker_run_loop(self.work_q, self.result_q, self.run_task_by_type,
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/runners.py", line 184, in worker_run_loop
return runners.worker_run_loop(self.work_q, self.result_q, self.run_task_by_type,
return runners.worker_run_loop(self.work_q, self.result_q, self.run_task_by_type,
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/runners.py", line 184, in worker_run_loop
return runners.worker_run_loop(self.work_q, self.result_q, self.run_task_by_type,
return cls.run_task_command(task, extra)
return runners.worker_run_loop(self.work_q, self.result_q, self.run_task_by_type,
return runners.worker_run_loop(self.work_q, self.result_q, self.run_task_by_type,
return cls.run_task_command(task, extra)
return runners.worker_run_loop(self.work_q, self.result_q, self.run_task_by_type,
return runners.worker_run_loop(self.work_q, self.result_q, self.run_task_by_type,
return runners.worker_run_loop(self.work_q, self.result_q, self.run_task_by_type,
return runners.worker_run_loop(self.work_q, self.result_q, self.run_task_by_type,
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/runners.py", line 184, in worker_run_loop
return runners.worker_run_loop(self.work_q, self.result_q, self.run_task_by_type,
return runners.worker_run_loop(self.work_q, self.result_q, self.run_task_by_type,
return runners.worker_run_loop(self.work_q, self.result_q, self.run_task_by_type,
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 533, in run_task_command
return runners.worker_run_loop(self.work_q, self.result_q, self.run_task_by_type,
return runners.worker_run_loop(self.work_q, self.result_q, self.run_task_by_type,
return runners.worker_run_loop(self.work_q, self.result_q, self.run_task_by_type,
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/runners.py", line 184, in worker_run_loop
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 533, in run_task_command
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/runners.py", line 184, in worker_run_loop
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/runners.py", line 184, in worker_run_loop
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/runners.py", line 184, in worker_run_loop
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/runners.py", line 184, in worker_run_loop
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/runners.py", line 184, in worker_run_loop
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/runners.py", line 184, in worker_run_loop
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/runners.py", line 184, in worker_run_loop
return runners.worker_run_loop(self.work_q, self.result_q, self.run_task_by_type,
return runners.worker_run_loop(self.work_q, self.result_q, self.run_task_by_type,
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/runners.py", line 184, in worker_run_loop
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/runners.py", line 184, in worker_run_loop
result = run_task(task, extra)
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/runners.py", line 184, in worker_run_loop
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/runners.py", line 184, in worker_run_loop
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/runners.py", line 184, in worker_run_loop
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/runners.py", line 184, in worker_run_loop
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 494, in run_task_by_type
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/runners.py", line 184, in worker_run_loop
while ( grid_queue.job_timeout(job_final_status, jobid, time) or grid_queue.job_memkill(job_final_status, jobid, memory) ) and resubmission < 3:
result = run_task(task, extra)
while ( grid_queue.job_timeout(job_final_status, jobid, time) or grid_queue.job_memkill(job_final_status, jobid, memory) ) and resubmission < 3:
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/sge.py", line 143, in job_timeout
result = run_task(task, extra)
return cls.run_task_command(task, extra)
result = run_task(task, extra)
result = run_task(task, extra)
result = run_task(task, extra)
result = run_task(task, extra)
result = run_task(task, extra)
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 494, in run_task_by_type
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/sge.py", line 143, in job_timeout
result = run_task(task, extra)
result = run_task(task, extra)
result = run_task(task, extra)
result = run_task(task, extra)
result = run_task(task, extra)
result = run_task(task, extra)
result = run_task(task, extra)
result = run_task(task, extra)
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 494, in run_task_by_type
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 533, in run_task_command
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 494, in run_task_by_type
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 494, in run_task_by_type
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 494, in run_task_by_type
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 494, in run_task_by_type
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 494, in run_task_by_type
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 494, in run_task_by_type
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 494, in run_task_by_type
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 494, in run_task_by_type
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 494, in run_task_by_type
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 494, in run_task_by_type
result = run_task(task, extra)
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 494, in run_task_by_type
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 494, in run_task_by_type
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 494, in run_task_by_type
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 494, in run_task_by_type
new_status, cpus, new_time, new_memory = self.get_benchmark(jobid)
new_status, cpus, new_time, new_memory = self.get_benchmark(jobid)
result = run_task(task, extra)
return cls.run_task_command(task, extra)
return cls.run_task_command(task, extra)
return cls.run_task_command(task, extra)
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 533, in run_task_command
while ( grid_queue.job_timeout(job_final_status, jobid, time) or grid_queue.job_memkill(job_final_status, jobid, memory) ) and resubmission < 3:
return cls.run_task_command(task, extra)
return cls.run_task_command(task, extra)
return cls.run_task_command(task, extra)
return cls.run_task_command(task, extra)
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 494, in run_task_by_type
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 346, in get_benchmark
return cls.run_task_command(task, extra)
return cls.run_task_command(task, extra)
return cls.run_task_command(task, extra)
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 533, in run_task_command
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 533, in run_task_command
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 533, in run_task_command
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 533, in run_task_command
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 533, in run_task_command
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 533, in run_task_command
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 533, in run_task_command
return cls.run_task_command(task, extra)
return cls.run_task_command(task, extra)
return cls.run_task_command(task, extra)
return cls.run_task_command(task, extra)
return cls.run_task_command(task, extra)
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 533, in run_task_command
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 346, in get_benchmark
return cls.run_task_command(task, extra)
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 533, in run_task_command
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/sge.py", line 143, in job_timeout
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 533, in run_task_command
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 533, in run_task_command
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 533, in run_task_command
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 533, in run_task_command
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 533, in run_task_command
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 533, in run_task_command
new_status, cpus, new_time, new_memory = self.get_benchmark(jobid)
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 346, in get_benchmark
return cls.run_task_command(task, extra)
while ( grid_queue.job_timeout(job_final_status, jobid, time) or grid_queue.job_memkill(job_final_status, jobid, memory) ) and resubmission < 3:
while ( grid_queue.job_timeout(job_final_status, jobid, time) or grid_queue.job_memkill(job_final_status, jobid, memory) ) and resubmission < 3:
memory="{:.1f}".format(float(memory.replace("K",""))/1024.0)
while ( grid_queue.job_timeout(job_final_status, jobid, time) or grid_queue.job_memkill(job_final_status, jobid, memory) ) and resubmission < 3:
while ( grid_queue.job_timeout(job_final_status, jobid, time) or grid_queue.job_memkill(job_final_status, jobid, memory) ) and resubmission < 3:
while ( grid_queue.job_timeout(job_final_status, jobid, time) or grid_queue.job_memkill(job_final_status, jobid, memory) ) and resubmission < 3:
while ( grid_queue.job_timeout(job_final_status, jobid, time) or grid_queue.job_memkill(job_final_status, jobid, memory) ) and resubmission < 3:
while ( grid_queue.job_timeout(job_final_status, jobid, time) or grid_queue.job_memkill(job_final_status, jobid, memory) ) and resubmission < 3:
while ( grid_queue.job_timeout(job_final_status, jobid, time) or grid_queue.job_memkill(job_final_status, jobid, memory) ) and resubmission < 3:
memory="{:.1f}".format(float(memory.replace("K",""))/1024.0)
while ( grid_queue.job_timeout(job_final_status, jobid, time) or grid_queue.job_memkill(job_final_status, jobid, memory) ) and resubmission < 3:
memory="{:.1f}".format(float(memory.replace("K",""))/1024.0)
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/sge.py", line 143, in job_timeout
while ( grid_queue.job_timeout(job_final_status, jobid, time) or grid_queue.job_memkill(job_final_status, jobid, memory) ) and resubmission < 3:
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/sge.py", line 143, in job_timeout
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 533, in run_task_command
while ( grid_queue.job_timeout(job_final_status, jobid, time) or grid_queue.job_memkill(job_final_status, jobid, memory) ) and resubmission < 3:
while ( grid_queue.job_timeout(job_final_status, jobid, time) or grid_queue.job_memkill(job_final_status, jobid, memory) ) and resubmission < 3:
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/sge.py", line 143, in job_timeout
ValueError: could not convert string to float: '8.430B'
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/sge.py", line 143, in job_timeout
ValueError: could not convert string to float: '8.434B'
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/sge.py", line 143, in job_timeout
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/sge.py", line 143, in job_timeout
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/sge.py", line 143, in job_timeout
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/sge.py", line 143, in job_timeout
ValueError: could not convert string to float: '8.480B'
while ( grid_queue.job_timeout(job_final_status, jobid, time) or grid_queue.job_memkill(job_final_status, jobid, memory) ) and resubmission < 3:
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/sge.py", line 143, in job_timeout
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/sge.py", line 143, in job_timeout
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/sge.py", line 143, in job_timeout
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/sge.py", line 143, in job_timeout
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/sge.py", line 143, in job_timeout
while ( grid_queue.job_timeout(job_final_status, jobid, time) or grid_queue.job_memkill(job_final_status, jobid, memory) ) and resubmission < 3:
while ( grid_queue.job_timeout(job_final_status, jobid, time) or grid_queue.job_memkill(job_final_status, jobid, memory) ) and resubmission < 3:
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/sge.py", line 143, in job_timeout
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/sge.py", line 143, in job_timeout
while ( grid_queue.job_timeout(job_final_status, jobid, time) or grid_queue.job_memkill(job_final_status, jobid, memory) ) and resubmission < 3:
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/sge.py", line 143, in job_timeout
while ( grid_queue.job_timeout(job_final_status, jobid, time) or grid_queue.job_memkill(job_final_status, jobid, memory) ) and resubmission < 3:
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/sge.py", line 143, in job_timeout
new_status, cpus, new_time, new_memory = self.get_benchmark(jobid)
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 346, in get_benchmark
new_status, cpus, new_time, new_memory = self.get_benchmark(jobid)
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 346, in get_benchmark
new_status, cpus, new_time, new_memory = self.get_benchmark(jobid)
new_status, cpus, new_time, new_memory = self.get_benchmark(jobid)
new_status, cpus, new_time, new_memory = self.get_benchmark(jobid)
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 346, in get_benchmark
new_status, cpus, new_time, new_memory = self.get_benchmark(jobid)
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 346, in get_benchmark
new_status, cpus, new_time, new_memory = self.get_benchmark(jobid)
new_status, cpus, new_time, new_memory = self.get_benchmark(jobid)
new_status, cpus, new_time, new_memory = self.get_benchmark(jobid)
new_status, cpus, new_time, new_memory = self.get_benchmark(jobid)
new_status, cpus, new_time, new_memory = self.get_benchmark(jobid)
new_status, cpus, new_time, new_memory = self.get_benchmark(jobid)
new_status, cpus, new_time, new_memory = self.get_benchmark(jobid)
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 346, in get_benchmark
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 346, in get_benchmark
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 346, in get_benchmark
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 346, in get_benchmark
new_status, cpus, new_time, new_memory = self.get_benchmark(jobid)
memory="{:.1f}".format(float(memory.replace("K",""))/1024.0)
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 346, in get_benchmark
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 346, in get_benchmark
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 346, in get_benchmark
new_status, cpus, new_time, new_memory = self.get_benchmark(jobid)
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 346, in get_benchmark
ValueError: could not convert string to float: '8.402B'
memory="{:.1f}".format(float(memory.replace("K",""))/1024.0)
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 346, in get_benchmark
memory="{:.1f}".format(float(memory.replace("K",""))/1024.0)
new_status, cpus, new_time, new_memory = self.get_benchmark(jobid)
memory="{:.1f}".format(float(memory.replace("K",""))/1024.0)
memory="{:.1f}".format(float(memory.replace("K",""))/1024.0)
memory="{:.1f}".format(float(memory.replace("K",""))/1024.0)
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 346, in get_benchmark
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 346, in get_benchmark
memory="{:.1f}".format(float(memory.replace("K",""))/1024.0)
memory="{:.1f}".format(float(memory.replace("K",""))/1024.0)
ValueError: could not convert string to float: '8.355B'
ValueError: could not convert string to float: '8.438B'
ValueError: could not convert string to float: '5.402B'
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 346, in get_benchmark
ValueError: could not convert string to float: '8.391B'
new_status, cpus, new_time, new_memory = self.get_benchmark(jobid)
ValueError: could not convert string to float: '8.457B'
ValueError: could not convert string to float: '8.441B'
memory="{:.1f}".format(float(memory.replace("K",""))/1024.0)
memory="{:.1f}".format(float(memory.replace("K",""))/1024.0)
ValueError: could not convert string to float: '8.402B'
memory="{:.1f}".format(float(memory.replace("K",""))/1024.0)
ValueError: could not convert string to float: '8.453B'
memory="{:.1f}".format(float(memory.replace("K",""))/1024.0)
memory="{:.1f}".format(float(memory.replace("K",""))/1024.0)
ValueError: could not convert string to float: '8.457B'
memory="{:.1f}".format(float(memory.replace("K",""))/1024.0)
ValueError: could not convert string to float: '8.512B'
ValueError: could not convert string to float: '8.559B'
File "/miniconda3/envs/biobakery/lib/python3.10/site-packages/anadama2/grid/grid.py", line 346, in get_benchmark
memory="{:.1f}".format(float(memory.replace("K",""))/1024.0)
ValueError: could not convert string to float: '8.445B'
memory="{:.1f}".format(float(memory.replace("K",""))/1024.0)
ValueError: could not convert string to float: '8.395B'
ValueError: could not convert string to float: '8.398B'
ValueError: could not convert string to float: '5.547B'
memory="{:.1f}".format(float(memory.replace("K",""))/1024.0)
ValueError: could not convert string to float: '8.434B'
Any insights would be really appreciated!