# Wmgx\_wmtx worfklow rna\_dna\_norm.py failure + why is metagenome needed in metatranscriptome analysis?

**URL:** <https://forum.biobakery.org/t/wmgx-wmtx-worfklow-rna-dna-norm-py-failure-why-is-metagenome-needed-in-metatranscriptome-analysis/2856>\
**Category:** bioBakery workflows\
**Created:** [December 1, 2021, 3:57pm UTC](https://forum.biobakery.org/t/wmgx-wmtx-worfklow-rna-dna-norm-py-failure-why-is-metagenome-needed-in-metatranscriptome-analysis/2856 "2021-12-01T15:57:25Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![hilasha2](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.biobakery.org/hilasha2/32/947_2.png) [@hilasha2](https://forum.biobakery.org/u/hilasha2)\
**Post date:** [December 1, 2021, 3:57pm UTC](https://forum.biobakery.org/t/wmgx-wmtx-worfklow-rna-dna-norm-py-failure-why-is-metagenome-needed-in-metatranscriptome-analysis/2856/1 "2021-12-01T15:57:25Z")

</div>

I have two questions -

1. I have tried to run the metagenome-metatranscriptome (wmgx\_wmtx) workflow in biobakery\_worfklows.  
The workflow fails whenever it reaches to the pipeline rna\_dna\_norm.py.  
I ran the worfklow on samples from the human microbiome project 2. I used some metatranscriptome samples and matching metagenome samples.  
I ran the workflow once using a tsv mapping file matching the metagenome and metranscriptome samples (I tried to follow closely the tutorial):

```auto
# wts	wms
CSM67UGO	CSM67UGO
CSM79HI3	CSM79HI3
CSM79HP4	CSM79HP4
ESM5ME9U	ESM5ME9U
HSM6XRVK	HSM6XRVK
HSM67VD2	HSM67VD2
HSMA33OT	HSMA33OT
MSM6J2RS	MSM6J2RS
MSM9VZMI	MSM9VZMI
PSM7J154	PSM7J154
PSM7J182	PSM7J182
PSMA264U	PSMA264U

```

This is the command I have used at first:

```auto
biobakery_workflows wmgx_wmtx --input-metagenome $INPUT_PATH/HMP2_samples_metagenomics --input-metatranscriptome $INPUT_PATH/HMP2_samples_metatranscriptomics --input-mapping $INPUT_PATH/mapping_samples_file.tsv --output $OUTPUT_PATH --threads 10 --local-jobs 10 --pair-identifier _R1 --qc-options="--trimmomatic ~/miniconda3/envs/$ENV_NAME/share/trimmomatic-0.39-2/ -db $INPUT_PATH/biobakery_workflows_databases/kneaddata_db/human_genome_bowtie2 -db $INPUT_PATH/biobakery_workflows_databases/kneaddata_db/human_transcriptome_bowtie2 -db $INPUT_PATH/biobakery_workflows_databases/kneaddata_db/ribosomal_RNA_bowtie2" --remove-intermediate-output --bypass-strain-profiling 

```

However, I got the following error message:

```auto
  Error executing action 0. Original Exception: 
  Traceback (most recent call last):
    File "~/miniconda3/envs/$ENV_NAME/lib/python3.7/site-packages/anadama2/runners.py", line 201, in _run_task_locally
      action_func(task)
    File "/~/miniconda3/envs/$ENV_NAME/lib/python3.7/site-packages/anadama2/helpers.py", line 89, in actually_sh
      ret = _sh(s, **kwargs)
    File "~/miniconda3/envs/$ENV_NAME/lib/python3.7/site-packages/anadama2/util/ __init__.py", line 320, in sh
      raise ShellException(proc.returncode, msg.format(cmd, ret[0], ret[1]))
  anadama2.util.ShellException: [Errno 1] Command `rna_dna_norm.py --input-dna $OUTPUT_PATH/whole_metagenome_shotgun/humann/merged/pathabundance.tsv --input-rna $OUTPUT_PATH/whole_metatranscriptome_shotgun/humann/merged/pathabundance.tsv --output $OUTPUT_PATH/humann/rna_dna_norm/paths --reduce-sample-name --mapping $INPUT_PATH/mapping_samples_file.tsv' failed. 
  Out: b'Reading RNA table\nReading DNA table\n'
  Err: b'The rna/dna sample names do not match. Please check the formatting of the mapping file.\n'

```

As mentioned above, the run fails when calling the rna\_dna\_norm.py command.  
According to the error message, I thought that the problem was with the mapping file. I checked it, and I don’t think it had any mistakes. It is a tab delimited file and matches exactly the paired metagenome and metatranscriptome samples (I do have paired end metagenome samples if it matters).  
At any rate, I realized I might not need the mapping file.  
The order of the samples’ names in the tables of ecs/genefamilies/pathabundance.tsv in the metagenome and metatranscriptome analyses are the same. These are the inputs for rna\_dna\_norm.py and since the order of the samples and names match, I thought the mapping file was unnecessary.

So I ran the command again, this time without the mapping table.  
However, I got the following error message:

```auto
Error executing action 0. Original Exception: 
  Traceback (most recent call last):
    File "~/miniconda3/envs/$ENV_NAME/lib/python3.7/site-packages/anadama2/runners.py", line 201, in _run_task_locally
      action_func(task)
    File "~/miniconda3/envs/$ENV_NAME/lib/python3.7/site-packages/anadama2/helpers.py", line 89, in actually_sh
      ret = _sh(s, **kwargs)
    File "~/miniconda3/envs/$ENV_NAME/lib/python3.7/site-packages/anadama2/util/ __init__.py", line 320, in sh
      raise ShellException(proc.returncode, msg.format(cmd, ret[0], ret[1]))
  anadama2.util.ShellException: [Errno 1] Command `rna_dna_norm.py --input-dna $OUTPUT_PATH/whole_metagenome_shotgun/humann/merged/genefamilies.tsv --input-rna $OUTPUT_PATH/whole_metatranscriptome_shotgun/humann/merged/genefamilies.tsv --output $OUTPUT_PATH/humann/rna_dna_norm/genes --reduce-sample-name' failed. 
  Out: b'Reading RNA table\nReading DNA table\nCompute unstratified features\nNormalize DNA\nNormalize RNA\nCompute stratified features\nNormalize DNA\nNormalize RNA\nCompute only classified features\nNormalize DNA\nNormalize RNA\nWriting unstratified table\n'
  Err: b'Traceback (most recent call last):\n File "~/miniconda3/envs/$ENV_NAME/bin/rna_dna_norm.py", line 272, in <module>\n main()\n File "~/miniconda3/envs/$ENV_NAME/bin/rna_dna_norm.py", line 258, in main\n output_unstrat_file)\n File "~/miniconda3/envs/$ENV_NAME/bin/rna_dna_norm.py", line 180, in write_file\n file_handle.write("\\t".join(column_labels)+"\\n")\nTypeError: a bytes-like object is required, not \'str\'\n'

```

I presume that I’ll have to debug the code in order to find out what’s the problem, but before I do that I thought I’d ask you whether this is a known issue or whether I am doing something wrong.

1. As for my second question - Why does the metagnome analysis run together with metatranscriptome analysis? Is there a way to run the metranscriptome analysis without any metagenome samples?

---

<div class="post-metadata">

**Author:** ![hilasha2](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.biobakery.org/hilasha2/32/947_2.png) [@hilasha2](https://forum.biobakery.org/u/hilasha2)\
**Post date:** [December 6, 2021, 5:16pm UTC](https://forum.biobakery.org/t/wmgx-wmtx-worfklow-rna-dna-norm-py-failure-why-is-metagenome-needed-in-metatranscriptome-analysis/2856/2 "2021-12-06T17:16:04Z")

</div>

I think I found one problem,  
In line 179 in the file biobakery\_workflows/scripts/rna\_dna\_norm.py, in the function:

```auto
def write_file(column_labels, row_labels, data, file):

```

You open a file to write the results to with the mode ‘wb’ (binary format) instead of ‘w’:

```auto
with open(file, "wb") as file_handle:

```

I changed this line to:

```auto
with open(file, "w") as file_handle:

```

I did manage to run the workflow that way.  
However, I got tables with many NaNs. I presume it’s a result of a division of 0/0. I also got Inf, and again I presume it’s the result of diving a number by zero. Is that correct?

For the pathways file I get:

 ![Screenshot from 2021-12-06 19-20-20](https://canada1.discourse-cdn.com/flex027/uploads/biobakery/original/2X/5/5be5ca58aeaf57565ca46daf5916e707cf9b86a6.png)

While the gene families file looks like this:

 ![Screenshot from 2021-12-06 19-21-37](https://canada1.discourse-cdn.com/flex027/uploads/biobakery/original/2X/c/c551ec2dbfa74247614c93d0342fef87b5f69793.png)

(samples were taken from the HMP2 database).

I hope that these results are typical.