# Loss of information when doing renorm then regroup **URL:** \ **Category:** HUMAnN\ **Created:** [December 12, 2019, 3:49pm UTC](https://forum.biobakery.org/t/loss-of-information-when-doing-renorm-then-regroup/80 "2019-12-12T15:49:54Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![kescobo](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.biobakery.org/kescobo/32/47_2.png) [@kescobo](https://forum.biobakery.org/u/kescobo)\ **Post date:** [December 12, 2019, 3:49pm UTC](https://forum.biobakery.org/t/loss-of-information-when-doing-renorm-then-regroup/80/1 "2019-12-12T15:49:54Z")
I recently noticed that a bunch of my KO and PFAM tables had large numbers of zeros in them. This seems to be caused by the fact that I used `humann2_regroup_table` on a TSS-normalized `genefamilies` table, which seems to cause loss of a ton of information: ```auto $ humann2_regroup_table -i output/humann2/main/C0200-3F-1A_S42_genefamilies.tsv -o ~/Desktop/test.tsv -g uniref90_pfam #... $ head ~/Desktop/test.tsv # Gene Family C0200-3F-1A_S42_Abundance-RPKs UNMAPPED 6126697.0 UNGROUPED 3035995.28 UNGROUPED|g __Adlercreutzia.s__ Adlercreutzia_equolifaciens 816.084 UNGROUPED|g __Akkermansia.s__ Akkermansia_muciniphila 1854.831 UNGROUPED|g __Alistipes.s__ Alistipes_finegoldii 3530.86 UNGROUPED|g __Alistipes.s__ Alistipes_senegalensis 9883.201 UNGROUPED|g __Alistipes.s__ Alistipes_shahii 5081.175 UNGROUPED|g __Anaerostipes.s__ Anaerostipes_hadrus 33344.871 UNGROUPED|g __Bacteroides.s__ Bacteroides_dorei 47247.771 $ humann2_renorm_table -i ~/Desktop/test.tsv -o ~/Desktop/test_relab.tsv -u relab -p # ... $ head ~/Desktop/test_relab.tsv # Gene Family C0200-3F-1A_S42_Abundance-RELAB UNMAPPED 0.446235 UNGROUPED 0.221125 UNGROUPED|g __Adlercreutzia.s__ Adlercreutzia_equolifaciens 5.94391e-05 UNGROUPED|g __Akkermansia.s__ Akkermansia_muciniphila 0.000135096 UNGROUPED|g __Alistipes.s__ Alistipes_finegoldii 0.000257168 UNGROUPED|g __Alistipes.s__ Alistipes_senegalensis 0.000719838 UNGROUPED|g __Alistipes.s__ Alistipes_shahii 0.000370085 UNGROUPED|g __Anaerostipes.s__ Anaerostipes_hadrus 0.00242866 UNGROUPED|g __Bacteroides.s__ Bacteroides_dorei 0.00344127 ``` So far so good, but when I regroup from `genefamilies_relab`… ```auto $ humann2_regroup_table -i output/humann2/main/C0200-3F-1A_S42_genefamilies_relab.tsv -o ~/Desktop/test2_relab.tsv -g uniref90_pfam # ... $ head ~/Desktop/test2_relab.tsv # Gene Family C0200-3F-1A_S42_Abundance-RELAB UNMAPPED 0.495 UNGROUPED 0.245 UNGROUPED|g __Adlercreutzia.s__ Adlercreutzia_equolifaciens 0.0 UNGROUPED|g __Akkermansia.s__ Akkermansia_muciniphila 0.0 UNGROUPED|g __Alistipes.s__ Alistipes_finegoldii 0.0 UNGROUPED|g __Alistipes.s__ Alistipes_senegalensis 0.001 UNGROUPED|g __Alistipes.s__ Alistipes_shahii 0.0 UNGROUPED|g __Anaerostipes.s__ Anaerostipes_hadrus 0.003 UNGROUPED|g __Bacteroides.s__ Bacteroides_dorei 0.004 ``` … everything is rounded to 3 decimal places. First way, none of them are zeros: ```auto $ julia -e "count(line-> parse(Float64, split(line, '\t')[2]) == 0, readlines(\"/home/kevin/Desktop/test_relab.tsv\")[2:end]) |> println" 0 ``` Second way, tons are: ```auto $ julia -e "count(line-> parse(Float64, split(line, '\t')[2]) == 0, readlines(\"/home/kevin/Desktop/test2_relab.tsv\")[2:end]) |> println" 50630 ``` Not sure if it’s a precision error or some assumption made in the regrouping script, but I couldn’t find any mention of it in the documentation. ---
**Author:** ![franzosa](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.biobakery.org/franzosa/32/3511_2.png) [@franzosa](https://forum.biobakery.org/u/franzosa)\ **Post date:** [December 12, 2019, 4:57pm UTC](https://forum.biobakery.org/t/loss-of-information-when-doing-renorm-then-regroup/80/2 "2019-12-12T16:57:02Z")
Looks like I added a `--precision` flag to this script that defaults to 3, probably assuming people were working with RPK/CPM units. Opening that up ought to rescue a bunch of the lost rare groups. I’ll make a note to change the default to NOT round to avoid this in the future. ---
**Author:** ![wfgui](https://avatars.discourse-cdn.com/v4/letter/w/67e7ee/32.png) [@wfgui](https://forum.biobakery.org/u/wfgui)\ **Post date:** [June 4, 2020, 9:12am UTC](https://forum.biobakery.org/t/loss-of-information-when-doing-renorm-then-regroup/80/3 "2020-06-04T09:12:42Z")
I see that the `UNMAPPED` and `UNGROUPED` have a high relative abundance ,is that normal? ```auto # Gene Family C0200-3F-1A_S42_Abundance-RELAB UNMAPPED 0.495 UNGROUPED 0.245 ``` Thanks~ ---
**Author:** ![franzosa](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.biobakery.org/franzosa/32/3511_2.png) [@franzosa](https://forum.biobakery.org/u/franzosa)\ **Post date:** [June 4, 2020, 12:13pm UTC](https://forum.biobakery.org/t/loss-of-information-when-doing-renorm-then-regroup/80/4 "2020-06-04T12:13:36Z")
Having about a 50/50 mix of known/unknown reads is very typical. The ungrouped fraction depends on how rare the group annotation is that you are using. For example most proteins have a pfam domain, so grouping by pfam leaves a small ungrouped fraction. Conversely only ~10% of proteins have an EC annotation, so that leaves a larger ungrouped fraction. ---
**Author:** ![DEEPCHANDA7](https://avatars.discourse-cdn.com/v4/letter/d/eb9ed0/32.png) [@DEEPCHANDA7](https://forum.biobakery.org/u/DEEPCHANDA7)\ **Post date:** [October 12, 2020, 12:01pm UTC](https://forum.biobakery.org/t/loss-of-information-when-doing-renorm-then-regroup/80/5 "2020-10-12T12:01:29Z")
Have you changed the default? I am running HUMAnN3 and still getting the same level of precision (3 decimals). As a result, I am getting tons of zeros. I have checked there is no optional argument for `humann_renorm_table` that can change the precision. When I used `uniref90_pfam` for regrouping, I found 0.2-0.3 UNGROUPED and on an average 0.5 UNMAPPED (used kneaddata only for decontamination). Is this normal? Thanks, DC7 ---
**Author:** ![lauren.j.mciver](https://avatars.discourse-cdn.com/v4/letter/l/f05b48/32.png) [@lauren.j.mciver](https://forum.biobakery.org/u/lauren.j.mciver)\ **Post date:** [October 14, 2020, 7:58pm UTC](https://forum.biobakery.org/t/loss-of-information-when-doing-renorm-then-regroup/80/6 "2020-10-14T19:58:27Z")
Hello - Yes, the default change from 3 decimals to None will be included in the next software release. For your current runs use the “–precision” option in the humann\_regroup\_table script (as I think this script and not the renorm table is where the precision changes occur). Sorry for any confusion about the precision settings. Thank you, Lauren