I am very much appreciate for your work on HUMAnN.
I used to use HUMAnN 2.0 to get community profiling, and it works fine. Recently, I have HUMAnN 3.0 installed since the HUMAnN 3.0 is available. I have run HUMAnN2 and HUMAnN3 using the same dataset, finding that there are lots of differences, both in taxonomy and pathway, especially in low abundance. And the correlation(spearman and person) of HUMAnN2 and HUMAnN3 is not much high as wished.
These results have confused me a lot, I don’t know which result is close to the real data. So, have you done any benchmarking analysis about HUMAnN3? And does it reasonable for such lots of differences?
We’ll have a preprint for the bioBakery 3 tools out shortly (which contains lots of detailed evaluations between the 2.0 and 3.0 tools). I can offer for now that the 3.0 tools are more accurate, especially for non-human-associated communities, so I would not hesitate to upgrade. There are a lot of reasons (including some you listed, like updated taxonomy) that might cause specific features to not correlate well between 2.0 and 3.0 profiles of the same samples.
Can I please follow-up on this thread? My colleague and I have also noted major differences between humann2 and humann3 when run on the same dataset. The sequencing depth is quite high. We understand that there are differences in the reference database, but are such large discrepancies in results expected? Attaching plots at the phylum level from humann2 vs humann3 for your thoughts.
Where are you getting the phylum-level abundances from? Are they taken directly from the MetaPhlAn abundance profile or are you doing something to the HUMAnN outputs to infer phylum abundances (and if so, what)?
