Multiple comparisons in run_lefse.py

Downstream analysis and statistics LEfSe
1

Hello,

I’m looking at the help menu for run_lefse.py and I see that there is an option for multiple comparisons, but I’m not understanding the distinctions.

Here is the option copied and pasted:

-s {0,1,2} set the multiple testing correction options. 0 no correction (more strict, default), 1 correction for independent comparisons, 2 correction for independent comparison

These are my questions:

  1. Could please explain how not performing multiple corrections is the most strict?

  2. What is the difference between 1 and 2; they look identical to me except 1 is plural (independent comparisons) while 2 is singular (independent comparison).

Thank you,
Samantha

4

Hello Samantha,

Thank you for reaching out to bioBakery support forum.

  • In this case “correction” is a little misleading, since setting the -s flag to 1 or 2 will make the p-value more permissive allowing multiple tests. By default, LEfSe will conservatively treat them as a single bulk test, hence more strict.
  • Apologies for the confusion on the ‘-s’ options flag.
    • 0 means no correction
    • 1 means correction for multiple independent tests
    • 2 means correction for multiple dependent tests.

The github repository has been updated as well with the typo fixed in the help menu (https://github.com/biobakery/lefse). Many thanks for pointing this out.

Regards,
Sagun

5

Hi there -
I’m trying to reconcile a few discussions on LEfSe and multiple testing corrections.

Regarding the -s parameter that can be set as discussed above - what type of multiple testing correction is performed when the independent tests (-s 1) option is selected?

Elsewhere in the support forum, it seems that multiple testing corrections was not previously baked into LEfSe (?) and so folks were running the run_lefse.py with the alpha set high (ie -a 1) to then take the output to do multiple testing correction with a different tool (ie p.adjust in R).

So as of now (2022), can we reasonably report multiple testing corrections have been performed if we run with the -s 1 option as above?

I’d also like to open the discussion regarding necessity of multiple testing corrections when we’ve got 3 different phases (Anova for class, KW for subclass, and LDA for effect size)

6

Hello,
As Sagun mentioned, the value 0 is the most stringent version of the subclass testing algorithm. When using -s 0, associations need to be significant at the Wilcoxon alpha (default 0.05) within all strata of the subclass in order for the feature to be considered significant. This may be more strict than some users want, and so by specifying -s 1 or -s 2, the alpha value for the individual Wilcoxon tests is raised, either multiplying by the total number of subclass tests (option 1) or by a slightly smaller adjustment, where the new alpha is equal to 1-(1-alpha)^n , where alpha is the original alpha and n is the number of tests (option 2). That way, all tests still need to be “significant” in order for the feature to be selected, but the significance threshold is raised when using options -s 1 or -s 2 to account for the fact that you’re conducting multiple tests.
I hope that helps, and let me know if I can make any clarifications.
Best,
Meg