Hypothesis testing

Tests and their validity and power: Neyman–Pearson, likelihood-ratio, score and Wald tests, monotone likelihood ratio, exact conditional tests (Fisher, Hardy–Weinberg), rank tests and ROC area (Mann–Whitney), permutation p-values, multiple testing (Bonferroni, Holm, closed testing, Benjamini–Hochberg, false discovery rates), and meta-analysis and heterogeneity statistics.

• Exact test for Hardy–Weinberg equilibrium
• Fisher's exact test
• Mann–Whitney statistic and ROC area
• Meta-analysis
• Monotone likelihood ratio
• Score, Wald and likelihood-ratio tests
• Local power of the likelihood-ratio test
• Valid p-values and false discovery proportion
• Benjamini–Hochberg procedure
• Bonferroni correction
• Holm–Bonferroni method
• Closed testing principle
• Permutation tests
• Two-groups model and local false discovery rates

TheoremNeyman–Pearson lemma#✓fully proved

Let \(\mu\) be a measure on a measurable space \(\Omega\), and let \(p, q : \Omega \to \mathbb{R}\) be measurable, nonnegative and \(\mu\)-integrable. Let \(k \ge 0\) be a real number and let \(\varphi^\star : \Omega \to [0, 1]\) be measurable with \(\varphi^\star(x) = 1\) whenever \(q(x) > k\,p(x)\) and \(\varphi^\star(x) = 0\) whenever \(q(x) < k\,p(x)\). Then every measurable test \(\varphi : \Omega \to [0, 1]\) satisfying \(\int \varphi\, p \, d\mu \le \int \varphi^\star p \, d\mu\) also satisfies \(\int \varphi\, q \, d\mu \le \int \varphi^\star q \, d\mu\).

No normalization of \(p\) or \(q\) and no condition on \(\varphi^\star\) on the set where \(q = k\,p\) are assumed.

theorem neyman_pearson_dominated_two_simple
    {Ω : Type*} [MeasurableSpace Ω] {μ : Measure Ω}
    {p q : Ω → ℝ}
    (hp_meas : Measurable p) (hq_meas : Measurable q)
    (hp_nonneg : ∀ x, 0 ≤ p x) (hq_nonneg : ∀ x, 0 ≤ q x)
    (hp_int : Integrable p μ) (hq_int : Integrable q μ)
    (k : ℝ) (hk : 0 ≤ k)
    (φstar : Ω → Set.Icc (0 : ℝ) 1) (hφstar_meas : Measurable φstar)
    (hφstar_gt : ∀ x, k * p x < q x → (φstar x).val = 1)
    (hφstar_lt : ∀ x, q x < k * p x → (φstar x).val = 0) :
    ∀ (φ : Ω → Set.Icc (0 : ℝ) 1), Measurable φ →
      (∫ x, (φ x).val * p x ∂μ ≤ ∫ x, (φstar x).val * p x ∂μ) →
      (∫ x, (φ x).val * q x ∂μ ≤ ∫ x, (φstar x).val * q x ∂μ)

Sources

• J. Neyman and E. S. Pearson, "On the problem of the most efficient tests of statistical hypotheses", Phil. Trans. R. Soc. Lond. A 231 (1933), 289–337, DOI 10.1098/rsta.1933.0009.

Used by 2

Used byThresholding the log-odds score is most powerful · PhyloCSF's score threshold test is most powerful for unnormalized weights

The dependency explorer needs JavaScript and its graph data.

Download the complete graph data.