Rao-Blackwell theorem

We observe data drawn from some $P_{\theta }$ for some $\theta \in \Theta$. Let $\ell :\Theta \to \mathbb{R}$ be any convex loss on estimates of $\theta$ (see statistical decision theory). The Rao-Blackwell theorem states that the expected loss of an estimator $\hat{\theta }$ can never be made worse by conditioning on a sufficient statistic $T$. That is, if we consider $\tilde{\theta }=\mathbb{E}[\hat{\theta }|T]$, then

$$\mathbb{E}\ell (\hat{\theta })\ge \mathbb{E}\ell (\tilde{\theta }).$$

Sufficiency is only used in order to define the estimator $\tilde{\theta }$ in order to make sure it’s actually independent from $\theta$ (otherwise the theorem is useless).

The estimator $\tilde{\theta }$ is often called the Rao-Blackwellization of $\hat{\theta }$. The theorem is usually applied with the squared error.

There is also a Rao-Blackwell-like result for e-values and e-processes (see this blog post or this paper), which says that conditioning on a sufficient statistic will never decrease log-power under the alternative (or $f$-power for any concave $f$). In sequential settings this means that the growth rate of the process will never be worse (growth rate conditions in sequential testing).

Relatedly, conditioning on a sufficient statistic was used by Lee and Ren in the context of multiple testing to improve the e-BH procedure.