Rao-Blackwell theorem

sufficiencyestimatorsdecision-theory

We observe data drawn from some $P_{\theta }$ for some $\theta \in \Theta$. Let $\ell :\Theta \to \mathbb{R}$ be any convex loss on estimates of $\theta$ (see statistical decision theory). The Rao-Blackwell theorem states that the expected loss of an estimator $\hat{\theta }$ can never be made worse by conditioning on a sufficient statistic $T$. That is, if we consider $\tilde{\theta }=\mathbb{E}[\hat{\theta }|T]$, then

$$\mathbb{E}\ell (\hat{\theta })\ge \mathbb{E}\ell (\tilde{\theta }).$$

Sufficiency is only used in order to define the estimator $\tilde{\theta }$ in order to make sure it’s actually independent from $\theta$ (otherwise the theorem is useless).

The estimator $\tilde{\theta }$ is often called the Rao-Blackwellization of $\hat{\theta }$. The theorem is usually applied with the squared error.