multivariate heavy-tailed mean estimation

As in heavy-tailed concentration, the sample mean ${\overline{X}}_n$ is too easily influenced by outliers and does have good concentration properties. Also as in the scalar setting, the polynomial rate (in terms of the confidence level, $\delta$) given by Chebyshev’s inequality is the best one can hope for when working with the sample mean. (See the discussion in Hopkins or by Catoni, section 6).

What can we hope for in the multivariate setting? In ${\mathbb{R}}^d$, the empirical mean of the Gaussian behaves as

$$\mathbb{P}\left(\hat{\mu }-\mu |\ge O\left(\sqrt{\frac{\text{Tr}(\Sigma )}{n}}+\sqrt{\frac{\Vert \Sigma \Vert \log (1/\delta )}{n}}\right)\right)\le \delta .$$

This is called sub-Gaussian performance of an estimator, and is what we’re trying to achieve even in heavy-tailed settings. See the recent survey by Lugosi and Mendelson for more.

Approaches include:

• median-of-means
• truncation-based estimators
• Catoni-Giulini M-estimator