Catoni-Giulini M-estimator

In 2017, Catoni and Giulini proposed to multivariate concentration based on M-estimation. Let $\psi$ be any symmetric “influence function” such that

$$-\log (1-x+x^2/2)\le \psi (x)\le \log (1+x+x^2/2),\forall x\in \mathbb{R}.$$

The motivation behind this condition is to choose a function $\psi$ such that $e^{\psi }$ is bounded by polynomials. The estimator is then

$$\xi (\theta )=\frac{1}{n\lambda }\sum _{i\le n}{\int }_{{\mathbb{R}}^d}\psi (\lambda ⟨\vartheta ,X_i⟩){\rho }_{\theta }(d\vartheta ),$$

where ${\rho }_{\theta }$ is Gaussian with mean $\theta$ and covariance ${\beta }^{-1}I$ for some $\beta >0$. This is the estimate of $⟨\theta ,\mathbb{E}X⟩$. An approximation $\xi (\theta )$ is computationally tractable for certain choices of $\psi$, but still doesn’t give a closed-form bound, since you can’t compute it for all $\theta$.

If we have a bound on the raw second moment, $\mathbb{E}{\left|X\right|}^2\le v^2$, then choosing $\lambda =\sqrt{2\log (1/\delta )/(n{v}^2)}$ and $\beta =\lambda \sqrt{n{v}^2}$ gives the bound

$$\underset{\theta \in {\mathbb{S}}^{d-1}}{\sup }|\xi (\theta )-⟨\theta ,\mathbb{E}X⟩|=O\left(\sqrt{\frac{v^2\log (1/\delta )}{n}}\right).$$

Again, this is not quite a sub-Gaussian rate. This method can also be extended to matrices (also done by Catoni and Giulini). It is also proved using PAC-Bayes arguments.

The same approach can of course be taken in the scalar case. There we can get explicit confidence intervals using numerical methods. See Wang and Ramdas (2023) who obtain confidence sequences in this way.