total variation distance

The total variation distance between two measures $\mu$ and $\nu$ on a measurable space $(\Omega ,\mathcal{F})$ is

$$D_{\text{TV}}(\rho \Vert \nu ):=\underset{A\in \mathcal{F}}{\sup }|\mu (A)-\nu (A)|.$$

This can also be written as

$$D_{\text{TV}}(\rho \Vert \nu )=\underset{{\Vert f\Vert }_{\infty }\le 1}{\sup }|{\mathbb{E}}_{\mu }[f(X)]-{\mathbb{E}}_{\nu }[f(X)]|,$$

which turns out to be a useful representation for robust statistics. Eg, this representation was used to construct Huber robust (see Huber contamination model) confidence sequences here.

If $\mu$ and $\nu$ admit densities $\text{d}\mu$ and $\text{d}\nu$ then we can also write the TV distance as

$$D_{\text{TV}}(\mu \Vert \nu )=\frac{1}{2}\int |\text{d}\mu (x)-\text{d}\nu (x)|\text{d}x.$$

The TV distance is an f-divergence.