total variation distance

The total variation distance between two measures $\mu$ and $\nu$ on a measurable space $(\Omega ,\mathcal{F})$ is

$$D_{\text{TV}}(\rho \Vert \nu ):=\underset{A\in \mathcal{F}}{\sup }|\mu (A)-\nu (A)|.$$

This can also be written as

$$D_{\text{TV}}(\rho \Vert \nu )=\underset{{|f|}_{\infty }\le 1}{\sup }|{\mathbb{E}}_{\mu }[f(X)]-{\mathbb{E}}_{\nu }[f(X)]|.$$

If $\mu$ and $\nu$ admit densities $\text{d}\mu$ and $\text{d}\nu$ then we can also write the TV distance as

$$D_{\text{TV}}(\mu \Vert \nu )=\frac{1}{2}\int |\text{d}\mu (x)-\text{d}\nu (x)|\text{d}x.$$

The TV distance is an f-divergence.