integral probability metric

Let $\Theta$ be some parameter space and $\mathcal{G}$ be a family of functions from $\Theta$ to $\mathbb{R}$. The integral probability metric (IPM) with respect to $\mathcal{G}$ between two distributions $\rho$ and $\nu$ is the distributional distance

$${\gamma }_{\mathcal{G}}(\rho ,\nu ):=\underset{g\in \mathcal{G}}{\sup }|{\mathbb{E}}_{\theta \sim \rho }g(\theta )-{\mathbb{E}}_{\theta \sim \nu }g(\theta )|.$$

IPMs recover the total variation distance, the Wasserstein distance, the KS distance, and the kernel MMD under different choices of $\mathcal{G}$. $\mathcal{G}$ is often taken to be symmetric so that we can get rid of the absolute value above.

IPMs and f-divergences intersect (non-trivially) only at the total variation distance. IPMs are convex and can thus be sequentially estimated by confidence sequences for convex functionals. See Sriperumbudur et al for more on plug-in estimation.