Wasserstein Distance

Wasserstein distances are a class of distributional metrics between distributions $\mu$ and $\nu$ over some common space $\mathcal{X}$. They come from considering the optimal transport cost when the cost function is a metric. In particular, if $d:\mathcal{X}\times \mathcal{X}\to \mathbb{R}$ is a metric, set

$$W_p(\mu ,\nu )=(\underset{\pi \in \Gamma (\mu ,\nu )}{\min }{\mathbb{E}}_{X,Y\sim \pi }[d(X,Y{)}^p]{)}^{1/p},$$

where $\Gamma (\mu ,\nu )$ is the set of joint distributions on $\mathcal{X}\times \mathcal{X}$ who marginals are $\mu$ and $\nu$, that is

$$\Gamma (\mu ,\nu )=\{\pi :\int \pi (x,y)dy=\mu (x),\int \pi (x,y)dx=\nu (y)\}.$$

Wasserstein distances are an optimal class of distances because they take advantage of the underlying geometry of the space $\mathcal{X}$ (as induced by $d$). This is contradistinction to, e.g., the KL divergence and the total variation distance.

For distributions over $\mathbb{R}$, the 1-Wasserstein distance is

$$W_1(\mu ,\nu )=\int |F_{\mu }(x)-F_{\mu }(x)|\text{d}x,$$

where $F_{\mu }$ and $F_{\nu }$ are the cdfs of $\mu$ and $\nu$.

The Wasserstein distance is an ideal metric of order 1, and useful for proving central limit theorems.