f-divergence

A broad class of distributional distances. For a convex function $f$ with $f(1)=0$ and two probability measures $\rho ,\nu$ defined on a measurable space $(\Omega ,\mathcal{F})$, set

$$D_f(\rho \Vert \nu )={\mathbb{E}}_{\nu }\left[f\left(\frac{\text{d}\rho }{\text{d}\nu }(X)\right)\right]=\int f\left(\frac{\text{d}\rho }{\text{d}\nu }(x)\right)\nu (\text{d}x).$$

Common examples are:

• total variation distance
• KL divergence
• Hellinger distance
• chi-squared divergence

todo variational inequality