KL divergence

For probability measures $\rho ,\nu$ on a measurable space $(\Omega ,\mathcal{F})$, the KL-divergence is

$$D_{\text{KL}}(\rho \Vert \nu )={\mathbb{E}}_{\rho }\left[\log \left(\frac{\text{d}\rho }{\text{d}\nu }(X)\right)\right]=\int \log \left(\frac{\text{d}\rho }{\text{d}\nu }(x)\right)\rho (\text{d}x),$$

if $\rho \ll \nu$ and $\infty$ otherwise.