kernel density estimation

A specific approach to nonparametric density estimation, which can be viewed as a kind of “smoothed out” histogram. Let $K$ be a smoothing Kernel, i.e., $\int K(x)dx=1$, $\int xK(x)dx=0$ and $\int x^{2}K(x)dx>0$. That is, $K$ enduces a distribution on the space $\mathcal{X}$ which is symmetric about the origin.

Our estimate for $p(x)$ is

$$\hat{p}(x)=\frac{1}{n}\sum _{i=1}^n\frac{1}{h^d}K\left(\frac{x-X_i}{h}\right).$$

Essentially, we’re placing small lumps of mass around the data points $X_1,\dots ,X_n$. The size of the lumps is controlled by the Kernel and $h$, called the bandwidth. As $h$ increases, $\hat{p}$ becomes more uniform.

We can generalize the above to allow for positive definite bandwidth matrices $H$ and write

$$\hat{p}(x)=\sum _{i=1}^n{K}_H(x-X_i),$$

where $K_H(z)=|H{|}^{-1/2}K(H^{-1/2}z)$. Taking $H=h^{2}I$ recovers the previous formula.