Mercer kernel

A Mercer Kernel on a space $\mathcal{X}$ is a symmetric function $K:\mathcal{X}\times \mathcal{X}\to \mathbb{R}$ which is symmetric and positive semi-definite. That is, $K(x,y)=K(y,x)$ for all $x$ and $y$ and

$$\int \int K(x,y)f(x)f(y)p(x)p(y)dxdy\ge 0,$$

for all $f:\mathcal{X}\to \mathbb{R}$, where $p$ is a probability measure on $\mathcal{X}$. This means that $K$ is positive semi-definite. In a discrete space this reduces to the usual definition: $x^{t}Kx\ge 0$ for all $x$.

Mercer kernels can be used to construct an RKHS (indeed, each $K$ uniquely defines an RKHS and each RKHS is defined some $K$) and are therefore a major feature of RKHS regression. They are also used in nonparametric density estimation.

Note that Mercer Kernels are distinct from smoothing kernels, which are used in kernel regression, for instance.

$L_2$ basis

If the Kernel obeys ${\sup }_{x,y}K(x,y)$ then we can find a series of orthonormal functions $\{{\psi }_i\}$ such that

$$\int K(x,y){\psi }_i(x)dx={\lambda }_i{\psi }_i(y),$$

for some ${\lambda }_i$. This impliest that $K(x,y)={\sum }_i{\lambda }_i{\psi }_i(x){\psi }_i(y)$ where we can order ${\lambda }_1\ge {\lambda }_2\ge \dots$.