Gaussian process

A stochastic process $\{X_{\theta }{\}}_{\theta \in \Theta }$ where any finite collection of random variables follows a Gaussian distribution. That is, for any collection, $X_{{\theta }_1},\dots ,X_{{\theta }_k}$, we require that

$$(X_{{\theta }_1},\dots ,X_{{\theta }_k})\sim N(\mu ,\Sigma ),$$

where $\mu$ and $\Sigma$ can (and will) depend on the specific collection of random variables. In fact, the mean is

$$\mu =(\mathbb{E}{X}_{{\theta }_1},\dots ,\mathbb{E}{X}_{{\theta }_k}),$$

and the covariance matrix is defined by $\Sigma (s,t)=\text{Cov}(X_{{\theta }_s},X_{{\theta }_t})$. If the means are constant and $\text{Cov}(X_{{\theta }_s},X_{{\theta }_t})$ depends only on some notion of distance between ${\theta }_s$ and ${\theta }_t$ then we call the process stationary.

Gaussian processes are elegant because they are completely defined by the means and covariances: once you know those, you know how the process behaves.

Gaussian processes play an important role in Bayesian nonparametrics via Gaussian process regression.

Gaussian processes are distinct (and more strict) than sub-Gaussian processes, which require only pairwise sub-Gaussian behavior.