Bayesian parametrics

Bayesian parametric inference is an approach to statistical inference. We consider a family of distributions $\Theta =\{P_{\theta }:\theta \in \Theta \}$ where $\Theta$ is finite dimensional and we have data $X=(X_1,\dots ,X_n)$ generated by some distribution. We call this parametric inference because the distributions are parameterized by $\Theta$. This is contrast to Bayesian nonparametrics (and nonparametric methods more generally), which replace $\Theta$ by some infinite dimensional parameter space.

In accordance with Bayesian statistics, we put a prior $\pi$ over the parameter space $\Theta$ and then compute our posterior using $X$. That is, we assume that $\theta \sim \pi$ and then compute the posterior using Bayes’ theorem: $\mathbb{P}(\theta |X)=\mathbb{P}(X|\theta )\pi (\theta )/\mathbb{P}(X)$. $\mathbb{P}(X)$ is called the “evidence” and computing/estimating this integral is a big research area.