method of moments for estimation

Method for parametric density estimation. We are trying estimate an unknown $k$-dimensional parameter $\theta =({\theta }_1,\dots ,{\theta }_k)$ (statistical inference). The method of moments is an old school method (though it’s recently seen a revival in interest, eg for estimating Gaussian mixtures), which equates the first $k$ theoretical moments with the first $k$ empirical moments. That is, we solve

$$\frac{1}{n}\sum _{i\le n}{X}_i^j={\mathbb{E}}_{\hat{\theta }}[X^j],\text{for }j=1,\dots ,k,$$

(where $\hat{\theta }$ is unknown). Note this is a purely frequentist approach. Because we’re only trying to solve the moment equations, the resulting parameter estimates can be outside of the parameter space $\Theta$. This doesn’t happen with the MLE or with Bayesian approaches, which both naturally respect $\Theta$.

The MoM is consistent under fairly weak assumptions, and it obeys a CLT. It’s not guaranteed to be unbiased. For exponential families, the MoM estimator coincides with the MLE.