The distinction between finite and infinite parameter families. Inference is of course easier in parametric families since we’re making more assumptions on the underlying data generating process. Nonparametric methods are thus more flexible and useful but have worse guarantees (eg convergence rates).

The canonical example of a nonparametric families of distributions is

This is nonparametric because there’s no obvious parameters associated with such a family, besides simply letting , which is a trivial way to describe it. Contrast this to, e.g., the family of Gaussians, which can be described by two parameters: the mean and the variance.