cdf estimation

Given a sample $X_1,\dots ,X_n\sim F$, we want to estimate the cdf $F_X(a)=\mathbb{P}(X\le a)$.

The most common approach in frequentist statistics is to simply use the empirical cdf:

$${\hat{F}}_n(x)=\frac{1}{n}\sum _{i}1(X_i\le x).$$

Glivenko and Cantelli showed that this estimator obeys the uniform convergence guarantee:

$$\underset{x}{\sup }|{\hat{F}}_n(x)-F(x)|\stackrel{P}{\to }0.$$

In fact, we can get almost sure convergence as well. (See also cdf concentration and the DKW inequality for explicit rates of convergence).

The most common approach in Bayesian statistics is to use the Dirichlet process prior.