empirical risk minimization

Extremely general, and well-subscribed to, framework for how to pick a good model in learning theory. Suppose we have some set of models $\mathcal{F}$. Given training data, a natural way to choose a predictor $\hat{f}\in \mathcal{F}$ is to minimize the empirical risk:

$$\hat{f}\in {\text{argmin}}_{f\in \mathcal{F}}{\hat{R}}_n(f),$$

where ${\hat{R}}_n(f)$ is the empirical risk (see statistical decision theory).

You can prove bounds on the performance of ERM compared to the best classifier in $\mathcal{F}$ via PAC learning or PAC-Bayes bounds, though the former is more common. Note that because $\hat{f}$ is data-driven, one apply usual concentration inequalities to argue about $|{\hat{R}}_n(f)-R(f)|$. One needs to use different machinery, such as eg uniform convergence bounds.