e-value

An e-variable with respect to a set of distributions $\mathcal{P}$ is a nonnegative random variable $E$ such that

$${\mathbb{E}}_P[E]\le 1,\forall P\in \mathcal{P}.$$

It is a measure of evidence against the null, somewhat analogous to p-values but defined in terms of expectations instead of probabilities. They are immune to lots of the issues with p-values and enable optional continuation, post-hoc hypothesis testing (see e-values enable post-hoc hypothesis testing) and sometimes optional stopping.

A note on terminology: The random variable is called an e-variable, and the realized value of the random variable is called an e-value. This is similar to p-variables and p-values. Depending on the author and situation, we often simply refer to the e-variable as an e-value (just as we do with p-variables and p-values).

E-values are intimately tied to hypothesis testing. If we are testing the null $P\in \mathcal{P}$, a large e-value can be used to reject the null at level $\alpha$ since, by Markov’s inequality (basic inequalities:Markov’s inequality), $\mathbb{P}(E\ge 1/\alpha )\le \alpha$. Thus, designing e-values which will be large under the alternative is a fruitful strategy in hypothesis testing. This is the focus of game-theoretic hypothesis testing. (See also growth rate conditions in sequential testing, which investigates how to design e-values and e-processes, and testing by betting—simple vs simple, testing by betting—simple vs composite, testing by betting—composite vs composite to see e-values in practice).

E-values have a natural interpretation in terms of game-theoretic probability (which leads to their applicability in hypothesis testing above). If we imagine paying 1 dollar for $E$, then under the null we expect not to gain any money.

E-values also have a sequential analogue, useful in sequential statistics and sequential hypothesis testing, called an e-process. An e-process is simply a stochastic process which is an e-value at each stopping time (which thus provides a level-$\alpha$ test at each stopping time, again by Markov’s inequality).

The numeraire e-variable is a special (and in some sense, optimal) e-value when testing composite alternatives.

Properties

The inverse of an e-value is a p-value: $\mathbb{P}(1/E\le \alpha )=\mathbb{P}(E\ge 1/\alpha )\le \alpha$. They tend to be more conservative than classical p-values, however (this is the price to pay for avoiding some of the issues with p-values).

E-values solve the optional continuation problem, in the sense that the product of conditionally independent e-values remains an e-value. This is because such a product forms a test-martingale. They also solve the optional stopping problem in the sense that e-processes solve optional stopping by construction, so if the e-value is secretly an e-process (eg if it is constructed with exponential inequalities or in the form of estimating means by betting) then this property carries over.

See merging e-values for a discussion on combining several e-values and e-value calibrators for a discussion on converting p-values to e-values and vice versa.

References

• Safe testing by Grunwald, De Heide, and Koolen
• The numeraire e-variable, by Larsson, Ramdas, and Ruf.
• Hypothesis testing with e-values