An e-variable with respect to a set of distributions is a nonnegative random variable such that
It is a measure of evidence against the null, somewhat analogous to p-values but defined in terms of expectations instead of probabilities. They are immune to lots of the issues with p-values and enable optional continuation, post-hoc hypothesis testing (see e-values enable post-hoc hypothesis testing) and sometimes optional stopping.
A note on terminology: The random variable is called an e-variable, and the realized value of the random variable is called an e-value. This is similar to p-variables and p-values. Depending on the author and situation, we often simply refer to the e-variable as an e-value (just as we do with p-variables and p-values).
E-values are intimately tied to hypothesis testing. If we are testing the null , a large e-value can be used to reject the null at level since, by Markov’s inequality (basic inequalities:Markov’s inequality), . Thus, designing e-values which will be large under the alternative is a fruitful strategy in hypothesis testing. This is the focus of game-theoretic hypothesis testing. (See also growth rate conditions in sequential testing, which investigates how to design e-values and e-processes, and testing by betting—simple vs simple, testing by betting—simple vs composite, testing by betting—composite vs composite to see e-values in practice).
E-values have a natural interpretation in terms of game-theoretic probability (which leads to their applicability in hypothesis testing above). If we imagine paying 1 dollar for , then under the null we expect not to gain any money.
E-values also have a sequential analogue, useful in sequential statistics and sequential hypothesis testing, called an e-process. An e-process is simply a stochastic process which is an e-value at each stopping time (which thus provides a level- test at each stopping time, again by Markov’s inequality).
The numeraire e-variable is a special (and in some sense, optimal) e-value when testing composite alternatives.
Properties
The inverse of an e-value is a p-value: . They tend to be more conservative than classical p-values, however (this is the price to pay for avoiding some of the issues with p-values).
E-values solve the optional continuation problem, in the sense that the product of conditionally independent e-values remains an e-value. This is because such a product forms a test-martingale. They also solve the optional stopping problem in the sense that e-processes solve optional stopping by construction, so if the e-value is secretly an e-process (eg if it is constructed with exponential inequalities or in the form of estimating means by betting) then this property carries over.
References
- Safe testing by Grunwald, De Heide, and Koolen
- The numeraire e-variable, by Larsson, Ramdas, and Ruf.