anytime-valid p-values

Most p-values hold only at fixed times. That is, a p-value $P_n$ based on $n$ observations has the guarantee

$$\text{for all data-independent times }n,{\mathbb{P}}_0(P_n\le \alpha )\le \alpha .$$

An anytime-valid p-value holds at all stopping times:

$$\text{for all stopping times }\tau ,{\mathbb{P}}_0(P_{\tau }\le \alpha ),\alpha .$$

Such p-values enable safe, anytime-valid inference (SAVI). Anytime-valid p-values are to p-values what e-processes are to e-values.

Here ${\mathbb{P}}_0$ is understood to be the probability under the null. Of course, it can also be replaced with the supremum over a class of null distributions, the case in composite hypothesis testing.