post-hoc valid confidence sequences

We might extend the desiderata of post-hoc hypothesis testing to post-hoc confidence sequences.

Confidence sequences are the time-uniform equivalent to confidence intervals. But what if we want time-uniformity in addition to post-hoc validity, i.e., the ability to change $\alpha$ after the fact? As we did in post-hoc hypothesis testing, formulating the notion of post-hoc validity requires discussing risk instead of error probabilities. We might say a family of confidence sequences $\{(C_t(\alpha ){)}_{t\ge 1}:\alpha \in (0,1)\}$ for a parameter $\mu$ is post-hoc valid if

$$\underset{P\in {\mathcal{P}}_{\mu }}{\sup }{\mathbb{E}}_P\left[\underset{\alpha \in (0,1)}{\sup }\frac{1(\mu \notin C_{\tau }(\alpha ))}{\alpha }\right]\le 1\text{for all stopping times }\tau ,$$

where $\mathcal{P}$ is the family of distributions (possibly a singleton) which have the relevant parameter $\mu$ (eg all distributions in some class with mean $\mu$). Note we need a family of confidence sequences since the CS may be different for different $\alpha$.

Post-hoc valid confidence can be built with e-processes: post-hoc confidence sequences via e-processes.