confidence sequences via predictable plug-ins

Suppose we have a sub-psi process or, more generally, a multiplicative e-process of the form $M_t={\prod }_{i\le t}{f}_i(\lambda ,X_i)$ where $(X_i)$ is some process and $\lambda$ is a scalar.

How does one choose $\lambda$? You might integrate over it (see confidence sequences via conjugate mixtures), or you might choose it differently at each timestep. That is, the process becomes $M_t={\prod }_{i\le t}{f}_i({\lambda }_i,X_i)$ where ${\lambda }_i$ can be predictable. This is called the method of predictable plug-ins.

If we’re building a confidence sequence from $M_t$ (eg perhaps $f_i$ has the form $f_i(\lambda ,X)=\exp \{\lambda (X-\mathbb{E}X)-\psi (\lambda )\}$), or see estimating means by betting), then choosing a different $\lambda$ every time step allows the width of the sequence to shrink to zero over time, whereas this will not be the case for constant $\lambda$.

Using predictable plug-ins is practically very convenient, but (usually) won’t lead to optimal rates laws of the iterated logarithm rates. For that, one needs stitching for LIL rates.