sub-exponential distributions

A sub-exponential distribution has tails which decay at an exponential rate. This is a weaker condition than sub-Gaussianity (meaning that sub-Gaussian tails decay at a faster rate, $\propto t^2$ instead of $\propto t$) but is still considered a light-tailed condition as the MGF exists in some neighborhood.

Sub-exponential conditions are also natural conditions to study because a variable $X$ is sub-Gaussian iff $X^2$ is sub-exponential. So sub-exponential distributions arise naturally when studied the square (or squared norm) of sub-Gaussian variables and processes.

The most straightforward definition of a sub-exponential random variable $X$ with mean $\mu$ is:

$$\mathbb{E}[\exp (\lambda (X-\mu ))]\le \exp ({\lambda }^2{\sigma }^2/2),$$

for all $\lambda \le 1/c$ where $c\in [0,\infty )$. In this case we say $X$ is $(\sigma ,c)$-sub-exponential. Note that $X$ is $\sigma$-sub-Gaussian if it is $(\sigma ,0)$-sub-exponential.

There is also a natural anisotropic version for a random vector $X$, namely:

$$\mathbb{E}\exp (\lambda ⟨\nu ,X-\mu ⟩)\le \exp ({\lambda }^2⟨\nu ,\Sigma \nu ⟩/2),$$

for all $\nu \in {\mathbb{S}}^{d-1}$ and some positive semi-definite matrix $\Sigma$. As in the case of sub-Gaussian distributions, there is also the accompanying isotropic version, which we don’t state.

In the scalar setting, equivalent formulations are:

• $\mathbb{P}(|X|\ge t)\le 2e^{-Kt}$ for some $K$
• $\sqrt{|X|}$ is sub-Gaussian.
• If ${\left|X\right|}_{{\psi }_1}$ is finite, where ${\psi }_1$ is the sub-exponential Orlicz norm.

Heavy-tailed distributions are sometimes characterized by way of contrast with (sub-)exponential distributions. For a sub-exponential random vector $X$, the tails decay more slowly than an exponential. So, for all $s>0$ one has ${\lim }_{t\to \infty }{e}^{ts}\mathbb{P}(⟨X-\mathbb{E}X,v⟩>t)<\infty$ for all unit vectors $v$. Therefore, heavy-tailed distributions are sometimes characterized as those for which ${\lim }_{t\to \infty }{e}^{ts}\mathbb{P}(⟨X-\mathbb{E}X,v⟩)=\infty$ for all $s$ and some unit vector $v$. See eg Hopkins 2020.