Banach space

A Banach space is a complete normed vector space. That is, it’s a vector space equipped with a norm $\left|\cdot \right|$ which induces a distance function in the obvious way. Every Cauchy sequence converges to some point in the space.

A major obstacle to working in Banach spaces is that an inner product does not necessarily exist.

In statistics, we usually assume the space is separable, meaning that it admits a countable dense subset. This makes sure that probability measures, Borel $\sigma$-algebras, and empirical means are well-defined.

We also often place a smoothness assumption on the space. Without such a smoothness assumption, roughly speaking, values that are close to each other in the space may behave so differently that statistical inference becomes impossible.

A Banach space $(B,\left|\cdot \right|)$ is $(2,\beta )$ smooth if

$${\left|x+y\right|}^2+{\left|x-y\right|}^2\le 2{\left|x\right|}^2+2{\beta }^2{\left|y\right|}^2,\forall x,y\in B.$$

An equivalent condition is that the Banach space $(B,\left|\cdot \right|)$ is $\beta$-smooth, where we say the space is $\beta$-smooth if the squared norm ${\left|\cdot \right|}^2$ is $\beta$-smooth with respect to $\left|\cdot \right|$, where we say that a function $f:B\to \mathbb{R}$ is $\beta$-smooth with respect to $\left|\cdot \right|$ if for any $x,y\in B$,

$$f(y)\le f(x)+D_{x}f(x)(y-x)+\frac{\beta }{2}{\left|y-x\right|}^2,$$

where $D_{x}f(x)(y-x)$ is the Gateaux derivative of $f$ at $x$ in the direction $y-x$.

Smooth, separable Banach spaces include any separable Hilbert space $(\beta =1)$ and Lp spaces ($\beta \le p\wedge 2$).

For statistical results in Banach spaces see concentration in Banach spaces and CLTs in Banach spaces.

Types and co-types

A Banach space has Type 2 if there exists some $T>0$ such that

$${\mathbb{E}}_{ϵ}\left|\sum _{i\le n}{ϵ}_i{x}_i\right|\le T\sqrt{\sum _{i\le n}{\left|x_i\right|}^2}.$$

where ${ϵ}_i$ are Rademacher. Hilbert spaces are Banach spaces of type 2 with $T=1$, which is easy to see if you write the norm in terms of the inner product.