Definition

Let $(Θ,ρ)$ be a metric space. The entropy number of $Θ$ with respect to $ρ$ is

$e_{n}(Θ)≡e_{n}(Θ,ρ):=Θ_{n}f θ∈Θsup ρ(θ,Θ_{n}),$

where the infimum is taken over all subset $Θ_{n}$ of $Θ$ subject to a cardinality constraint $N_{n}$ (usually $N_{n}=2_{2_{n}}$ in the case of Dudley chaining).

Entropy numbers show up when proving maximal inequalities via chaining-type arguments. Dudley’s entropy bound can be expressed in terms of entropy numbers.

They are related to covering numbers (covering and packing) via the formula

$e_{n}(Θ)=f{ϵ>0:N(Θ,ρ,ϵ)≤N_{n}}.$That is, they are the minimum width $ϵ$ required to cover $Θ$ in balls of width $ϵ$. Entropy numbers are equivalent to the metric entropy of $Θ$ in the sense that

$n≥0∑ 2_{n/2}e_{n}(Θ)≍∫_{0}gN(Θ;ρ,ϵ) dϵ,$which is what yields the equivalence between the two forms of Dudley’s entropy bound.