Econometric Theory/Asymptotic Convergence

Asymptotic Convergence

Modes of Convergence

Convergence in Probability

Convergence in probability is going to be a very useful tool for deriving asymptotic distributions later on in this book. Alongside convergence in distribution it will be the most commonly seen mode of convergence.

Definition

A sequence of random variables $\{X_n;n=1,2,\cdots \}$ converges in probability to $X_{}$ if:

$\mathrm{\forall }ϵ,\delta >0,$

$\mathrm{\exists }N\mathrm{s.t.}\mathrm{\forall }n\ge N,$

$\mathrm{Pr}\{|X_n-X|>\delta \}<ϵ$

an equivalent statement is:

$\mathrm{\forall }\delta >0,$

$\underset{n\to \mathrm{\infty }}{\lim }\mathrm{Pr}\{|X_n-X|>\delta \}=0$

This will be written as either $X_n\begin{array}{c}\begin{array}{c}{}_p\\ ⟶\end{array}\end{array}X$ or $\mathrm{plim}{X}_n=X$.

Example

$X_n=\{\begin{array}{cc}\eta & 1-\begin{array}{c}\frac{1}{n}\end{array}\\ \theta & \begin{array}{c}\frac{1}{n}\end{array}\end{array}$

We'll make an intelligent guess that this series converges in probability to the degenerate random variable $\eta$. So we have that:

$\mathrm{\forall }\delta >0,\mathrm{Pr}\{|X_n-\eta |>\delta \}\le \mathrm{Pr}\{|X_n-\eta |>0\}=\mathrm{Pr}\{X_n=\theta \}=\begin{array}{c}\frac{1}{n}\end{array}$

Therefore our definition for convergence in probability in this case is:

$\mathrm{\forall }ϵ,\delta >0,$

$\mathrm{\exists }N\mathrm{s.t.}\mathrm{\forall }n\ge N,$

$\mathrm{Pr}\{|X_n-\eta |>\delta \}\le \mathrm{Pr}\{|X_n-\eta |>0\}=\mathrm{Pr}\{X_n=\theta \}=\begin{array}{c}\frac{1}{n}\end{array}<ϵ$

So for any positive values of $ϵ\in ℝ$ we can always find an $N\in \mathbb{N}$ large enough so that our definition is satisfied. Therefore we have proved that $X_n\begin{array}{c}{}_p\\ ⟶\end{array}\eta$.

Convergence Almost Sure

Almost-sure convergence has a marked similarity to convergence in probability, however the conditions for this mode of convergence are stronger; as we will see later, convergence almost surely actually implies that the sequence also converges in probability.

Definition

A sequence of random variables $\{X_n;n=1,2,\cdots \}$ converges almost surely to the random variable $X$ if:

$\mathrm{\forall }\delta >0,$

$\underset{n\to \mathrm{\infty }}{\lim }\mathrm{Pr}\{\bigcup _{m\ge n}|X_m-X|>\delta ,\}=0$

equivalently

$\mathrm{Pr}\{\underset{n\to \mathrm{\infty }}{\lim }{X}_n=X\}=1$

Under these conditions we use the notation $X_n\begin{array}{c}\begin{array}{c}{}_{a.s.}\\ ⟶\end{array}\end{array}X$ or $\underset{n\to \mathrm{\infty }}{\lim }{X}_n=X\mathrm{a.s.}$.

Example

Template:Disputed Let's see if our example from the convergence in probability section also converges almost surely. Defining:

$X_n=\{\begin{array}{cc}\eta & 1-\begin{array}{c}\frac{1}{n}\end{array}\\ \theta & \begin{array}{c}\frac{1}{n}\end{array}\end{array}$

we again guess that the convergence is to $\eta$. Inspecting the resulting expression we see that:

$\mathrm{Pr}\{\underset{n\to \mathrm{\infty }}{\lim }{X}_n=\eta \}=1-\mathrm{Pr}\{\underset{n\to \mathrm{\infty }}{\lim }{X}_n\ne \eta \}=1-\mathrm{Pr}\{\underset{n\to \mathrm{\infty }}{\lim }{X}_n=\theta \}\ge 1-\underset{n\to \mathrm{\infty }}{\lim }\begin{array}{c}\frac{1}{n}\end{array}=1$

Thereby satisfying our definition of almost-sure convergence.

Convergence in Distribution

Convergence in distribution will appear very frequently in our econometric models through the use of the Central Limit Theorem. So let's define this type of convergence.

Definition

A sequence of random variables $\{X_n;n=1,2,\cdots \}$ asymptotically converges in distribution to the random variable $X$ if $F_{X_n}(\zeta )\to F_X(\zeta )$ for all continuity points. $F_{X_n}(\zeta )$ and $F_{X_{}}(\zeta )$ are the cumulative density functions of $X_n$ and $X$ respectively.

It is the distribution of the random variable that we are concerned with here. Think of a students-T distribution: as the degrees of freedom, $n$, increases our distribution becomes closer and closer to that of a gaussian distribution. Therefore the random variable $Y_n\sim t(n)$ converges in distribution to the random variable $Y\sim N(0,1)$ (n.b. we say that the random variable $Y_n\begin{array}{c}{}_d\\ ⟶\end{array}Y$ as a notational crutch, what we really should use is $f_{Y_n}(\zeta )\begin{array}{c}{}_d\\ ⟶\end{array}{f}_Y(\zeta )$/

Example

Let's consider the distribution X_n whose sample space consists of two points, 1/n and 1, with equal probability (1/2). Let X be the binomial distribution with p = 1/2. Then X_n converges in distribution to X.

The proof is simple: we ignore 0 and 1 (where the distribution of X is discontinuous) and prove that, for all other points a, $\lim F_{X_n}(a)=F_X(a)$. Since for a < 0 all Fs are 0, and for a > 1 all Fs are 1, it remains to prove the convergence for 0 < a < 1. But $F_{X_n}(a)=\frac{1}{2}([a\ge \frac{1}{n}]+[a\ge 1])$ (using Iverson brackets), so for any a chose N > 1/a, and for n > N we have:

$n>1/a\to a>1/n\to [a\ge \frac{1}{n}]=1\wedge [a\ge 1]=0\to F_{X_n}(a)=\frac{1}{2}$

So the sequence $F_{X_n}(a)$ converges to $F_X(a)$ for all points where F_X is continuous.

Convergence in R-mean Square

Convergence in R-mean square is not going to be used in this book, however for completeness the definition is provided below.

Definition

A sequence of random variables $\{X_n;n=1,2,\cdots \}$ asymptotically converges in r-th mean (or in the $L^r$ norm) to the random variable $X$ if, for any real number $r>0$ and provided that $E(|X_n{|}^r)<\mathrm{\infty }$ for all n and $r\ge 1$,

$\underset{n\to \mathrm{\infty }}{\lim }E\left({|X_n-X|}^r\right)=0.$

Cramer-Wold Device

The Cramer-Wold device will allow us to extend our convergence techniques for random variables from scalars to vectors.

Definition

A random vector ${𝐗}_n\begin{array}{c}{}_d\\ ⟶\end{array}𝐗⟺{\mathit{\lambda }}^{\mathrm{T}}{𝐗}_n\begin{array}{c}{}_d\\ ⟶\end{array}{\mathit{\lambda }}^{\mathrm{T}}𝐗\mathrm{\forall }\Vert \mathit{\lambda }\Vert \ne 0$.

Central Limit Theorem

Let $X_1,X_2,X_3,...$ be a sequence of random variables which are defined on the same probability space, share the same probability distribution D and are independent. Assume that both the expected value μ and the standard deviation σ of D exist and are finite.

Consider the sum $S_n=X_1+...+X_n$. Then the expected value of $S_n$ is nμ and its standard error is σ n1/2. Furthermore, informally speaking, the distribution of Sn approaches the normal distribution N(nμ,σ2n) as n approaches ∞.