Famous Theorems of Mathematics/Law of large numbers

Given X1, X2, ... an infinite sequence of i.i.d. random variables with finite expected value E(X1) = E(X2) = ... = µ < ∞, we are interested in the convergence of the sample average

${\bar{X}}_n=\frac{1}{n}(X_1+\cdots +X_n).$

The weak law

Theorem: ${\bar{X}}_n\stackrel{P}{\to }\mu \text{for}n\to \mathrm{\infty }.$

Proof:

This proof uses the assumption of finite variance $\mathrm{Var}(X_i)={\sigma }^2$ (for all $i$). The independence of the random variables implies no correlation between them, and we have that

$\mathrm{Var}({\bar{X}}_n)=\frac{n{\sigma }^2}{n^2}=\frac{{\sigma }^2}{n}.$

The common mean μ of the sequence is the mean of the sample average:

$E({\bar{X}}_n)=\mu .$

Using Chebyshev's inequality on ${\bar{X}}_n$ results in

$\mathrm{P}(|{\bar{X}}_n-\mu |\ge \epsilon )\le \frac{{\sigma }^2}{n{\epsilon }^2}.$

This may be used to obtain the following:

$\mathrm{P}(|{\bar{X}}_n-\mu |<\epsilon )=1-\mathrm{P}(|{\bar{X}}_n-\mu |\ge \epsilon )\ge 1-\frac{{\sigma }^2}{n{\epsilon }^2}.$

As n approaches infinity, the expression approaches 1. And by definition of convergence in probability (see Convergence of random variables), we have obtained

${\bar{X}}_n\stackrel{P}{\to }\mu \text{for}n\to \mathrm{\infty }.$