Statistics/Distributions/Normal (Gaussian)

Normal Distribution

Normal distribution is without exception the most widely used distribution. It also goes under the name Gaussian distribution. It assumes that the observations are closely clustered around the mean, μ, and this amount is decaying quickly as we go farther away from the mean. The measure of spread is quantified by the variance, ${\sigma }^2$.

Some examples of applications are:

• If the average man is 175 cm tall with a variance of 6 cm, what is the probability that a man found at random will be 183 cm tall?
• If the average man is 175 cm tall with a variance of 6 cm and the average woman is 168 cm tall with a variance of 3cm, what is the probability that the average man will be shorter than the average woman?
• If cans are assumed to have a variance of 4 grams, what does the average weight need to be in order to ensure that the 99% of all cans have a weight of at least 250 grams?

The density function is:

$f_{\mu ,\sigma }(x)=\frac{1}{\sigma \sqrt{2\pi }}{e}^{-(x-\mu {)}^2/2{\sigma }^2}$

where $-\mathrm{\infty }<x<\mathrm{\infty }$.

and the cumulative distribution function cannot be integrated into a single expression.

Normal distribution with parameters μ and σ is denoted as $N(\mu ,\sigma )$. If the rv X is normally distributed with expectation μ and standard deviation σ, one denotes: $X\sim N(\mu ,\sigma )$

Probability mass function

To verify that f(x) is a valid pmf we must verify that (1) it is non-negative everywhere, and (2) that the total integral is equal to 1. The first is obvious, so we move on to verify the second.

$\underset{-\mathrm{\infty }}{\overset{\mathrm{\infty }}{\int }}\frac{1}{\sigma \sqrt{2\pi }}{e}^{-(x-\mu {)}^2/2{\sigma }^2}dx=\frac{1}{\sigma \sqrt{2\pi }}\underset{-\mathrm{\infty }}{\overset{\mathrm{\infty }}{\int }}{e}^{-(x-\mu {)}^2/2{\sigma }^2}dx$

Now let $w=\frac{x-\mu }{\sigma \sqrt{2}}$. We see that $dw=\frac{dx}{\sigma \sqrt{2}}$.

$\frac{1}{\sqrt{\pi }}\underset{-\mathrm{\infty }}{\overset{\mathrm{\infty }}{\int }}{e}^{-w^2}dw$

Now we use the Gaussian integral that $\underset{-\mathrm{\infty }}{\overset{\mathrm{\infty }}{\int }}{e}^{-w^2}dw=\sqrt{\pi }$

$\frac{1}{\sqrt{\pi }}\sqrt{\pi }=1$

Mean

We derive the mean as follows

$\mathrm{E}[X]=\underset{-\mathrm{\infty }}{\overset{\mathrm{\infty }}{\int }}x\cdot f(x)dx$

$=\underset{-\mathrm{\infty }}{\overset{\mathrm{\infty }}{\int }}x\frac{1}{\sigma \sqrt{2\pi }}{e}^{-(x-\mu {)}^2/2{\sigma }^2}dx$

$=\underset{-\mathrm{\infty }}{\overset{\mathrm{\infty }}{\int }}[(x-\mu )+\mu ]\frac{1}{\sigma \sqrt{2\pi }}{e}^{-(x-\mu {)}^2/2{\sigma }^2}dx$

$=\underset{-\mathrm{\infty }}{\overset{\mathrm{\infty }}{\int }}(x-\mu )\frac{1}{\sigma \sqrt{2\pi }}{e}^{-(x-\mu {)}^2/2{\sigma }^2}dx+\underset{-\mathrm{\infty }}{\overset{\mathrm{\infty }}{\int }}\mu \frac{1}{\sigma \sqrt{2\pi }}{e}^{-(x-\mu {)}^2/2{\sigma }^2}dx$

$=\frac{1}{\sigma \sqrt{2\pi }}(-{\sigma }^2)\underset{-\mathrm{\infty }}{\overset{\mathrm{\infty }}{\int }}\frac{-x+\mu }{{\sigma }^2}{e}^{-(x-\mu {)}^2/2{\sigma }^2}dx+\mu \underset{-\mathrm{\infty }}{\overset{\mathrm{\infty }}{\int }}\frac{1}{\sigma \sqrt{2\pi }}{e}^{-(x-\mu {)}^2/2{\sigma }^2}dx$

We now see that the right integral is the complete integral over a normal pmf. This is therefore 1.

$\mathrm{E}[X]=\frac{1}{\sigma \sqrt{2\pi }}(-{\sigma }^2){\left[e^{-(x-\mu {)}^2/2{\sigma }^2}\right]}_{-\mathrm{\infty }}^{\mathrm{\infty }}+\mu$

$=\frac{1}{\sigma \sqrt{2\pi }}(-{\sigma }^2)[0-0]+\mu$

$=\mu$

Variance

$\mathrm{Var}(X)=\mathrm{E}[(X-E[X]{)}^2]=\underset{-\mathrm{\infty }}{\overset{\mathrm{\infty }}{\int }}(x-\mu {)}^2\cdot f(x)dx=\underset{-\mathrm{\infty }}{\overset{\mathrm{\infty }}{\int }}(x-\mu {)}^2\frac{1}{\sigma \sqrt{2\pi }}{e}^{-\frac{1}{2}\cdot {\left(\frac{x-\mu }{\sigma }\right)}^2}dx$

We let $w=\frac{x-\mu }{\sigma \sqrt{2}}$

$\mathrm{Var}(X)=\underset{-\mathrm{\infty }}{\overset{\mathrm{\infty }}{\int }}{\sigma }^{2}2w^2\frac{1}{\sigma \sqrt{2\pi }}{e}^{-w^2}\sigma \sqrt{2}dw=\frac{2{\sigma }^2}{\sqrt{\pi }}\underset{-\mathrm{\infty }}{\overset{\mathrm{\infty }}{\int }}w\cdot w{e}^{-w^2}dw$

We now use integration by parts with u=w and v=(-1/2)e^(-w^2)

$\mathrm{Var}(X)=\frac{2{\sigma }^2}{\sqrt{\pi }}({\left[w\frac{-1}{2}{e}^{-w^2}\right]}_{-\mathrm{\infty }}^{\mathrm{\infty }}-\underset{-\mathrm{\infty }}{\overset{\mathrm{\infty }}{\int }}\frac{-1}{2}{e}^{-w^2}dw)$

We see that the bracketed term is zero by L'Hôpital's rule.

$\mathrm{Var}(X)=\frac{2{\sigma }^2}{\sqrt{\pi }}\left(\frac{1}{2}\underset{-\mathrm{\infty }}{\overset{\mathrm{\infty }}{\int }}{e}^{-w^2}dw\right)$

Now we use the Gaussian integral again

$\mathrm{Var}(X)=\frac{2{\sigma }^2}{\sqrt{\pi }}\frac{1}{2}\sqrt{\pi }={\sigma }^2$

External links

• Interactive Normal (Gaussian) Distribution Web Applet (Java)