Random Processes in Communication and Control/M-Sep14

Last Time

PMF $P[X=x]=P[s:X(s)=x]$

a) $P_X(x)\ge 0\mathrm{\forall }x\in S_x$

b) $\sum _{x\in S_X}{P}_X(x)=1$

c) event $B\subset S_X$

$P[B]=\sum _{x\in B}{P}_X(x)$

Some Useful Random Variables

Bernoulli R.V

$P_X(x)=\{\begin{array}{cc}1-p& x=0\\ p& x=1\\ 0& o.w.\end{array}$

$0\le p\le 1$ success probability

Example

1) Flip a coin $X=$# of H

2) Manufacture a Chip $X=$# of acceptable chips

3) Bits you transmit successfully by a modem

Geometric Random Variable

Number of trials until (and including) a success for an underlying Bernoulli

$P_X(x)=p(1-p{)}^{x-1}x=1,2,3,\dots$

Example

1) Repeated coin flips $X=$# of tosses until H

2) Manufacture chips $X=$3 of chips produced until an acceptable time

Binomial R.V

"# of successes in n trials"

$P_x(x)=(\genfrac{}{}{0ex}{}{n}{x})p^x(1-p{)}^{n-x}x=0,1,\dots n$

Example

1) Flip a coin n times. $X=$ # of heads.

2) Manufacture n chips. $X=$ # of acceptable chips.

Note: Binomial $X=Y_1+Y_2+Y_3+\dots +Y_n$ where $Y_1+Y_2+Y_3\dots +Y_n$ are independent Bernoulli trials

Note: n=1; Binomial=Bernoulli; $X=Y_1$

Pascal R.V

"number of trials until (and including) the kth success with an underlying Bernoulli"

$P_X(x)=(\genfrac{}{}{0ex}{}{x-1}{k-1})p^k(1-p{)}^{x-k}x=k,k+1,k+2,\dots$

where $(\genfrac{}{}{0ex}{}{x-1}{k-1})$ is $k-1$ successes in $x-1$ trials

Note: Pascal $X=Y_1+Y_2+\dots +Y_k$ where $Y_1,Y_2,\dots Y_k$ are geometric R.V.

Note: K=1 Pascal=Geometric

Example

$X=$# of flips until the kth H

Discrete Uniform R.V.

$P_X(x)=\{\begin{array}{cc}\frac{1}{b-a+1}& x=a,a+1,\dots ,b\\ 0& \text{otherwise}\end{array}$

Example

1) Rolling a die. $a=1,b=6$

$P_X(x)=\{\begin{array}{cc}\frac{1}{6}& x=1,\dots ,6\\ 0& \text{otherwise}\end{array}$

2) Flip a fair coin. $X$=# of H

$P_X(x)=\{\begin{array}{cc}\frac{1}{2}& x=0\\ \frac{1}{2}& x=1\\ 0& \text{ otherwise }\end{array}$

Poisson R.V.

$P_X(x)=e^{-\alpha }\frac{{\alpha }^x}{x!}x=0,1,2\dots$

(Exercise) limiting case of binomial with $n\to \mathrm{\infty },p\to 0,np=\alpha$

PMF is a complete model for a random variable

Cumulative Distribution Function

$F_X(x)=P[X\le x]=\sum _{x^{\prime }\le x}{P}_X(X=x^{\prime })$

Like PMF, CDF is a complete description of random variable.

Example

Flip the coins $X=$# of H

$P_X(x)=\{\begin{array}{cc}\frac{1}{4}& x=0\\ \frac{1}{2}& x=1\\ \frac{1}{2}& x=2\\ 0& \text{otherwise}\end{array}$

$\begin{array}{cc}P[X\le 0]& =P[X=0]\\ P[X\le 0.5]& =P[X=0]\\ P[X\le 1]& =\underset{\frac{1}{4}}{\underbrace{P[X=0]}}+\underset{\frac{1}{2}}{\underbrace{P[X=1]}}\\ \end{array}$

Properties of CDF

• a) $F_X(-\mathrm{\infty })=0\Leftarrow F_X(-\mathrm{\infty })=P[X\le -\mathrm{\infty }]=0$

$F_X(\mathrm{\infty })=1\Leftarrow F_X(\mathrm{\infty })=P[X\le \mathrm{\infty }]=1$

"starts at 0 and ends at 1"

• b) For all $x^{\prime }\ge x$, $F_X(x^{\prime })\ge F_X(x)$

"non-decreasing in x"

$\begin{array}{cc}{F}_X(x^{\prime })& F_X(x)\\ P[X\le x^{\prime }]& P[X\le x]\\ P[s:X(s)\le x^{\prime }]& P[s:X(s)\le x]\\ \{s:X(s)\le x\}& \subset \{s:X(s)\le x^{\prime }\}\\ P[X\le x]& \le P[X\le x^{\prime }]\\ F_X(x)& \le F_X(x^{\prime })\end{array}$

• c) For all $x,x^{\prime }$

$P[x\le X\le x^{\prime }]=F_X(x^{\prime })-F_X(x)$

"probabilities can be found by difference of the CDF"

$\{s:X(x)\le x^{\prime }\}=\{s:X(x)\le x\}\cup \{s:x\le X(s)\le x^{\prime }\}$

$P[X\le x^{\prime }]=P[X\le x]+P[x\le X\le x^{\prime }]$

• d) For all $x$,

$\underset{ϵ\to 0}{\lim }{F}_X(x+ϵ)=F_X(x)$

"CDF is right continuous"

• e) For $x_i\in S_X$

$F_X(x_i)-F_X(x_i-ϵ)=P_X(x_i)$

"For a discrete random variable, there is a jump (discontinuity) in the CDF at each value $x_i\in S_X$. This jump equals $P_X(x_i)$

$P[x_i\le x]-P[x_i\le x_i-ϵ]=P[x_i-ϵ<x<x_i]=P[X=x_i]$

• f) $F_X(x)=F_X(x_i)$ for all $x_i\le x\le x_{i+1}$

"Between two jumps the CDF is constant"

$\begin{array}{cc}P[X\le x]& =P[X\le x_i\cup x_i<X\le x]\\ & =P[X\le x_i+\underset{0}{\underbrace{P[x_i<X\le x]}}\\ & =F_X(x_i)\end{array}$

• g) $P[X>x]=1-\underset{P[X\le x]}{\underbrace{F_X(x)}}$

Continuous Random Variables

outcomes uncountable many

Example

T: arrival of a partical

$S_T=\{t:0\le t<\mathrm{\infty }\}$

V: voltage

$S_V=\{v:-\mathrm{\infty }<v<\mathrm{\infty }\}$

$\theta$: angle

$S_{\theta }=\{\theta :0\le \theta \le 2\pi \}$

$X$: distance

$S_X=\{X:0\le x\le 1\}$

$P[x\in A]=\frac{1}{n}\to 0$

No PMF, $P[X=x]=0\mathrm{\forall }x$

Theorem

For any random variable (continuous or discrete)

• a) $F_X(-\mathrm{\infty })=0F_X(\mathrm{\infty })=1$

• b) $F_X(x)$ is nondecreasing in $X$

• c) $P[x<X\le x^{\prime }]=F_X(x^{\prime })-F_X(x)$

• d) $F_X(x)$ is right continuous

Example

$S_X=[0,1]$

$P_{[}x\in A]=P[xinB]$ where A, B are intervals of the same length contained in [0,1]

$P[X\le 1]=1\iff F_X(1)=1$

$P[X\le 0]=0\iff F_X(0)=0$

$P[x_1<x<x_2]=P[0<x<x_2-x_1]$

(exercise)$F_X(x_2)-F_X(x_1)=F_X(x_2-x_1)\to F_x(x)=x$

Probability Density Function (PDF)

$f_X(x)=\frac{d{F}_x(x)}{dx}$

discrete: PMF <--> CDF (sum/difference)

continuous <---> (derivative/integral)

Theorem: Properties of PDF

• a) $f_X(x)\ge 0$ ($F_X(x)$ is nondecreasing)

• b) $F_X(x)=\underset{\mathrm{\infty }}{\overset{x}{\int }}{f}_X(x)dx(F_X(\mathrm{\infty })=0)$

• c) $\underset{-\mathrm{\infty }}{\overset{\mathrm{\infty }}{\int }}{f}_X(x)dx=1(F_x(\mathrm{\infty })=1)$

Theorem

$\begin{array}{cc}P[x_1\le X\le x_2]& =F_X(x_2)-F_X(x_1)\\ & =\underset{-\mathrm{\infty }}{\overset{x_2}{\int }}{f}_X(x)dx-\underset{-\mathrm{\infty }}{\overset{x_1}{\int }}{f}_X(x)dx\\ & =\underset{x_1}{\overset{x_2}{\int }}{f}_X(x)\end{array}$

Some useful continuous Random Variables

Uniform R.V

$f_X(x)=\{\begin{array}{cc}\frac{1}{b-a}& a\le x\le b\\ 0& \text{otherwise}\end{array}$

Exponential R.V

$f_X(x)=a{e}^{-ax}x\ge 0$

$F_x(x)=1-e^{-ax}$

Gaussian (Normal) R.V.

$𝒩(\mu ,{\sigma }^2)\mathrm{\infty }<x<\mathrm{\infty }$

$f_X(x)=\frac{1}{\sqrt{2\pi {\sigma }^2}}{e}^{-\frac{(x-\mu {)}^2}{2{\sigma }^2}}$

$F_X(x)=\underset{-{\mathrm{\infty }}^1}{\int }\frac{1}{\sqrt{2\pi {\sigma }^2}}{e}^{-\frac{x-\mu }{2{\sigma }^2}}$ Template:BookCat