View source for Random Processes in Communication and Control/M-Sep14

==Last Time==

PMF <math>P[X=x]=P[s:X(s)=x] \!\,</math>


a) <math>P_X(x) \geq 0 \forall x \in S_x \!\,</math>


b) <math>\sum_{x \in S_X} P_X(x)=1 \!\,</math>


c) event <math>B \subset S_X \!\,</math>


<math>P[B]= \sum_{x\in B} P_X(x) \!\,</math>

==Some Useful Random Variables==

===Bernoulli R.V===

<math> 
P_X(x)= \begin{cases}
1-p & x=0 \\
p   & x=1 \\
0   & o.w.
\end{cases}

\!\,</math>


<math>0 \leq p \leq 1 \!\,</math> success probability

====Example====
1) Flip a coin <math>X= \!\,</math># of H


2) Manufacture a Chip <math>X=\!\,</math># of acceptable chips


3) Bits you transmit successfully by a modem

===Geometric Random Variable===
Number of trials until (and including) a success for an underlying Bernoulli


<math>P_X(x)=p(1-p)^{x-1} \quad x=1,2,3,\ldots \!\,</math>

====Example====
1) Repeated coin flips <math>X= \!\,</math># of tosses until H


2) Manufacture chips <math>X=  \!\,</math>3 of chips produced until an acceptable time

===Binomial R.V===
"# of successes in n trials"


<math>P_x(x)= {n \choose x} p^x (1-p)^{n-x} \quad x=0,1,\ldots n \!\,</math>


====Example====

1) Flip a coin n times.  <math>X= \!\,</math> # of heads.


2) Manufacture n chips.  <math>X=  \!\,</math> # of acceptable chips.


Note:  Binomial <math>X=Y_1+ Y_2+Y_3 + \ldots+Y_n \!\,</math> where <math>Y_1+Y_2+Y_3 \ldots+ Y_n \!\,</math> are independent Bernoulli trials


Note: n=1; Binomial=Bernoulli; <math>X=Y_1 \!\,</math>

===Pascal R.V===
"number of trials until (and including) the kth success with an underlying Bernoulli"


<math>P_X(x)= {x-1 \choose k-1} p^k (1-p)^{x-k} \quad x=k,k+1,k+2,\ldots \!\,</math>


where <math>{x-1 \choose k-1}  \!\,</math> is <math>k-1 \!\,</math> successes in <math> x-1\!\,</math> trials


Note: Pascal <math>X=Y_1+Y_2+\ldots+Y_k \!\,</math> where <math>Y_1,Y_2,\ldots Y_k \!\,</math> are geometric R.V.


Note: K=1 Pascal=Geometric

====Example====

<math>X=\!\,</math># of flips until the kth H

===Discrete Uniform R.V.===

<math> P_X(x)=
\begin{cases}
\frac{1}{b-a+1} & x=a,a+1,\ldots,b \\
0               & \mbox{otherwise}

\end{cases}
\!\,</math>


====Example====
1) Rolling a die. <math>a=1, b=6 \!\,</math>


<math>P_X(x)=
\begin{cases}
\frac16 & x=1,\ldots,6 \\
0       & \mbox{otherwise}
\end{cases}
 \!\,</math>


2) Flip a fair coin.  <math>X \!\,</math>=# of H


<math> 
P_X(x)=
\begin{cases}
\frac12 & x=0 \\
\frac12 & x=1 \\
0       & \mbox{ otherwise }
\end{cases}
\!\,</math>


===Poisson R.V.===

<math>P_X(x)= e^{-\alpha} \frac{\alpha^x}{x!} \quad x=0,1,2\ldots \!\,</math>


(Exercise) limiting case of binomial with <math>n \rightarrow \infty, p \rightarrow 0, np=\alpha \!\,</math>


PMF is a complete model for a random variable

==Cumulative Distribution Function==
<math>F_X(x)= P[X \leq x] = \sum_{x' \leq x} P_X(X=x') \!\,</math>


Like PMF, CDF is a complete description of random variable.

===Example===
Flip the coins <math>X= \!\,</math># of H


<math>P_X(x)=
\begin{cases}
\frac14 & x=0 \\
\frac12 & x=1 \\
\frac12 & x=2 \\
0       & \mbox{otherwise}

\end{cases}
 \!\,</math>


<math>
\begin{align}
P[X \leq 0]&=P[X=0]\\
P[X \leq 0.5] &= P[X=0] \\
P[X \leq 1 ]  &= \underbrace{P[X=0]}_{\frac14} + \underbrace{P[X=1]}_{\frac12} \\

\end{align}
 \!\,</math>

===Properties of CDF===

* a) <math>F_X(-\infty)=0 \Leftarrow F_X(-\infty)=P[X \leq -\infty] = 0 \!\,</math>


<math>F_X(\infty)=1 \Leftarrow F_X(\infty)=P[X \leq \infty]=1 \!\,</math>


"starts at 0 and ends at 1"


* b) For all <math>x' \geq x \!\,</math>, <math>F_X(x') \geq F_X(x) \!\,</math>


"non-decreasing in x"


<math> 
\begin{align}
F_X(x') &\quad F_X(x) \\
P[X \leq x'] &\quad P[X \leq x ] \\
P[s: X(s) \leq x'] &\quad P[s:X(s) \leq x] \\
\{s: X(s) \leq x \} &\subset \{s: X(s) \leq x' \} \\
P[X \leq x] &\leq P[X \leq x']\\
F_X(x) &\leq F_X(x')

\end{align}
\!\,</math>


* c) For all <math>x,x' \!\,</math>


<math>P[x \leq X \leq x'] = F_X(x')-F_X(x) \!\,</math>


"probabilities can be found by difference of the CDF"


<math>\{ s: X(x) \leq x' \} = \{ s: X(x) \leq x \} \cup \{s: x \leq X(s) \leq x' \} \!\,</math>


<math> P[X \leq x' ] = P[X \leq x ] + P [x \leq X \leq x']\!\,</math>


* d) For all <math>x \!\,</math>,


<math>\lim_{\epsilon \rightarrow 0} F_X(x+\epsilon)=F_X(x) \!\,</math>


"CDF is right continuous"


* e) For <math>x_i \in S_X \!\,</math>


<math>F_X(x_i)- F_X(x_i - \epsilon) = P_X(x_i) \!\,</math>


"For a discrete random variable, there is a jump (discontinuity) in the CDF at each value <math>x_i \in S_X \!\,</math>. This jump equals <math>P_X(x_i) \!\,</math>


<math>P[x_i \leq x] - P[x_i \leq x_i - \epsilon]=P[x_i- \epsilon < x < x_i] = P[X=x_i] \!\,</math>


* f) <math>F_X(x)=F_X(x_i) \!\,</math> for all <math>x_i \leq x \leq x_{i+1} \!\,</math>


"Between two jumps the CDF is constant"


<math>
\begin{align}
P[X \leq x] &= P[X \leq x_i \cup x_i < X \leq x ] \\
            &= P[X \leq x_i + \underbrace{P[x_i < X \leq x ]}_0 \\
            &= F_X(x_i)
\end{align}
</math>


* g) <math>P[X > x] = 1- \underbrace{F_X(x)}_{P[X \leq x]} \!\,</math>

==Continuous Random Variables==

outcomes uncountable many


===Example===

T: arrival of a partical


<math>S_T= \{t : 0 \leq t < \infty \} \!\,</math>


V: voltage


<math>S_V=\{v : -\infty < v < \infty \} \!\,</math>


<math>\theta \!\,</math>: angle


<math>S_{\theta}=\{ \theta: 0 \leq \theta \leq 2 \pi \} \!\,</math>


<math>X \!\,</math>: distance


<math>S_X= \{ X: 0 \leq x \leq 1 \} \!\,</math>


<math>P[x \in A] = \frac{1}{n } \rightarrow 0\!\,</math>


No PMF, <math>P[X=x]=0 \forall x \!\,</math>

===Theorem===
For any random variable (continuous or discrete)


* a) <math>F_X(-\infty)=0 \quad F_X(\infty)=1 \!\,</math>


* b) <math>F_X(x) \!\,</math> is nondecreasing in <math>X \!\,</math>


* c) <math>P[x < X \leq x'] = F_X(x')-F_X(x) \!\,</math>


* d) <math>F_X(x) \!\,</math> is right continuous


===Example===


<math>S_X=[0,1] \!\,</math>


<math>P_[x \in A]= P[ x\ in B]\!\,</math> where A, B are intervals of the same length contained in [0,1]


<math>P[X \leq 1] =1 \Leftrightarrow F_X(1)=1 \!\,</math>


<math>P[X \leq 0] =0 \Leftrightarrow F_X(0)=0 \!\,</math>


<math>P[x_1 < x < x_2]=P[0 < x < x_2-x_1] \!\,</math>


(exercise)<math>F_X(x_2)-F_X(x_1)=F_X(x_2-x_1) \rightarrow F_x(x)=x \!\,</math>


===Probability Density Function (PDF)===

<math>f_X(x)= \frac{dF_x(x)}{dx} \!\,</math>



discrete:  PMF <--> CDF  (sum/difference)


continuous   <--->  (derivative/integral)


===Theorem: Properties of PDF===

* a) <math>f_X(x) \geq 0 \!\,</math> (<math>F_X(x) \!\,</math> is nondecreasing) 


* b) <math>F_X(x) = \int_{\infty}^x f_X(x) dx \quad (F_X(\infty)=0)\!\,</math> 


* c) <math>\int_{-\infty}^\infty f_X(x)dx=1 \quad (F_x(\infty)=1) \!\,</math>

===Theorem===

<math> 
\begin{align}
P[x_1 \leq X \leq x_2] &= F_X(x_2)-F_X(x_1) \\
                       &= \int_{-\infty}^{x_2} f_X(x)dx - \int_{-\infty}^{x_1} f_X(x) dx \\
                       &= \int_{x_1}^{x_2} f_X(x)
\end{align}
\!\,</math>

===Some useful continuous Random Variables===

====Uniform R.V====


<math>f_X(x)=
\begin{cases}
\frac{1}{b-a} & a \leq x \leq b \\
0             & \mbox{otherwise}
\end{cases}

 \!\,</math>


====Exponential R.V====

<math>f_X(x)=ae^{-ax} \quad x \geq 0\!\,</math>


<math>F_x(x)=1-e^{-ax} \!\,</math>


====Gaussian (Normal) R.V.====


<math>\mathcal{N} (\mu, \sigma^2) \quad \infty < x< \infty\!\,</math>


<math>f_X(x)=\frac{1}{\sqrt{2\pi \sigma^2}}e^{-\frac{(x-\mu)^2}{2\sigma^2}} \!\,</math>


<math>F_X(x) = \int_{-\infty^1}\frac{1}{\sqrt{2 \pi \sigma^2}}e^{-\frac{x-\mu}{2\sigma^2}} \!\,</math>
{{BookCat}}