UMD Probability Qualifying Exams/Aug2009Probability

Problem 1

Let $X_1,...,X_n$ be i.i.d. random variables with moment generating function $M(t)=E[\exp (t{X}_1)]$ which is finite for all $t$. Let ${\tilde{X}}_n=(X_1+\cdots +X_n)/n$. (a) Prove that $P[X_1>a]\le exp[-h(a)]$ where $h(a)=\underset{t\ge 0}{\sup }[at-\psi (t)]$ and $\psi (t)=\log M(t)$ (b) Prove that $P[{\tilde{X}}_n\ge a]\le \exp [-nh(a)]$. (c) Assume $E[X_1]=0$. Use the result of (b) to establish that ${\tilde{X}}_n\to 0$ almost surely.

Solution

(a) $\begin{array}{cc}P[X_1>a]=& \underset{X_1>a}{\int }1dF=\underset{X_1>a}{\int }\frac{\exp (ta)}{\exp (ta)}dF\\ =& e^{-at}\underset{X_1>a}{\int }{e}^{at}dF\le e^{-at}\underset{X_1>a}{\int }{e}^{X_{1}t}dF\\ \le & e^{-at}\underset{\mathrm{\Omega }}{\int }{e}^{X_{1}t}dF=e^{-at}E[\exp (t{X}_1)]\end{array}$

Thus far, we have not imposed any conditions on $t$. So the above inequality will hold for all $t$, hence for the supremum as well, which gives us the desired result.

(b) $\begin{array}{cc}P[{\tilde{X}}_n>a]=& \underset{{\tilde{X}}_n>a}{\int }1dF=\underset{{\tilde{X}}_n>a}{\int }\frac{\exp (nta)}{\exp (nta)}dF\\ =& e^{-nat}\underset{\sum _{i=1}^n{X}_i>an}{\int }{e}^{nat}dF\le e^{-nat}\underset{\sum _{i=1}^n{X}_i>an}{\int }{e}^{\sum _{i=1}^n{X}_{i}t}dF\\ \le & e^{-nat}\underset{\mathrm{\Omega }}{\int }{e}^{\sum _{i=1}^n{X}_{i}t}dF=e^{-nat}(\underset{\mathrm{\Omega }}{\int }{e}^{X_{i}t}dF{)}^n\end{array}$ where the last equality follows from the fact that the $X_i$ are independent and identically distributed.

(c)

Problem 2

Let $(\mathrm{\Omega },ℱ,P)$ be a probability space; let $X$ be a random variable with finite second moment and let ${𝒢}_1\subset {𝒢}_2$ be sub $\sigma$-fields. Prove that $E[(X-E(X|{𝒢}_2){)}^2]\le E[(X-E(X|{𝒢}_1){)}^2].$

Problem 3

Let $N_1(t),N_2(t)$ be independent homogeneous Poisson processes with rates ${\lambda }_1,{\lambda }_2$, respectively. Let $Z$ be the time of the first jump for the process $N_1(t)+N_2(t)$ and let $J$ be the random index of the component process that made the first jump. Find the joint distribution of $(J,Z)$. In particular, establish that $J,Z$ are independent and that $Z$ is exponentially distributed.

Solution

Show $Z$ is exponentially distributed

Let $\tau$ be the first time that a Poisson process $N(t)$ jumps.

$\begin{array}{cc}{p}_{\tau }(x)=\underset{ϵ\to 0}{\lim }\frac{F_{\tau }(x)-F_{\tau }(x-ϵ)}{ϵ}& =\underset{ϵ\to 0}{\lim }\frac{P(N(x)>0\cap N(x-ϵ)=0)}{ϵ}\\ =& \underset{ϵ\to 0}{\lim }1/ϵP(N(x-ϵ)=0)\cdot P(N(x)-N(x-ϵ)>0)\\ =& \underset{ϵ\to 0}{\lim }1/ϵe^{-\lambda (x-ϵ)}\cdot \frac{\lambda ϵ}{1}{e}^{-\lambda ϵ}=\lambda e^{-\lambda x}\end{array}$

$N_1(t)+N_2(t)$ is a Poisson Process with parameter ${\lambda }_1+{\lambda }_2$

Proof: There are three conditions to check:

(i) $N_1(0)+N_2(0)=0$ almost surely

(ii) For $t>s$ is $(N_1(t)+N_2(t)-N_1(s)-N_2(s)$ independent of $N_1(s)+N_2(s)$? This is true since both $N_1,N_2$ are Poisson Processes and are both independent of each other.

(iii) For $t>s$ is $(N_1(t)+N_2(t)-N_1(s)-N_2(s)$ distributed Poisson with parameter $t-s$? This is true since the sum of independent Poisson processes are also poison. (see second bullet)

Joint distribution of (J,Z)

$P(J=1,Z=x)={\lambda }_1{e}^{-{\lambda }_{1}x}$

$P(J=2,Z=x)={\lambda }_2{e}^{-{\lambda }_{2}x}$

Problem 4

Let $(X_n,{ℱ}_n)$ be a martingale sequence and for each $n$ let ${ϵ}_n$ be an ${ℱ}_{n-1}$-measurable random variable. Define $Y_n=\sum _{i=1}^n{ϵ}_i(X_i-X_{i-1}),Y_0=0$ Assuming that $Y_n$ is integrable for each $n$, show that $Y_n$ is a martingale.

Problem 5

Let $X_1,...,X_n$ be an i.i.d. sequence with $E[X_i]=0$ and $V[X_i]={\sigma }^2<\mathrm{\infty }$. Prove that for any $\gamma >1/2$, the series $\sum _{k=1}^{\mathrm{\infty }}{X}_k/k^{\gamma }$ converges almost surely.

Solution

Define $Z_k:=X_k/k^{\gamma }$. Then $E[Z_k]=0$ and $V[Z_k]=\frac{{\sigma }^2}{k^{2\gamma }}$. We check the three components of Kolmogorov's three-series theorem to conclude that $\sum _{k=1}^{\mathrm{\infty }}{Z}_k$ converges almost surely.

Problem 6

Consider the following process $\{X_n\}$ taking values in $\{0,1,...\}$. Assume $U_n,n=1,2,...$ is an i.i.d. sequence of positive integer valued random variables and let $X_0$ be independent of the $U_n$. Then $X_n=\{\begin{array}{cc}{X}_{n-1}-1& \text{if }{X}_{n-1}\ne 0\\ U_k-1& \text{if }{X}_{n-1}=0\text{ for the kth time}\end{array}$