Story

The Exponential distribution is the continuous counterpart to the Geometric distribution. The story of the Exponential distribution is analogous, but we are now waiting for a success in continuous time, where successes arrive at a rate of λ successes per unit of time. The average number of successes in a time interval of length t is λt, though the actual number of successes varies randomly. An Exponential random variable represents the waiting time until the first arrival of a success.

——adapted from Book BH

Basic

Definition: A continuous r.v. X is said to have the Exponential distribution with parameter λ if its PDF is f(x)=λeλx,x>0 The corresponding CDF is

F(x)=1eλx,x>0

To calculate the expectation and variance, we first consider XExp(1) with PDF f(x)=ex, then

E(X)=0xexdx=1E(X2)=0x2exdx=x2ex|0+20xexdx=2E(X)=2Var(X)=E(X2)E2(X)=21=1MX(t)=E(etX)=0etxexdx=0e(1t)xdx=11tfor t<1

Now let Y=XλExp(λ) for

fY(y)=fX(X(y))dxdy=eλyλExp(λ)

or

P(Yy)=P(Xλy)=1eλyExp(λ).

Hence, we can get

  • E(Y)=E(X/λ)=1/λ
  • Var(Y)=Var(X/λ)=1/λ2
  • MGF (moment generating function):

MY(t)=E(etY)=E(etX/λ)=E(etλX)=MX(tλ)=11t/λ=λλtfor t<λ

Memeoryless Property

Memoryless is something like $P(X \ge s+t | X \ge s) = P(X \ge t),letX \sim Exp(\lambda)$, then

$$ \begin{split} P(X \ge s+t | X \ge s) &= \frac{P(X \ge s+t, ~X \ge s)}{P(X \ge s)} \newline &= \frac{P(X \ge s+t)}{P(X \ge s)} \newline &= \frac{e^{-\lambda (s+t)}}{e^{-\lambda s}} = e^{-\lambda t} \newline &= P(X \ge t) \end{split} $$

Theorem: If X is a positive continuous r.v. with memoryless property, then X has an exponential distribution. Similarly, if X is discrete, then it has a geometric distribution.

Proof idea: use survival function and solve differential equations.

Examples

eg.1 X1Exp(λ1), X2Exp(λ2), and X1X2. Then P(X1<X2)=λ1λ1+λ2.

Proof: By LOTP (law of total probability),

$$ \begin{split} P(X_1 < X_2) &= \int_0^{\infty} f_{X_1}(x) P(X_2 > X_1 | X_1=x) dx \newline &= \int_0^{\infty} f_{X_1}(x) P(X_2 > x | X_1=x) dx \newline &= \int_0^{\infty} f_{X_1}(x) P(X_2 > x) dx \quad \text{(independence)} \newline &= \int_0^{\infty} \lambda_1 e^{-\lambda_1 x} e^{-\lambda_2 x} dx \newline &= \lambda_1 \int_0^{\infty} e^{-(\lambda_1 + \lambda_2) x} dx \newline &= \frac{\lambda_1}{\lambda_1 + \lambda_2} \end{split} $$

eg.2 {Xi}i=1n are independent with XjExp(λj). Let L=min(X1,,Xn), then LExp(λ1+λn).

Proof:

P(L>t)=P(min(X1,,Xn)>t)=P(X1>t,,Xn>t)=P(X1>t)P(Xn>t)indep.=eλ1teλnt=e(λ1+λn)tExp(jλj)

The intuition of this result is that if you consider n Poisson processes with rate λj,

  • X1 as the waiting time for a green car
  • X2 as the waiting time for a red car

Then L is the waiting time for a car of any color (i.e., any car). So it makes sense, the rate is λ1++λn.

eg.3 (Difference of two exponetial) Let XExp(λ) and YExp(μ), XY. Then what is the PDF of Z=XY?

Solution: Recall the story of exponential, one can think of X and Y as waiting times for two independent things. For example,

  • X as the waiting time for a red car passing by
  • Y as the waiting time for a blue car

If we see a blue car passing by, then the further waiting time for a red car is still distributed as same distribution as Y, for the memoryless property of exponential. Likewise, if we see a red car passing by, then the further waiting time is distributed as same as X. The further waiting time is somehow what we are interested in, say Z.

The above intuition says that, the conditional distribution of XY given X>Y is the distribution of X, and the conditional distribution of XY given XY is the distribution of Y (or in other words, the conditional distribution of YX given YX is same as the distribution of Y).

To make full use of our intuition, we know that

  • If X>Y, which means Z>0, then $Z~|X>Y = Xa.s.holds,thatis$ \begin{gathered} f_Z(z|X>Y) = \lambda e^{-\lambda z} \newline \text{and since }P(X<Y) = 0 \newline \implies f_Z(z) = f_Z(z|~X>Y)P(X>Y) \newline = \frac{\mu}{\lambda + \mu}\lambda e^{-\lambda z}. \end{gathered} $$

  • If X<Y, which means Z<0, then $Z~|X<Y = -Ya.s.holds,thatis$ \begin{gathered} f_Z(z|X<Y) = f_Y(y(z))\left|\frac{dy}{dz}\right| = \mu e^{\mu z} \newline \implies f_Z(z) = f_Z(z|~X<Y)P(X<Y) \ = \frac{\lambda}{\lambda + \mu} \mu e^{\mu z} \end{gathered} $$

However, this is just a sketch. Later we will see how to derivate the form mathematically.

From the above point of view, the PDF of Z had better be discussed by the sign of Z.

  • If Z>0, which implies X>YP(X>Y)=0, then

$$ \begin{split} P(Z > z) &= P(X-Y>z | X>Y)P(X>Y) + P(Z>z~|~X<Y)P(X<Y) \newline &= P(X>z)P(X>Y) + 0 \quad \text{(memoryless)} \newline &= \frac{\mu}{\lambda + \mu} e^{-\lambda z} \quad \text{(by eg.1)} \newline \implies f_Z(z) &= \frac{\lambda\mu}{\lambda + \mu} e^{-\lambda z} \quad \text{for }z>0 \end{split} $$

  • If Z0, which implies XY, then

$$ \begin{split} P(Z < z) &= P(Z<z | X>Y)P(X>Y) + P(X-Y<z~|~X<Y)P(X<Y) \newline &= 0 + P(Y-X > -z | Y>X)P(Y>X) \newline &= P(Y>X)P(Y > -z) \quad \text{(memoryless)} \newline &= \frac{\lambda}{\lambda + \mu}e^{\mu z} \quad \text{(by eg.1)} \newline \implies f_Z(z) &= \frac{\lambda\mu}{\lambda + \mu}e^{\mu z} \quad \text{for }z<0 \end{split} $$

Therefore, the PDF of Z has the form

fZ(z)=λμλ+μ{eλzz>0eμzz<0

Note: P(X=Y)=0 since the integral domain is a line (y=x) whose measure is 0. That is P(Z=0)=0. This is why we can give no care of the case X=Y.