Discrete Random Variables

Properties

Probability Mass Function

• Describes the distribution of a discrete random variable
• $p_X(x)=P(X=x)$ for every $x\in \mathcal{X}$
• Properties
  • $p_X(x)\ge 0$
  • $\sum _{x\in \mathcal{X}}{p}_X(x)=1$

Cumulative Distribution Function

• The culmination of the random values up to a point
• $F_X(b)=P(X\le b)$ for all $b\in \mathbb{R}$
• $F_X(b)=\sum _{x\in \mathcal{X}:x\le b}{p}_X(x)$

Discrete Expectation

• $\mathbb{E}(X)=\sum _{x\in \mathcal{X}}x{p}_X(x)$
• If $X$ is a discrete random random variable and $g:\mathbb{R}\to \mathbb{R}$ then $\mathbb{E}[g(X)]=\sum _{x\in \mathcal{X}}g(x)p_X(x)$
• Note: $\mathbb{E}(X)$ is only well defined if $\sum _{x\in \mathcal{X}}|x|p_X(x)<\infty$
  • If $\mathcal{X}$ is not a finite space, make sure the absolute expectation is finite

Distributions

Bernoulli

• Bernoulli Experiment/Trial = an experiment with 2 possible outcomes (success/failure)
• Define $X$ to be 1 on success and 0 on failure
• Parameter $p$ is the probability of success
  • A random variable is Bernoulli if the only parameter needed is $p$
• Properties
  • $\mathbb{E}(X)=p$
  • $V(X)=p(1-p)$

Binomial

• $X$ = number of total successes in $n$ independent Bernoulli trials
• Properties
  • $p_X(x)=\left(\genfrac{}{}{0ex}{}{n}{x}\right)p^x(1-p{)}^{n-x}$
  • $\mathbb{E}(X)=np$
  • $V(X)=np(1-p)$

Geometric

• $X$ = number of failures before a success in repeated Bernoulli trials
• Properties
  • $p_X(x)=p(1-p{)}^x$
  • $F_X(x)=1-(1-p{)}^{x+1}$
  • $\mathbb{E}(x)=\frac{1-p}{p}$
  • $\mathbb{V}(x)=\frac{1-p}{p^2}$
• Memoryless Property
  • If there have been $j$ failures, the odds of having $k$ more failures is the same as starting fresh and having $k$ failures
  • $P(X\ge j+k|X\ge j)=P(X\ge k)$
  • Only type of discrete random variable with this property

Negative Binomial

• $X$ = number of failures before $k^{th}$ success in repeated Bernoulli trials
• Properties
  • $p_X(x)=\left(\genfrac{}{}{0ex}{}{x+k-1}{x}\right)p^k(1-p{)}^x$
  • $\mathbb{E}(x)=k\frac{1-p}{p}$
  • $V(x)=k\frac{1-p}{p^2}$
• Can be expressed as the sum of $k$ geometric random variables

Possion

• Defined as the limit of the binomial distribution when $n\to \infty ,p\to 0,np\to \lambda >0$
• Parameter is $\lambda$ (the mean)
• Useful for modeling rare events
• Properties
  • $p_X(x)=\frac{e^{-\lambda }{\lambda }^x}{x!}$
  • $\mathbb{E}(X)=\lambda$
  • $V(X)=\lambda$
• The variance and expected values being the same makes this less useful

Hypergeometric

• Sampling with replacement
  • Bernoulli trials are sampling with replacement
• $N$ items. $k$ type 1 and $N-k$ type 2. $n$ chosen
• $X$ = number of type 1 items in the $n$ chosen
• Properties
  • $p_X(x)=\frac{\left(\genfrac{}{}{0ex}{}{k}{x}\right)\left(\genfrac{}{}{0ex}{}{N-k}{n-x}\right)}{\left(\genfrac{}{}{0ex}{}{N}{n}\right)},x=0\dots k$
  • $\mathbb{E}(X)=\frac{nk}{N}$
  • $V(X)=n\frac{k}{N}(1-\frac{k}{N})(\frac{N-n}{N-1})$
• Binomial Approximation
  • As $N\to \infty$ and $\frac{k}{N}\to p$, $p_X(x)\to \left(\genfrac{}{}{0ex}{}{n}{x}\right)p^x(1-p{)}^{n-x}$
  • Hence, if $N$ is very large, it can be approximated by a binomial random variable with parameters $n$ and $p=k/n$