Approximate Discrete Distribution With Continuous X4

Approximation means that, under certain conditions, another distribution provides a description of the data which is similar to the distribution the data were sample from. The limit theorems (e.g. Central Limit Theorem) provide theoretical tool for deriving such approximations. These limit theorems can be used to approximate a number of common distributions. Since these are approximations to the true distribution there may be some errors in the approximation.  However, there are methods for evaluating the quality of the approximation. In the what follows we present approximations for a number of distributions as well as some of the criteria that can be used to evaluate the quality of these approximations. Normal distribution as limit of other distributions:

• Approximation of Binomial distribution by Normal distribution:

  This approximation is based on Laplace and DeMoivre's limit theorem.Let ${\displaystyle X_{1},\dots ,X_{n}}$ be independent, Bernoulli distributed random variables with ${\displaystyle E(X_{i})=p}$ and ${\displaystyle Var(X_{i})=p(1-p)}$ for all i. Then ${\displaystyle X=X_{1}+\dots +X_{n}}$ is random variable with Binomial distribution B(n,p), expected value E(X) = np and variance Var(X) = np(1-p).For ${\displaystyle n\rightarrow \infty }$ , the distribution of the standardized random variable

  ${\displaystyle Z={\frac {X-np}{\sqrt {np(1-p}}}}$

  converges to a standard Normal distribution N(0;1). For large n we have:

  ${\displaystyle X_{n}\approx N(np;{\sqrt {np(1-p)}})}$

  with the expected value ${\displaystyle \mu =np}$ and variance ${\displaystyle \sigma ^{2}=np(1-p)}$ .

  Since the Binomial distribution is discrete and the Normal distribution is continuous we improve the quality of approximation by using a continuity adjustment:

  ${\displaystyle P(X\leq x)=F_{B}(x;n,p)\approx \Phi \left({\frac {x+0.5-np}{\sqrt {np(1-p)}}}\right)}$

  ${\displaystyle P(X=x)=f_{B}(x;n,p)\approx \Phi \left({\frac {x+0.5-np}{\sqrt {np(1-p)}}}\right)-\Phi \left({\frac {x-0.5-np}{\sqrt {np(1-p)}}}\right)}$

  A rough rule of thumb for a good approximation for the Binomial distribution requires:np ${\displaystyle \geq }$ 5 or n(1-p) ${\displaystyle \geq }$ 5 .

• Approximation of the Poisson distribution by the Normal distribution

  The Poisson distribution with ${\displaystyle \lambda }$ = np can be derived from a Binomial distribution. Since the Binomial distribution can be approximated by the Normal distribution this suggests that the Normal distribution can also approximate the Poisson distribution.

  Let X be a random variable with the distribution PO( ${\displaystyle \lambda }$ ). Then for large ${\displaystyle \lambda ,}$ we approximate the Poisson distribution using a Normal distribution with expected value ${\displaystyle \mu =\lambda }$ and variance ${\displaystyle \sigma ^{2}=\lambda }$ (with the continuity correction):

  ${\displaystyle P(X\leq x)=F_{PO}(x;\lambda )\approx \Phi \left({\frac {x+0.5-\lambda }{\sqrt {\lambda }}}\right)}$

  The rule of thumb for a "reasonable " approximation requires: ${\displaystyle \lambda \geq }$ 10

• Approximation of Hypergeometric distribution by Normal distribution

  Let ${\displaystyle nM/N\geq 5,\ n(1-M/N)\geq 5}$ and ${\displaystyle n/M\leq 0.05}$ . Then a random variable with Hypergeometric distribution can be approximated using a Normal distribution with the parameters:

  ${\displaystyle E(X)=\mu =n\cdot {\frac {M}{N}}\quad Var(X)=\sigma ^{2}=n\cdot {\frac {M}{N}}\cdot \left(1-{\frac {M}{N}}\right)}$

  We can also use the continuity correction to improve the approximation.

• Approximation of Hypergeometric distribution by Binomial distribution

  The Binomial and Hypergeometric distributions use different sampling methods: the Binomial distribution uses draws with replacement and the Hypergeometric distribution uses draws without replacement. As M and N increase, M/N converges to a constant p, the difference between these two distributions becomes much smaller. As N ${\displaystyle \rightarrow \infty }$ (and M ${\displaystyle \rightarrow \infty }$ ) the Hypergeometric distribution converges to a Binomial distribution. This implies: for large N and M as well as small n/N , the Hypergeometric distribution can approximated by the Binomial distribution with parameters p = M/N. The rule of thumb requires: n/M ${\displaystyle \leq }$ 0.05 .

• Approximation of Binomial distribution by Poisson distribution

  The Poisson distribution can also derived from a Binomial distribution. Consequently, the Binomial distribution can also be approximated by a Poisson distribution PO( ${\displaystyle \lambda }$ =np), if n is large and the probability p is small.Rule of thumb: ${\displaystyle n>30}$ and p ${\displaystyle \leq }$ 0.05.

In a certain town, one house in each 100 is damaged every year because of storms. What is the probability that storms damage four houses in a year if the town contains 100 houses?For each house, there are only two possible outcomes – "damage" and "no damage". The probabilities of these outcomes are constant: ${\displaystyle p=0.01}$ and ${\displaystyle 1-p=0.99}$ . The random variable ${\displaystyle X}$ = {number of damaged houses} has the binomial distribution ${\displaystyle B(n,p)=B(100;\,0,01)}$ . We compute the probability ${\displaystyle P(X=4)}$ : ${\displaystyle P(X=4)=f_{B}(4;\,100;\,0,01)=\left({\begin{array}{c}100\\4\end{array}}\right)\cdot 0.01^{4}\cdot 0.99^{96}=0.01494\,.}$ We could also use the Poisson distribution (with parameter ${\displaystyle \lambda =np=1}$ ) to approximate this probability since the conditions for a good approximation are satisfied: ${\displaystyle F_{PO}(4;\,1)={\frac {1^{4}}{4\,!}}e^{-1}=0.01533\,.}$ We see that the probabilities ${\displaystyle f_{B}(4)}$ and ${\displaystyle F_{PO}(4)}$ are fairly close. More generally, the approximation is also good at other points in the distribution.

${\displaystyle x}$	${\displaystyle B(100;\,0,1)}$	${\displaystyle PO(1)}$
0	0.36603	0.36788
1	0.36973	0.36788
2	0.18486	0.18394
3	0.06100	0.06131
4	0.01494	0.01533
5	0.00290	0.00307
6	0.00046	0.00051
7	0.00006	0.00007
8	0.00000	0.00000

After a storm, there are damaged 300 houses out of total 2000 in a region. What is the probability that there are exactly 2 damaged houses in 10 randomly chosen houses? Again, there are only two possible outcomes for each house – "damage" and "no damage". Furthermore, ${\displaystyle N=2000}$ , ${\displaystyle M=300}$ , and ${\displaystyle N-M=1700}$ . The probability ${\displaystyle P(X=2)}$ is equal to ${\displaystyle P(X=2)=f_{H}(2)={\frac {\left({\begin{array}{c}300\\2\end{array}}\right)\cdot \left({\begin{array}{c}1700\\8\end{array}}\right)}{\left({\begin{array}{c}2000\\10\end{array}}\right)}}=0.2766\,.}$ This calculation is fairly demanding. Fortunately, we can use Binomial distribution (with parameter ${\displaystyle p=M/N=0.15)}$ to approximate this probability: ${\displaystyle P(X=2)-f_{B}(2)=\left({\begin{array}{c}10\\2\end{array}}\right)\cdot 0.15^{2}\cdot 0.85^{8}=0.2759\,.}$

Based on experience, we know that 10% of tax returns from a certain town have errors. Using a sample of 100 tax returns from this town – what is the probability that 12 of them contain errors? There are only two possible outcomes for the experiment – "wrong" or "correct", with corresponding probabilities ${\displaystyle p=0.1}$ and ${\displaystyle 1-p=0.9}$ . The random variables ${\displaystyle X}$ – "number of wrong tax returns from 100 randomly chosen ones" has the Binomial distribution ${\displaystyle B(n,p)=B(100;\,0.1)}$ . We need to compute the probability ${\displaystyle P(X=12)=f_{B}(12)}$ : ${\displaystyle f_{B}(12;\,100;\,0,1)=\left({\begin{array}{c}2000\\10\end{array}}\right)\cdot 0.1^{12}\cdot 0.9^{88}=0.0988\,.}$ If the value ${\displaystyle f_{B}(12;\,100;\,0,1)}$ is not contained in the tables, we would have to compute, which might be fairly cumbersome. However, since the conditions for the validity of an approximation using the Normal distribution are satisfied ( ${\displaystyle np=10\geq 5}$ and ${\displaystyle n(1-p)=90\geq 5}$ ), we could approximate the probability with a Normal distribution ${\displaystyle N(\mu ;\,\sigma )}$ . The expected value and the variance of the Binomial distribution are: ${\displaystyle \mu =np=100\cdot 0.1=10\,,\quad \sigma ^{2}=np(1-p)=100\cdot 0.1\cdot 0.9=9\,.}$ so we could use a ${\displaystyle N(10;\,3)}$ distribution (see the diagram). Recall: for a continuous random variable, the probabilities are given by the area under the density and thus the probability of one specific value is always equal to zero, e.g., ${\displaystyle P(X=12)=0}$ . Therefore, we subtract and add 0.5 to 12; this is a sort of continuity correction. Instead of ${\displaystyle x=12}$ (for the discrete variable) we use an interval for the continuous ${\displaystyle 11.5\leq x\leq 12.5}$ and ${\displaystyle f_{B}(12;\,100;\,0,1)}$ is then approximated by ${\displaystyle P(11.5\leq x\leq 12.5)}$ , i.e., the area under the density ${\displaystyle N(10;\,3)}$ between the points 11.5 and 12.5.

The tables only contain the distribution function of a ${\displaystyle N(0,1)}$ random variable so we have to standardize the random variable ${\displaystyle X}$ : ${\displaystyle z_{1}=(12.5-10)/3=0.83\ {\text{and}}\ z_{2}=(11.2-10)/3=0.5\,.}$ Using the Normal tables, we obtain ${\displaystyle \Phi (0.83)=0.7967}$ and ${\displaystyle \Phi (0.5)=0.6915}$ .Hence, ${\displaystyle P(11.5\leq x\leq 12.5)=\Phi (0.83)-\Phi (0.5)=0.7967-0.6915=0.1052\,.}$ The approximation works reasonably well, the error of the approximation is only 0.1052 - 0.0988 = 0.0064. We can also see that:

lietzelable73.blogspot.com

Source: https://wikis.hu-berlin.de/mmint/Basics:_Approximation_of_Distributions/en

Share this post