This is a summary of common probability distributions in engineering and statistics. This chart has the plots of the pdf or pmf (LaTeX source):

# discrete distributions

binomial distribution

- A big urn with balls in either white or black color. Drawing a white ball from urn has probability (i.e., black ball has probability ). If we draw balls from urn with replacement, the probability of getting white balls:

Poisson distribution

- Balls are added to the urn at rate of per unit time, under exponential distribution. The probability of having balls added to the urn within time :

geometric distribution

- The probability of have to draw balls to see the first white ball being drawn:

negative binomial distribution

- same as the distribution of the sum of iid geometric random variable
- negative binomial approximates Poisson with with large and
- Drawing balls from the urn. If we have to draw balls to see the -th white ball (we have drawn white balls and black balls). The probability of :

hypergeometric distribution

- A urn with balls (finite) and balls amongst are white. Draw, without replacement, balls from the urn to get white balls:

# continuous distributions

uniform distribution

- extreme of flattened distribution
- with upper and lower bounds

triangular distribution

- with upper and lower bounds

normal distribution

- strong tendency for data at central value; symmetric, equally likely for positive and negative deviations from its central value
- frequency of deviations falls off rapidly as we move further away from central value

- approximation to Poisson distribution: if is large, Poisson distribution approximates normal with
- approximation to binomial distribution: if is large and , binomial distribution approximates normal with and
- approximation to beta distribution: if and are large, beta distribution approximates normal with and

Laplace distribution

- absolute difference from mean compared to squared difference in normal distribution
- longer (fatter) tails, higher kurtosis (flattened peak)
- pdf:

logistic distribution

- symmetric, with longer tails and higher kurtosis than normal distribution
- logistic distribution has finite mean and variance defined
- logistic pdf:

Cauchy distribution

- symmetric, with longer tails and higher kurtosis than normal distribution
- Cauchy distribution has mean and variance undefined, but mean & mode at
- Cauchy pdf:

lognormal distribution

- , positively skewed
- parameterised by shape (), scale (, or median), shift ()
- is standard lognormal distribution
- as rises, the peak shifts to left and skewness increases

- sum of two lognormal random variable is a lognormal random variable with and

Pareto distribution

- power law probability distribution
- continuous counterpart of Zipf’s law
- positively skewed, no negative tail, peak at

gamma distribution

- support for , positive skewness (lean left)
- decreasing will push distribution towards the left; at low , left tail will disappear and distribution will resemble exponential
- models the time to the -th Poisson arrival with arrival rate
- gamma pdf ( becomes exponential pdf with rate ):

Weibull distribution

- support for , positive skewness (lean left)
- decreasing will push distribution towards the left; at low , left tail will disappear and distribution will resemble exponential
- If , then
- Weibull pdf ( becomes exponential pdf with rate ):

Erlang distribution

- arise from teletraffic engineering: time to -th call

beta distribution

- support for
- allows negative skewness
- two shape parameters and , and lower- and upper-bounds on data ( and )

extreme value distribution (i.e. Gumbel minimum distribution)

- negatively skewed
- Gumbel maximum distribution, , is positively skewed
- Limiting distribution of the max/min value of iid samples from with
- standard cdf:

Rayleigh distribution

- positively skewed
- modelling the -norm of two iid normal distribution with zero mean (e.g., orthogonal components of a 2D vector)

Maxwell-Boltzmann distribution

- positively skewed
- 3D counterpart of Rayleigh distribution
- arise from thermodynamic: probability of a particle in speed if temperature is

Chi-squared distribution

- distribution of the sum of the square of i.i.d. standard normal random variables
- mean , variance
- PDF with degrees of freedom:

F-distribution

- Distribution of a random variable defined as the ratio of two independent -distributed random variables, with degrees of freedom and respectively
- Commonly used in ANOVA
- PDF, with degrees of freedom and , involves beta function :

Student’s t distribution

- Distribution of
*normalized*sample mean of observations from a normal distribution, - Equivalently, this is the distribution of for is standard normal and is chi-square with degrees of freedom
- t distribution with is Cauchy distribution
- PDF with degree of freedom :

# test of fit for distributions

Kolmogorov-Smirnov test (K-S test, on cumulative distribution function )

- if sample comes from distribution, converges to 0 a.s. as number of samples goes to infinity

Shapiro-Wilk test

- test of normality in frequentist statistics (i.e. for in normal distribution)
- is the sample mean
- where is vector of expected values of the order statistics from normal distribution and the covariance matrix of those order statistics

Anderson-Darling test

- test whether a sample comes from a specified distribution
- is weighted distance between and , with more weight on tails of the distribution

Pearson’s test

- test for categories fit a distribution: checking observed frequency against expected frequency according to distribution for each of categories
- degree of freedom: minus number of parameters of the fitted distribution

# Reference

Lawrence M. Leemis and Jacquelyn T. McQuestion. Univariate Distribution Relationships, Am Stat, 62(1) pp.45–53, 2008, DOI: 10.1198/000313008X270448

Aswath Damodaran. Probabilistic approaches: Scenario analysis, decision trees and simulations (PDF, the appendix is also available separately) and includes the following chart for choosing a distribution: