This is a summary of common probability distributions in engineering and statistics. This chart has the plots of the pdf or pmf (LaTeX source):

discrete distributions

binomial distribution

  • A big urn with balls in either white or black color. Drawing a white ball from urn has probability (i.e., black ball has probability ). If we draw balls from urn with replacement, the probability of getting white balls:

Poisson distribution

  • Balls are added to the urn at rate of per unit time, under exponential distribution. The probability of having balls added to the urn within time :

geometric distribution

  • The probability of have to draw balls to see the first white ball being drawn:

negative binomial distribution

  • same as the distribution of the sum of iid geometric random variable
  • negative binomial approximates Poisson with with large and
  • Drawing balls from the urn. If we have to draw balls to see the -th white ball (we have drawn white balls and black balls). The probability of :

hypergeometric distribution

  • A urn with balls (finite) and balls amongst are white. Draw, without replacement, balls from the urn to get white balls:

continuous distributions

uniform distribution

  • extreme of flattened distribution
  • with upper and lower bounds

triangular distribution

  • with upper and lower bounds

normal distribution

  • strong tendency for data at central value; symmetric, equally likely for positive and negative deviations from its central value
  • frequency of deviations falls off rapidly as we move further away from central value
  • approximation to Poisson distribution: if is large, Poisson distribution approximates normal with
  • approximation to binomial distribution: if is large and , binomial distribution approximates normal with and
  • approximation to beta distribution: if and are large, beta distribution approximates normal with and

Laplace distribution

  • absolute difference from mean compared to squared difference in normal distribution
  • longer (fatter) tails, higher kurtosis (flattened peak)
  • pdf:

logistic distribution

  • symmetric, with longer tails and higher kurtosis than normal distribution
  • logistic distribution has finite mean and variance defined
  • logistic pdf:

Cauchy distribution

  • symmetric, with longer tails and higher kurtosis than normal distribution
  • Cauchy distribution has mean and variance undefined, but mean & mode at
  • Cauchy pdf:

lognormal distribution

  • , positively skewed
  • parameterised by shape (), scale (, or median), shift ()
  • is standard lognormal distribution
  • as rises, the peak shifts to left and skewness increases
  • sum of two lognormal random variable is a lognormal random variable with and

Pareto distribution

  • power law probability distribution
  • continuous counterpart of Zipf’s law
  • positively skewed, no negative tail, peak at

gamma distribution

  • support for , positive skewness (lean left)
  • decreasing will push distribution towards the left; at low , left tail will disappear and distribution will resemble exponential
  • models the time to the -th Poisson arrival with arrival rate
  • gamma pdf ( becomes exponential pdf with rate ):

Weibull distribution

  • support for , positive skewness (lean left)
  • decreasing will push distribution towards the left; at low , left tail will disappear and distribution will resemble exponential
  • If , then
  • Weibull pdf ( becomes exponential pdf with rate ):

Erlang distribution

  • arise from teletraffic engineering: time to -th call

beta distribution

  • support for
  • allows negative skewness
  • two shape parameters and , and lower- and upper-bounds on data ( and )

extreme value distribution (i.e. Gumbel minimum distribution)

  • negatively skewed
  • Gumbel maximum distribution, , is positively skewed
  • Limiting distribution of the max/min value of iid samples from with
  • standard cdf:

Rayleigh distribution

  • positively skewed
  • modelling the -norm of two iid normal distribution with zero mean (e.g., orthogonal components of a 2D vector)

Maxwell-Boltzmann distribution

  • positively skewed
  • 3D counterpart of Rayleigh distribution
  • arise from thermodynamic: probability of a particle in speed if temperature is

Chi-squared distribution

  • distribution of the sum of the square of i.i.d. standard normal random variables
  • mean , variance
  • PDF with degrees of freedom:

F-distribution

  • Distribution of a random variable defined as the ratio of two independent -distributed random variables, with degrees of freedom and respectively
  • Commonly used in ANOVA
  • PDF, with degrees of freedom and , involves beta function :

Student’s t distribution

  • Distribution of normalized sample mean of observations from a normal distribution,
  • Equivalently, this is the distribution of for is standard normal and is chi-square with degrees of freedom
  • t distribution with is Cauchy distribution
  • PDF with degree of freedom :

test of fit for distributions

Kolmogorov-Smirnov test (K-S test, on cumulative distribution function )

  • if sample comes from distribution, converges to 0 a.s. as number of samples goes to infinity

Shapiro-Wilk test

  • test of normality in frequentist statistics (i.e. for in normal distribution)
  • is the sample mean
  • where is vector of expected values of the order statistics from normal distribution and the covariance matrix of those order statistics

Anderson-Darling test

  • test whether a sample comes from a specified distribution
  • is weighted distance between and , with more weight on tails of the distribution

Pearson’s test

  • test for categories fit a distribution: checking observed frequency against expected frequency according to distribution for each of categories
  • degree of freedom: minus number of parameters of the fitted distribution

Reference

Lawrence M. Leemis and Jacquelyn T. McQuestion. Univariate Distribution Relationships, Am Stat, 62(1) pp.45–53, 2008, DOI: 10.1198/000313008X270448

Aswath Damodaran. Probabilistic approaches: Scenario analysis, decision trees and simulations (PDF, the appendix is also available separately) and includes the following chart for choosing a distribution: