Distributions

Discrete univariate with infinite support

Rademacher

1 or -1 with probability 0.5 each.

Bernoulli

1 with probability p and 0 with probability 1-p. Uncorrelatedness implies independence.

Negative binomial distribution

Models the number of failures in a sequence of iid Bernoulli trials before a specified (non-random) number of successes (denoted r) occurs.

Continuous univariate supported on a semi-infinite interval

A power-law distribution that can be used to describe many observable phenomena, such as the distribution of wealth in a society.

Continuous univariate with infinite support

Normal

Can transform a uniform [0,1) rv to normal by using

Derivative

Functions:

Student’s t

Suppose X \sim N(\mu, \sigma^2); take a sample of n iid variables from this distribution. Let \bar{X} be the sample mean, and S^2 be the Bessel-corrected sample variance.

Then \frac {\bar{X} - \mu }{\sigma / \sqrt{n}} has a standard normal distribution and \frac {\bar{X} - \mu }{S / \sqrt{n}} has t-distribution with \nu = n-1 degrees of freedom.

The t-distribution can be defined as the distribution of the RV T with T = \frac{Z}{\sqrt{V / \nu}}, where

The multivariate t-distribution has marginal distributions that are also t-distributed. That is if, X \sim t_p(\mu, \Sigma, v), where \mu is the location and \Sigma is the scale matrix, then X_i \sim t(\mu_i, \Sigma_{ii}, v). The mean of the distribution is \mu and its variance is \frac{v}{v-2}\Sigma (if v>2).

The sum of two independent t-distributed random variables is not t-distributed.

Multivariate

Wishart

A generalization to multiple dimensions of the gamma distribution.

It is a family of probability distributions defined over symmetric, nonnegative-definite random matrices (i.e. matrix-valued random variables). In random matrix theory, the space of Wishart matrices is called the Wishart ensemble.

It arises as the distribution of the sample covariance matrix (estimated using the MLE) for a sample from a multivariate normal distribution.

Location-Scale family

Family of probability distributions parametrized by a location parameter and a non-negative scale parameter.

Given the “standard” CDF of a RV, the location-scale family CDF can be written as cdf_ls(x, df, loc, scale) = cdf((x - loc)/scale, df)

References