Exponential Gamma and Normal Continuous Distributions

Let us address the question posed, This is all somewhat mysterious to me. Is the normal distribution fundamental to the derivation of the gamma distribution...? No mystery really, it is simply that the normal distribution and the gamma distribution are members, among others of the exponential family of distributions, which family is defined by the ability to convert between equational forms by substitution of parameters and/or variables. As a consequence, there are many conversions by substitution between distributions, a few of which are summarized in the figure below.

LEEMIS, Lawrence M.; Jacquelyn T. MCQUESTON (February 2008). "Univariate Distribution Relationships" (PDF). American Statistician. 62 (1): 45–53. doi:10.1198/000313008x270448 cite

Here are two normal and gamma distribution relationships in greater detail (among an unknown number of others, like via chi-squared and beta).

First A more direct relationship between the gamma distribution (GD) and the normal distribution (ND) with mean zero follows. Simply put, the GD becomes normal in shape as its shape parameter is allowed to increase. Proving that that is the case is more difficult. For the GD, $$\text{GD}(z;a,b)=\begin{array}{cc} & \begin{cases} \dfrac{b^{-a} z^{a-1} e^{-\dfrac{z}{b}}}{\Gamma (a)} & z>0 \\ 0 & \text{other} \\ \end{cases} \,. \\ \end{array}$$

As the GD shape parameter $a\rightarrow \infty$, the GD shape becomes more symmetric and normal, however, as the mean increases with increasing $a$, we have to left shift the GD by $(a-1) \sqrt{\dfrac{1}{a}} k$ to hold it stationary, and finally, if we wish to maintain the same standard deviation for our shifted GD, we have to decrease the scale parameter ($b$) proportional to $\sqrt{\dfrac{1}{a}}$.

To wit, to transform a GD to a limiting case ND we set the standard deviation to be a constant ($k$) by letting $b=\sqrt{\dfrac{1}{a}} k$ and shift the GD to the left to have a mode of zero by substituting $z=(a-1) \sqrt{\dfrac{1}{a}} k+x\ .$ Then $$\text{GD}\left((a-1) \sqrt{\frac{1}{a}} k+x;\ a,\ \sqrt{\frac{1}{a}} k\right)=\begin{array}{cc} & \begin{cases} \dfrac{\left(\dfrac{k}{\sqrt{a}}\right)^{-a} e^{-\dfrac{\sqrt{a} x}{k}-a+1} \left(\dfrac{(a-1) k}{\sqrt{a}}+x\right)^{a-1}}{\Gamma (a)} & x>\dfrac{k(1-a)}{\sqrt{a}} \\ 0 & \text{other} \\ \end{cases} \\ \end{array}\,.$$

Note that in the limit as $a\rightarrow\infty$ the most negative value of $x$ for which this GD is nonzero $\rightarrow -\infty$. That is, the semi-infinite GD support becomes infinite. Taking the limit as $a\rightarrow \infty$ of the reparameterized GD, we find

$$\lim_{a\to \infty } \, \frac{\left(\frac{k}{\sqrt{a}}\right)^{-a} e^{-\frac{\sqrt{a} x}{k}-a+1} \left(\frac{(a-1) k}{\sqrt{a}}+x\right)^{a-1}}{\Gamma (a)}=\dfrac{e^{-\dfrac{x^2}{2 k^2}}}{\sqrt{2 \pi } k}=\text{ND}\left(x;0,k^2\right)$$

Graphically for $k=2$ and $a=1,2,4,8,16,32,64$ the GD is in blue and the limiting $\text{ND}\left(x;0,\ 2^2\right)$ is in orange, below

Second Let us make the point that due to the similarity of form between these distributions, one can pretty much develop relationships between the gamma and normal distributions by pulling them out of thin air. To wit, we next develop an "unfolded" gamma distribution generalization of a normal distribution.

Note first that it is the semi-infinite support of the gamma distribution that impedes a more direct relationship with the normal distribution. However, that impediment can be removed when considering the half-normal distribution, which also has a semi-infinite support. Thus, one can generalize the normal distribution (ND) by first folding it to be half-normal (HND), relating that to the generalized gamma distribution (GD), then for our tour de force, we "unfold" both (HND and GD) to make a generalized ND (a GND), thusly.

The generalized gamma distribution

$$\text{GD}\left(x;\alpha ,\beta ,\gamma ,\mu \right)=\begin{array}{cc} & \begin{cases} \dfrac{\gamma e^{-\left(\dfrac{x-\mu }{\beta }\right)^{\gamma }} \left(\dfrac{x-\mu }{\beta }\right)^{\alpha \gamma -1}}{\beta \,\Gamma (\alpha )} & x>\mu \\ 0 & \text{other} \\ \end{cases} \\ \end{array}\,,$$

Can be reparameterized to be the half-normal distribution,

$$\text{GD}\left(x;\frac{1}{2},\frac{\sqrt{\pi }}{\theta },2,0 \right)=\begin{array}{cc} & \begin{cases} \dfrac{2 \theta e^{-\dfrac{\theta ^2 x^2}{\pi }}}{\pi } & x>0 \\ 0 & \text{other} \\ \end{cases} \\ \end{array}\,\,\,=\text{HND}(x;\theta)$$

Note that $\theta=\frac{\sqrt{\pi}}{\sigma\sqrt{2}}.$ Thus, $$\text{ND}\left(x;0,\sigma^2\right)=\frac{1}{2}\text{HND}(x;\theta)+\frac{1}{2}\text{HND}(-x;\theta)=\frac{1}{2}\text{GD}\left(x;\frac{1}{2},\frac{\sqrt{\pi }}{\theta },2,0 \right)+\frac{1}{2}\text{GD}\left(-x;\frac{1}{2},\frac{\sqrt{\pi }}{\theta },2,0 \right)\,,$$

which implies that

$$ \begin{align} \text{GND}(x;\mu,\alpha,\beta) &= \frac{1}{2}\text{GD}\left(x;\frac{1}{\beta},\alpha,\beta,\mu \right)+\frac{1}{2}\text{GD}\left(-x;\frac{1}{\beta},\alpha,\beta,\mu \right)\\ &= \frac{\beta e^{-\left(\dfrac{\left|x-\mu\right|}{\alpha }\right)^{\mathrm{\Large{\beta}}}}}{2 \alpha \Gamma \left(\dfrac{1}{\beta }\right)}\\ \end{align} \,,$$

is a generalization of the normal distribution, where $\mu$ is the location, $\alpha>0$ is the scale, and $\beta>0$ is the shape and where $\beta=2$ yields a normal distribution. It includes the Laplace distribution when $\beta=1$. As $\beta\rightarrow\infty$, the density converges pointwise to a uniform density on $(\mu-\alpha,\mu+\alpha)$. Below is the generalized normal distribution plotted for $\alpha =\frac{\sqrt{\pi} }{2}\,,\beta=1/2,1,4$ in blue with the normal case $\alpha =\frac{\sqrt{\pi} }{2},\,\beta=2$ in orange.

The above can be seen as the generalized normal distribution Version 1 and in different parameterizations is known as the exponential power distribution, and the generalized error distribution, which are in turn one of several other generalized normal distributions.

kingfrophe1946.blogspot.com

Source: https://stats.stackexchange.com/questions/37461/the-relationship-between-the-gamma-distribution-and-the-normal-distribution

Share this post