Chi Chi Again

| 0 Comments

Several years ago we saw that, under some relatively easily met assumptions, the averages of independent observations of a random variable tend toward the normal distribution[1]. Derived from that is the chi-squared distribution which describes the behaviour of sums of squares of independent standard normal random variables, having means of zero and standard deviations of one.
In this post we shall see how it is related to the gamma distribution[2] and implement its various functions in terms of those of the latter.

The Relationship Between The Chi-Squared And Gamma Distributions

The characteristic function, or CF, of the square of a standard normal random variable \(Z\) is given by
\[ \hat{f}_{Z^2}\left(t\right) = (1 - 2 \times i \times t)^{-\frac12} \]
where \(i\) is the square root of minus one, as proven in derivation 1.

Derivation 1: The CF Of The Square Of A Standard Normal Random Variable
Given a standard normal random variable
\[ Z \sim N(0, 1) \]
where \(\sim\) means drawn from and \(N\) is the normal distribution, the CF of its square is given by
\[ \begin{align*} \hat{f}_{Z^2}\left(t\right) = \mathrm{E}\left[e^{i \times t \times Z^2}\right] &= \int_{-\infty}^{+\infty} e^{i \times t \times z^2} \times \phi(z) \, \mathrm{d}z = \int_{-\infty}^{+\infty} e^{i \times t \times z^2} \times \tfrac{1}{\sqrt{2 \times \pi}} \times e^{-\frac12 \times z^2} \, \mathrm{d}z\\ &= 2 \times \int_{0}^{\infty} \tfrac{1}{\sqrt{2 \times \pi}} \times e^{-\frac12 \times (1 - 2 \times i \times t) \times z^2} \, \mathrm{d}z \end{align*} \]
where \(\mathrm{E}\) stands for the expectation of the term between the square brackets following it and \(\phi\) is the probability density function, or PDF, of the standard normal distribution. Defining
\[ x = (1 - 2 \times i \times t)^\frac12 \times z \]
we can make the substitution
\[ \hat{f}_{Z^2}\left(t\right) = (1 - 2 \times i \times t)^{-\frac12} \times 2 \times \int_{0}^{(1 - 2 \times i \times t)^{\frac12} \times \infty} \tfrac{1}{\sqrt{2 \times \pi}} \times e^{-\frac12 \times x^2} \, \mathrm{d}x \]
By symmetry we have
\[ \hat{f}_{Z^2}\left(t\right) = (1 - 2 \times i \times t)^{-\frac12} \times 2 \times \int_{0}^{( 1 + i ) \times \infty} \phi(x) \, \mathrm{d}x \]
and exploiting the property of contour integrals
\[ \int_{0}^{( 1 + i ) \times \infty} \phi(x) \, \mathrm{d}x + \int_{( 1 + i ) \times \infty}^\infty \phi(x) \, \mathrm{d}x + \int_\infty^0 \phi(x) \, \mathrm{d}x = 0 \]
yields
\[ \begin{align*} \hat{f}_{Z^2}\left(t\right) &= (1 - 2 \times i \times t)^{-\frac12} \times 2 \times \left(\int_{0}^{\infty} \phi(x) \, \mathrm{d}x + \int_{\infty}^{(1 + i) \times \infty} \phi(x) \, \mathrm{d}x\right)\\ &= (1 - 2 \times i \times t)^{-\frac12} \times 2 \times \left(\frac12 + 0\right) = (1 - 2 \times i \times t)^{-\frac12} \end{align*} \]

A chi-squared random variable with \(k\) degrees of freedom is the sum of the squares of \(k\) independent standard normal random variables so that
\[ X^2_k = \sum_{i=0}^{k-1} Z_i^2 \]
where \(\sum\) is the summation sign. Since the CF of the sum of independent random variables is equal to the product of their CFs, its CF is
\[ \widehat{\chi^2}_k(t) = (1 - 2 \times i \times t)^{-\frac{k}2} \]
Recall that the CF of the gamma distribution is
\[ \hat\gamma_{k,\lambda}(t) = \left(1-\frac{i \times t}{\lambda}\right)^{-k} \]
meaning that
\[ \widehat{\chi^2}_k(t) = \hat\gamma_{\frac{k}2,\frac12}(t) \]
and consequently the chi-squared distribution is simply a special case of the gamma distribution.

Implementing The Chi-Squared Distribution Functions

Whilst we could just use the gamma distribution to determine the properties of chi-squared random variables it's convenient to define functions that will do so for us. For example, listing 1 implements the chi-squared random variable using our ak.gammaRnd function

Listing 1: ak.chiSquaredRnd
ak.chiSquaredRnd = function(k, rnd) {
 var f;

 if(k!==ak.floor(k) || k<1) {
  throw new Error('invalid degrees of freedom in ak.chiSquaredRnd');
 }
 rnd = ak.gammaRnd(k/2, 0.5, rnd);

 f = function(){return rnd();};
 f.k = function(){return k;};
 f.rnd = function(){return rnd.rnd();};
 return Object.freeze(f);
};

as demonstrated by program 1.

Program 1: Using ak.chiSquaredRnd

Similarly, listing 2 implements the chi-squared PDF in terms of the gamma PDF

Listing 2: ak.chiSquaredPDF
ak.chiSquaredPDF = function(k) {
 var f, pdf;

 if(k!==ak.floor(k) || k<1) {
  throw new Error('invalid degrees of freedom in ak.chiSquaredPDF');
 }
 pdf = ak.gammaPDF(k/2, 0.5);

 f = function(x){return pdf(x);};
 f.k = function(){return k;};
 return Object.freeze(f);
};

which program 2 plots for a range of values of \(k\).

Program 2: The Chi Squared PDF

In the same vein the chi-squared CF is defined in terms of the gamma CF in listing 3

Listing 3: ak.chiSquaredCF
ak.chiSquaredCF = function(k) {
 var f, cf;

 if(k!==ak.floor(k) || k<1) {
  throw new Error('invalid degrees of freedom in ak.chiSquaredCF');
 }
 cf = ak.gammaCF(k/2, 0.5);

 f = function(t){return cf(t);};
 f.k = function(){return k;};
 return Object.freeze(f);
};

and is plotted by program 3.

Program 3: The Chi Squared CF

Finally, listing 4 implements the chi-squared cumulative distribution function

Listing 4: ak.chiSquaredCDF
ak.chiSquaredCDF = function(k) {
 var f, cdf;

 if(k!==ak.floor(k) || k<1) {
  throw new Error('invalid degrees of freedom in ak.chiSquaredCDF');
 }
 cdf = ak.gammaCDF(k/2, 0.5);

 f = function(x){return cdf(x);};
 f.k = function(){return k;};
 return Object.freeze(f);
};

and listing 5 implements its inverse

Listing 5: ak.chiSquaredInvCDF
ak.chiSquaredInvCDF = function(k, eps) {
 var f, inv;

 if(k!==ak.floor(k) || k<1) {
  throw new Error('invalid degrees of freedom in ak.chiSquaredInvCDF');
 }
 inv = ak.gammaInvCDF(k/2, 0.5, eps);

 f = function(c){return inv(c);};
 f.k = function(){return k;};
 return Object.freeze(f);
};

both of which are demonstrated by program 4.

Program 4: The Chi Squared Inverse CDF

In conclusion, all of these functions can be found in ChiSquaredDistribution.js.

References

[1] Define Normal, www.thusspakeak.com, 2014.

[2] Kth Time's The Charm, www.thusspakeak.com, 2015.

Leave a comment