Unit 5: Exact Sampling Distributions: Student’s t-distribution and Snedecore’s F-distribution

Student’s t-distribution

Introduction to t-distribution

The Student’s t-distribution is a crucial probability distribution used in inferential statistics, particularly when dealing with small sample sizes or when the population standard deviation is unknown. It is named after William Sealy Gosset, who published under the pseudonym “Student”.

Definition: The Student’s t-distribution is a probability distribution that arises when estimating the mean of a normally distributed population in situations where the sample size is small and the population standard deviation is unknown. It is symmetric and bell-shaped, similar to the normal distribution, but with heavier tails, indicating more probability in the tails and thus greater uncertainty compared to the normal distribution. The shape of the t-distribution is determined by its degrees of freedom (ν).

Derivation of its Probability Density Function (p.d.f.)

The t-distribution is derived from the ratio of a standard normal variate and the square root of a chi-squared variate, divided by its degrees of freedom. Specifically, if Z is a standard normal random variable (Z ~ N(0, 1)) and χ² is a chi-squared random variable with ν degrees of freedom (χ² ~ χ²(ν)), and Z and χ² are independent, then the random variable T defined as:

T = Z / √(χ² / ν)

follows a t-distribution with ν degrees of freedom. The p.d.f. of a t-distribution with ν degrees of freedom is given by:

f(t; ν) = (Γ((ν+1)/2) / (√νπ Γ(ν/2))) × (1 + t²/ν)^-(ν+1)/2
where Γ is the Gamma function and ν > 0.

Mean, Variance, and Moments

Mean

• For ν > 1, the mean of the t-distribution is:

  E(T) = 0

• For ν = 1, the mean is undefined.

Variance

• For ν > 2, the variance of the t-distribution is:

  Var(T) = ν / (ν - 2)

• For 1 < ν ≤ 2, the variance is undefined.

Moments

• Due to its symmetry, all odd moments about the mean (if they exist) are zero. For instance, E(T^2k+1) = 0 for k = 0, 1, 2, ... if ν > 2k+1.
• Even moments exist if the degrees of freedom are greater than the order of the moment. For example, E(T^2k) exists if ν > 2k.

Limiting Form of t-distribution

An important property of the t-distribution is its limiting behavior. As the degrees of freedom (ν) approach infinity, the t-distribution approaches the standard normal distribution (Z ~ N(0, 1)).

This means that for large sample sizes, the t-distribution and the normal distribution are practically indistinguishable. The concept highlights that as sample size increases, the sample standard deviation becomes a more reliable estimate of the population standard deviation, reducing the uncertainty that the t-distribution accounts for.

Test of Significance based on t-distribution

The t-distribution is widely used for hypothesis testing, especially when dealing with population means and when the population standard deviation is unknown, typically with small to moderate sample sizes.

One-Sample t-test

This test determines if the mean of a single sample is significantly different from a known or hypothesized population mean (μ₀).

Test statistic: t = (x̄ - μ₀) / (s / √n)
where x̄ is the sample mean, μ₀ is the hypothesized population mean, s is the sample standard deviation, and n is the sample size. The degrees of freedom (ν) for this test are n - 1.

Two-Sample t-test (Independent Samples)

This test determines if there is a significant difference between the means of two independent populations.

• Pooled t-test (assuming equal population variances): This version assumes that the variances of the two populations are equal and pools the sample variances to estimate the common population variance.

  Test statistic: t = (x̄₁ - x̄₂) / (s_p√(1/n₁ + 1/n₂))
  where x̄₁ and x̄₂ are the sample means, n₁ and n₂ are the sample sizes, and s_p is the pooled standard deviation. The pooled variance is calculated as: s_p² = (((n₁-1)s₁²) + ((n₂-1)s₂²)) / (n₁+n₂-2). The degrees of freedom (ν) are n₁ + n₂ - 2.

• Welch's t-test (assuming unequal population variances): This version is more robust when the population variances are not assumed to be equal.

  Test statistic: t = (x̄₁ - x̄₂) / √(s₁²/n₁ + s₂²/n₂)
  where s₁² and s₂² are the sample variances. The degrees of freedom are calculated using a complex Satterthwaite's approximation, which often results in a non-integer value.

Paired Sample t-test

This test is used when samples are dependent or paired (e.g., measurements taken before and after a treatment on the same subjects). It analyzes the mean of the differences between paired observations.

1. Calculate the difference (d_i) for each pair.
2. Calculate the mean of these differences (d̄) and their standard deviation (s_d).

Test statistic: t = d̄ / (s_d / √n)
where n is the number of pairs. The degrees of freedom (ν) are n - 1.

Confidence Intervals based on t-distribution

Confidence intervals provide a range of values within which the true population parameter (e.g., mean or difference in means) is estimated to lie with a certain level of confidence.

For a single population mean (μ):

CI = x̄ ± t_{α/2, ν} × (s / √n)

For the difference between two population means (μ₁ - μ₂) for independent samples with equal variances:

CI = (x̄₁ - x̄₂) ± t_{α/2, ν} × s_p√(1/n₁ + 1/n₂)

For the mean difference of paired samples (μ_d):

CI = d̄ ± t_{α/2, ν} × (s_d / √n)

In these formulas, t_{α/2, ν} is the critical t-value obtained from the t-distribution table for a specified confidence level (1-α) and degrees of freedom (ν).

Snedecore's F-distribution

Introduction to F-distribution

The F-distribution, also known as Snedecore's F-distribution or the Fisher-Snedecore distribution, is another fundamental continuous probability distribution in inferential statistics. It is primarily used when comparing variances of two populations or when comparing means of three or more populations (Analysis of Variance - ANOVA).

Definition: The F-distribution is a probability distribution that arises in the testing of hypotheses concerning the equality of two population variances, and is fundamental to analysis of variance (ANOVA). It is a skewed distribution, defined only for non-negative values (F ≥ 0), and its shape is determined by two parameters: the numerator degrees of freedom (ν₁) and the denominator degrees of freedom (ν₂).

Derivation of p.d.f.

The F-distribution is derived from the ratio of two independent chi-squared variates, each divided by its respective degrees of freedom. Specifically, if χ₁² is a chi-squared random variable with ν₁ degrees of freedom (χ₁² ~ χ²(ν₁)) and χ₂² is an independent chi-squared random variable with ν₂ degrees of freedom (χ₂² ~ χ²(ν₂)), then the random variable F defined as:

F = (χ₁² / ν₁) / (χ₂² / ν₂)

follows an F-distribution with ν₁ and ν₂ degrees of freedom, denoted as F ~ F(ν₁, ν₂).

The p.d.f. of an F-distribution with ν₁ and ν₂ degrees of freedom is given by:

f(F; ν₁, ν₂) = (√( (ν₁ν₂)^ν₁ (ν₂)^ν₂ ) / ( (ν₁F + ν₂)^{(ν₁+ν₂)} )) × (Γ((ν₁+ν₂)/2) / (Γ(ν₁/2) Γ(ν₂/2))) × F^(ν₁/2)-1
for F > 0, where Γ is the Gamma function. A more commonly encountered form using the Beta function is:
f(F; ν₁, ν₂) = ( (ν₁/ν₂)^ν₁/2 / B(ν₁/2, ν₂/2) ) × F^(ν₁/2)-1 × (1 + (ν₁/ν₂)F)^{-(ν₁+ν₂)/2}
where B is the Beta function.

Mean, Variance, and Mode

Mean

• For ν₂ > 2, the mean of the F-distribution is:

  E(F) = ν₂ / (ν₂ - 2)

• For ν₂ ≤ 2, the mean is undefined.

Variance

• For ν₂ > 4, the variance of the F-distribution is:

  Var(F) = (2ν₂²(ν₁ + ν₂ - 2)) / (ν₁(ν₂ - 2)²(ν₂ - 4))

• For ν₂ ≤ 4, the variance is undefined.

Mode

• For ν₁ > 2, the mode of the F-distribution is:

  Mode(F) = ((ν₁ - 2)ν₂) / (ν₁(ν₂ + 2))

• If ν₁ ≤ 2, the mode is typically considered to be at F=0, though the p.d.f. might be decreasing from F=0.

Distribution of 1/F(n1, n2)

An important property of the F-distribution relates to the reciprocal of an F-variate:

If F follows an F-distribution with ν₁ (numerator) and ν₂ (denominator) degrees of freedom, i.e., F ~ F(ν₁, ν₂), then its reciprocal, 1/F, follows an F-distribution with the degrees of freedom reversed, i.e., 1/F ~ F(ν₂, ν₁).

This property is particularly useful in hypothesis testing, especially for two-tailed F-tests, as it allows for the use of a single F-table by transforming the critical value for the lower tail.

Test of Significance based on F-distribution

The F-distribution is used for various tests of significance, primarily involving comparisons of variances or means of multiple groups.

Comparing two population variances (F-test for variances)

This test assesses whether the variances of two independent populations (σ₁² and σ₂²) are equal.

Test statistic: F = s₁² / s₂²
where s₁² and s₂² are the sample variances from two independent samples. The numerator has ν₁ = n₁ - 1 degrees of freedom, and the denominator has ν₂ = n₂ - 1 degrees of freedom. It is common practice to place the larger sample variance in the numerator to ensure F ≥ 1, converting a two-tailed test into a one-tailed test.

Analysis of Variance (ANOVA)

ANOVA is a statistical technique that uses the F-distribution to test for significant differences between the means of three or more groups. It does this by comparing the variability between group means to the variability within the groups.

Test statistic: F = MS_Between / MS_Within
where MS_Between (Mean Square Between groups) represents the variance explained by the differences among group means, and MS_Within (Mean Square Within groups or Mean Square Error) represents the unexplained variance or error variance. The degrees of freedom are (k-1) for the numerator (MS_Between) and (N-k) for the denominator (MS_Within), where k is the number of groups and N is the total sample size across all groups.

Confidence Intervals based on F-distribution

Confidence intervals for the ratio of two population variances (σ₁² / σ₂²) can be constructed using the F-distribution.

CI for σ₁² / σ₂²:
(s₁² / s₂²) × (1 / F_{α/2, ν1, ν2}) < σ₁² / σ₂² < (s₁² / s₂²) × F_{α/2, ν2, ν1}
Alternatively, using notation for lower and upper critical values directly:
(s₁² / s₂²) / F_U < σ₁² / σ₂² < (s₁² / s₂²) / F_L
where F_L = F_{1-α/2, ν1, ν2} (the lower critical value) and F_U = F_{α/2, ν1, ν2} (the upper critical value). Note that F_{1-α/2, ν1, ν2} = 1 / F_{α/2, ν2, ν1}.

Relationship between t, F and χ² distributions

The t-distribution, F-distribution, and Chi-squared (χ²) distribution are closely related, as they are all derived from the standard normal distribution and often arise in the context of statistical inference for normally distributed data.

Relationship between t-distribution and Chi-squared (χ²) distribution

As previously mentioned in its derivation, a t-distributed random variable can be expressed in terms of a standard normal variate and a chi-squared variate:

T = Z / √(χ² / ν)
where Z ~ N(0, 1) is a standard normal random variable, χ² ~ χ²(ν) is a chi-squared random variable with ν degrees of freedom, and Z and χ² are independent.

Relationship between F-distribution and Chi-squared (χ²) distribution

Similarly, an F-distributed random variable is defined as a ratio of two independent chi-squared variates:

F = (χ₁² / ν₁) / (χ₂² / ν₂)
where χ₁² ~ χ²(ν₁) and χ₂² ~ χ²(ν₂) are two independent chi-squared random variables with ν₁ and ν₂ degrees of freedom, respectively.

Relationship between t-distribution and F-distribution

There is a direct and fundamental relationship between the t-distribution and the F-distribution:

If T follows a t-distribution with ν degrees of freedom (T ~ t(ν)), then its square, T², follows an F-distribution with 1 and ν degrees of freedom (T² ~ F(1, ν)).

This relationship implies that a two-tailed t-test with α significance level is equivalent to a one-tailed F-test with the same α significance level, where the numerator degrees of freedom for the F-test is 1. Both tests will yield the same p-value and conclusion regarding the null hypothesis concerning the equality of two population means (or a single population mean against a hypothesized value).