Unit 3: Hypothesis Testing: Core Concepts and Large Sample Tests with CLT

Introduction to Hypothesis Testing

Null and Alternative Hypotheses

Hypothesis testing is a statistical method used to make decisions about a population parameter based on sample data. It involves setting up two opposing hypotheses:

• Null Hypothesis (H₀): This is a statement of no effect, no difference, or no relationship. It represents the status quo or a commonly accepted belief. We assume the null hypothesis is true until there is sufficient evidence to reject it.
• Alternative Hypothesis (H₁ or H_a): This is a statement that contradicts the null hypothesis. It represents what we are trying to find evidence for. The alternative hypothesis can be one-sided (e.g., greater than or less than) or two-sided (e.g., not equal to).

Null Hypothesis (H₀): A statement indicating no statistical significance, assumed true until evidence suggests otherwise.

Alternative Hypothesis (H₁): A statement that contradicts the null hypothesis, representing the claim to be tested.

Examples:

• Testing if a new drug is better:
  H₀: The new drug has no effect (mean recovery time is the same as old drug).
  H₁: The new drug reduces recovery time (mean recovery time is less than old drug).
• Testing if a coin is fair:
  H₀: The coin is fair (proportion of heads = 0.5).
  H₁: The coin is not fair (proportion of heads ≠ 0.5).

Level of Significance (α)

The level of significance, denoted by α (alpha), is the probability of rejecting the null hypothesis when it is actually true. It represents the maximum acceptable risk of making a Type I error. Common values for α are 0.05 (5%) and 0.01 (1%).

Level of Significance (α): The probability of rejecting a true null hypothesis, often set at 0.05 or 0.01.

A smaller α value means we require stronger evidence to reject H₀, reducing the chance of a Type I error but increasing the chance of a Type II error.

Type I and Type II Errors, their Probabilities and Critical Region

In hypothesis testing, decisions are made under uncertainty, which means there's always a risk of making an incorrect decision. There are two types of errors:

• Type I Error (False Positive): This occurs when we reject the null hypothesis (H₀) when it is actually true. The probability of making a Type I error is denoted by α (the level of significance).
• Type II Error (False Negative): This occurs when we fail to reject the null hypothesis (H₀) when it is actually false. The probability of making a Type II error is denoted by β (beta).

Type I Error: Rejecting the null hypothesis when it is true (Probability = α).

Type II Error: Failing to reject the null hypothesis when it is false (Probability = β).

Relationship between α and β: Usually, if you decrease α, β tends to increase, and vice versa, for a fixed sample size. The power of a test is 1 - β, which is the probability of correctly rejecting a false null hypothesis.

Critical Region

The critical region (also known as the rejection region) is the set of values for the test statistic that leads to the rejection of the null hypothesis. These values are considered extreme enough to be unlikely if the null hypothesis were true. The boundaries of the critical region are determined by the level of significance (α) and the sampling distribution of the test statistic.

• For a two-tailed test, the critical region is split into two tails of the distribution.
• For a one-tailed test (left or right), the critical region is entirely in one tail.

Concept of p-value

The p-value (probability value) is a measure of the strength of evidence against the null hypothesis. It is the probability of observing a test statistic as extreme as, or more extreme than, the one calculated from the sample, assuming that the null hypothesis is true.

p-value: The probability of obtaining test results as extreme as the observed results, assuming the null hypothesis is true. A small p-value indicates strong evidence against the null hypothesis.

Interpreting the p-value:

• If p-value ≤ α: We reject the null hypothesis. The observed data are statistically significant, providing strong evidence against H₀.
• If p-value > α: We fail to reject the null hypothesis. There is not enough evidence to conclude that H₀ is false.

A smaller p-value means stronger evidence against H₀.

Large Sample Tests using the Central Limit Theorem (CLT)

For large sample sizes (typically n ≥ 30), the Central Limit Theorem (CLT) allows us to assume that the sampling distribution of sample means or proportions will be approximately normal, regardless of the shape of the population distribution. This enables the use of Z-tests for hypothesis testing in various scenarios.

The general steps for large sample hypothesis testing are:

1. State the null (H₀) and alternative (H₁) hypotheses.
2. Choose a level of significance (α).
3. Calculate the test statistic (Z-score) using sample data.
4. Determine the critical value(s) or calculate the p-value.
5. Make a decision: Reject H₀ if the test statistic falls in the critical region or if p-value ≤ α; otherwise, fail to reject H₀.
6. State the conclusion in the context of the problem.

Testing Single Proportion

This test is used to determine if a sample proportion (p̂) is significantly different from a hypothesized population proportion (p₀).

• Hypotheses:
  H₀: p = p₀
  H₁: p ≠ p₀ (two-tailed), p > p₀ (right-tailed), or p < p₀ (left-tailed)
• Test Statistic (Z):
  

  Z = (p̂ - p₀) / √[p₀(1 - p₀) / n]

  
  Where:
  • p̂ is the sample proportion (x/n)
  • p₀ is the hypothesized population proportion
  • n is the sample size
• Decision Rule: Compare the calculated Z-value with the critical Z-value(s) from the standard normal distribution based on α and the type of test (one-tailed or two-tailed). Alternatively, compare the p-value with α.

Testing Difference of Two Proportions

This test compares two sample proportions (p̂₁ and p̂₂) to determine if there is a significant difference between their corresponding population proportions (p₁ and p₂).

• Hypotheses:
  H₀: p₁ = p₂ (or p₁ - p₂ = 0)
  H₁: p₁ ≠ p₂ (two-tailed), p₁ > p₂ (right-tailed), or p₁ < p₂ (left-tailed)
• Pooled Proportion (p̂_c): Used under H₀ when p₁ = p₂.
  

  p̂_c = (x₁ + x₂) / (n₁ + n₂)

• Test Statistic (Z):
  

  Z = (p̂₁ - p̂₂) / √[p̂_c(1 - p̂_c)(1/n₁ + 1/n₂)]

  
  Where:
  • p̂₁ = x₁/n₁ and p̂₂ = x₂/n₂ are the sample proportions
  • n₁ and n₂ are the sample sizes
• Decision Rule: Same as for a single proportion test.

Testing Single Mean

This test determines if a sample mean (x̄) is significantly different from a hypothesized population mean (μ₀).

• Hypotheses:
  H₀: μ = μ₀
  H₁: μ ≠ μ₀ (two-tailed), μ > μ₀ (right-tailed), or μ < μ₀ (left-tailed)
• Test Statistic (Z):
  If population standard deviation (σ) is known:
  

  Z = (x̄ - μ₀) / (σ / √n)

  
  If population standard deviation (σ) is unknown but sample size (n) is large (typically n ≥ 30), we use the sample standard deviation (s) as an estimate for σ:
  

  Z = (x̄ - μ₀) / (s / √n)

  
  Where:
  • x̄ is the sample mean
  • μ₀ is the hypothesized population mean
  • σ is the population standard deviation (or s, sample standard deviation, for large n)
  • n is the sample size
• Decision Rule: Same as for proportion tests.

Testing Difference of Two Means

This test compares two sample means (x̄₁ and x̄₂) to determine if there is a significant difference between their corresponding population means (μ₁ and μ₂).

• Hypotheses:
  H₀: μ₁ = μ₂ (or μ₁ - μ₂ = 0)
  H₁: μ₁ ≠ μ₂ (two-tailed), μ₁ > μ₂ (right-tailed), or μ₁ < μ₂ (left-tailed)
• Test Statistic (Z):
  If population standard deviations (σ₁ and σ₂) are known:
  

  Z = (x̄₁ - x̄₂) / √(σ₁²/n₁ + σ₂²/n₂)

  
  If population standard deviations (σ₁ and σ₂) are unknown but sample sizes (n₁ and n₂) are large (n₁, n₂ ≥ 30), we use sample standard deviations (s₁ and s₂) as estimates:
  

  Z = (x̄₁ - x̄₂) / √(s₁²/n₁ + s₂²/n₂)

  
  Where:
  • x̄₁ and x̄₂ are the sample means
  • n₁ and n₂ are the sample sizes
  • σ₁, σ₂ (or s₁, s₂) are the standard deviations
• Decision Rule: Same as for other Z-tests.

Testing Standard Deviation and Difference of Standard Deviations

While standard tests for population variance or standard deviation often use the Chi-square (χ²) distribution for a single variance and the F-distribution for comparing two variances, the Central Limit Theorem can be applied for large samples to approximate the sampling distribution of the sample standard deviation itself as normal, allowing for Z-tests.

Testing Single Standard Deviation

This test is used to determine if a sample standard deviation (s) is significantly different from a hypothesized population standard deviation (σ₀) for large samples.

• Hypotheses:
  H₀: σ = σ₀
  H₁: σ ≠ σ₀ (two-tailed), σ > σ₀ (right-tailed), or σ < σ₀ (left-tailed)
• Test Statistic (Z) for large samples:
  For large samples, the sampling distribution of s is approximately normal with mean σ and standard deviation σ / √[2(n-1)].
  

  Z = (s - σ₀) / (σ₀ / √[2(n-1)])

  
  Where:
  • s is the sample standard deviation
  • σ₀ is the hypothesized population standard deviation
  • n is the sample size
• Decision Rule: Compare the calculated Z-value with the critical Z-value(s).

Testing Difference of Two Standard Deviations

This test is used to compare two sample standard deviations (s₁ and s₂) to determine if there is a significant difference between their corresponding population standard deviations (σ₁ and σ₂) for large samples.

• Hypotheses:
  H₀: σ₁ = σ₂ (or σ₁ - σ₂ = 0)
  H₁: σ₁ ≠ σ₂ (two-tailed), σ₁ > σ₂ (right-tailed), or σ₁ < σ₂ (left-tailed)
• Test Statistic (Z) for large samples:
  For large samples, the difference (s₁ - s₂) is approximately normally distributed. If H₀ is true (σ₁ = σ₂ = σ), we can use a pooled estimate for σ or assume they are approximately equal.
  

  Z = (s₁ - s₂) / √[σ₁² / (2n₁) + σ₂² / (2n₂)]

  
  Under H₀: σ₁ = σ₂ = σ, the formula simplifies to:
  

  Z = (s₁ - s₂) / (σ √[1/(2n₁) + 1/(2n₂)])

  
  If σ is unknown, for large samples, sample standard deviations (s₁ and s₂) can be used as estimates:
  

  Z = (s₁ - s₂) / √[s₁² / (2n₁) + s₂² / (2n₂)]

  
  Where:
  • s₁ and s₂ are the sample standard deviations
  • n₁ and n₂ are the sample sizes
  • σ is the common population standard deviation under H₀ (if applicable)
• Decision Rule: Compare the calculated Z-value with the critical Z-value(s).