Unit III: Statistical Inference Fundamentals
Basic Concepts: Random Sample, Parameter, and Statistic
Statistical inference involves drawing conclusions about a large population based on a smaller subset of data. To understand this process, we must define the foundational building blocks: random samples, parameters, and statistics.
Random Sample
A random sample is a subset of individuals or observations chosen from a larger population in such a way that every individual has an equal and independent chance of being chosen. This independence and randomness ensure that the sample is representative of the population without systematic bias.
Parameter
A parameter is a numerical value or characteristic that describes a specific feature of an entire population. Parameters are usually fixed but unknown constants. Common examples include population mean (μ) and population standard deviation (σ).
Statistic
A statistic is a numerical value or characteristic calculated from the data of a random sample. Unlike parameters, statistics vary from sample to sample and are known values once the sample data is collected. Common examples include sample mean (x̄) and sample variance (s2).
| Feature | Parameter | Statistic |
|---|---|---|
| Source | Entire Population | Random Sample |
| Nature | Fixed and usually unknown | Variable and known from data |
| Notation | Greek letters (μ, σ, p) | Latin letters (x̄, s, p̂) |
| Primary Role | Describes population characteristics | Estimates population parameters |
Statistical Hypotheses: Null, Alternative, Simple, and Composite
Hypothesis testing is a formal procedure for investigating statistical claims. It relies on setting up competing statements about population parameters.
Null and Alternative Hypotheses
Null Hypothesis (H0): The statement of no difference, no effect, or status quo. It assumes that any observed difference is purely due to random chance.
Alternative Hypothesis (H1 or Ha): The statement that contradicts the null hypothesis. It represents the claim or effect that the researcher aims to support or prove.
Simple and Composite Hypotheses
Hypotheses are further classified based on how precisely they specify the population parameter:
- Simple Hypothesis: A hypothesis that completely specifies the exact numerical value of the population parameter. For example, H0: μ = 50.
- Composite Hypothesis: A hypothesis that does not specify a single exact value, but rather a range or inequality for the parameter. For example, H1: μ > 50 or H1: μ ≠ 50.
Level of Significance, Error Types, and Critical Region
When testing a hypothesis using sample data, decisions are subject to uncertainty and potential error. We classify these errors and define the decision boundaries.
Level of Significance
The level of significance, denoted by the Greek letter α (alpha), is the maximum allowable probability of incorrectly rejecting a true null hypothesis. Commonly chosen values for α are 0.05 (5%) or 0.01 (1%).
Probabilities of Type I and Type II Errors
Because decisions are made based on samples rather than complete population data, two types of errors can occur:
| Decision / Reality | H0 is True | H0 is False |
|---|---|---|
| Accept / Retain H0 | Correct Decision (Probability = 1 - α) | Type II Error (Probability = β) |
| Reject H0 | Type I Error (Probability = α) | Correct Decision (Probability = 1 - β, Power) |
- Type I Error: Rejecting the null hypothesis (H0) when it is actually true. The probability of committing a Type I error is equal to the level of significance (α).
- Type II Error: Accepting (or failing to reject) the null hypothesis (H0) when it is actually false. The probability of committing a Type II error is denoted by β (beta).
Critical Region
Critical Region (Definition only): The critical region (or rejection region) is the set of all possible values of a test statistic that would lead to the rejection of the null hypothesis H0.
Concept of p-Value
In modern statistical testing, the p-value (probability value) provides a standardized way to measure the strength of evidence against the null hypothesis.
- A very small p-value (typically ≤ α) indicates strong evidence against H0, leading to its rejection.
- A large p-value indicates weak evidence against H0, meaning we fail to reject it.
Definition: The p-value is the probability of obtaining test results at least as extreme as the results actually observed, under the assumption that the null hypothesis is correct.
Sampling Distribution of a Statistic
Because samples are selected at random, every calculated statistic (like the sample mean) is itself a random variable with its own probability distribution.
Sampling Distribution: The probability distribution of a given test statistic computed from all possible random samples of the exact same size drawn from the same population.
Key Exam-Oriented Notes:
- The shape and properties of a sampling distribution depend on the underlying population distribution, the sample size (n), and the specific statistic being calculated.
- Understanding the sampling distribution is essential because it allows statisticians to calculate probabilities associated with a statistic, forming the theoretical backbone for confidence intervals and hypothesis tests.