Unit 2: Sampling Distributions and Order Statistics: A Comprehensive Study Guide

Definitions

Random Sample

A random sample is a subset of individuals or items selected from a larger population such that each member of the population has an equal chance of being chosen, and the selection of one member does not influence the selection of another. It ensures that the sample is representative of the population, allowing for valid statistical inferences.

Random Sample: A set of independent and identically distributed random variables, X₁, X₂, ..., X_n, drawn from the same population distribution.

Parameter

A parameter is a numerical characteristic or measure that describes an entire population. Parameters are typically unknown and are often estimated using statistics from a sample.

• Examples: Population mean (μ), population standard deviation (σ), population proportion (p).

Parameter: A fixed, but unknown, numerical value that describes a characteristic of the entire population.

Statistic

A statistic is a numerical characteristic or measure calculated from sample data. It is used to estimate an unknown population parameter.

• Examples: Sample mean (X̄), sample standard deviation (s), sample proportion (p̂).

Statistic: A numerical value calculated from a sample, used to estimate a population parameter.

Sampling Distributions

Sampling Distribution of a Statistic

The sampling distribution of a statistic is the probability distribution of a statistic obtained from all possible samples of a specific sample size drawn from a population. It describes how the statistic varies from sample to sample.

Sampling Distribution: The probability distribution of a statistic, if we were to take all possible samples of a given size from a population.

Sampling Distribution of Sample Mean

The sampling distribution of the sample mean (X̄) is the distribution of all possible sample means that could be obtained from samples of a given size n from a population.

• If the population is normally distributed with mean μ and standard deviation σ, then the sampling distribution of X̄ will also be normally distributed with mean μ and standard deviation σ/√n, regardless of sample size n.
• By the Central Limit Theorem (CLT), even if the population distribution is not normal, the sampling distribution of X̄ will be approximately normal if the sample size n is sufficiently large (typically n ≥ 30), with mean μ and standard deviation σ/√n.

For a population with mean μ and standard deviation σ, the sampling distribution of the sample mean X̄ has:

• Mean: E(X̄) = μ
• Standard Deviation: SD(X̄) = σ/√n (also known as the standard error of the mean)

Standard Errors

The standard error of a statistic is the standard deviation of its sampling distribution. It measures the variability or precision of the statistic as an estimator of a population parameter. A smaller standard error indicates a more precise estimate.

Standard Error: The standard deviation of the sampling distribution of a statistic.

Standard Error of Sample Mean

The standard error of the sample mean (SE_X̄) measures how much the sample mean is expected to vary from the true population mean.

SE_X̄ = σ/√n

If the population standard deviation σ is unknown, it can be estimated using the sample standard deviation s, resulting in the estimated standard error:

Estimated SE_X̄ = s/√n

Standard Error of Sample Variance

The standard error of the sample variance (s²) measures the variability of sample variances across different samples. For a normal population, the sampling distribution of (n-1)s²/σ² follows a chi-squared distribution with (n-1) degrees of freedom.

For a normal population, the variance of the sample variance is:

Var(s²) = 2σ⁴ / (n-1)

Therefore, the standard error of the sample variance is:

SE_s² = √[2σ⁴ / (n-1)] = σ²√[2 / (n-1)]

Standard Error of Sample Proportion

The sample proportion (p̂) is used to estimate the population proportion (p). Its sampling distribution is approximately normal for large sample sizes.

SE_p̂ = √[p(1-p) / n]

If the population proportion p is unknown, it can be estimated using the sample proportion p̂, resulting in the estimated standard error:

Estimated SE_p̂ = √[p̂(1-p̂) / n]

Order Statistics

Introduction

Order statistics are the values in a random sample arranged in non-decreasing order. If X₁, X₂, ..., X_n are n independent and identically distributed random variables, then the order statistics are denoted as X₍₁₎, X₍₂₎, ..., X_(n), where:

• X₍₁₎ = min(X₁, X₂, ..., X_n) (the smallest value)
• X_(n) = max(X₁, X₂, ..., X_n) (the largest value)
• X_(r) is the r^th smallest value in the sample.

Order Statistics: The sample values arranged in ascending order of magnitude.

Distribution of the r^th Order Statistic

Let X_(r) be the r^th order statistic from a sample of size n with probability density function (pdf) f(x) and cumulative distribution function (CDF) F(x). The pdf of X_(r) is given by:

f_X(r)(x) = n! / [(r-1)!(n-r)!] × [F(x)]^r-1 × [1 - F(x)]^n-r × f(x)

Explanation:

• n! / [(r-1)!(n-r)!] is the binomial coefficient (ⁿC_r-1) representing the number of ways to choose r-1 elements to be less than x and n-r elements to be greater than x.
• [F(x)]^r-1 is the probability that r-1 observations are less than x.
• [1 - F(x)]^n-r is the probability that n-r observations are greater than x.
• f(x) is the probability density that one observation is exactly x.

Smallest and Largest Order Statistics

These are special cases of the r^th order statistic.

Smallest Order Statistic (X₍₁₎)

This is the first order statistic, where r = 1.

f_X(1)(x) = n × [1 - F(x)]^n-1 × f(x)

Its CDF is:

F_X(1)(x) = 1 - [1 - F(x)]ⁿ

Largest Order Statistic (X_(n))

This is the n^th order statistic, where r = n.

f_X(n)(x) = n × [F(x)]^n-1 × f(x)

Its CDF is:

F_X(n)(x) = [F(x)]ⁿ

Joint Distribution of r^th and s^th Order Statistics

Let X_(r) and X_(s) be two order statistics, where 1 ≤ r < s ≤ n. The joint probability density function f_{X(r), X(s)}(u, v) for u < v is:

f_{X(r), X(s)}(u, v) = n! / [(r-1)!(s-r-1)!(n-s)!] × [F(u)]^r-1 × [F(v) - F(u)]^s-r-1 × [1 - F(v)]^n-s × f(u) × f(v)

for -∞ < u < v < ∞, and 0 otherwise.

Explanation:

• r-1 observations are less than u.
• s-r-1 observations are between u and v.
• n-s observations are greater than v.
• Two observations are at exactly u and v.

Distribution of Sample Median

The sample median is a measure of central tendency. For a sample of size n, it is an order statistic.

• If n is odd, say n = 2k + 1, the median is X_(k+1).
If n is even, say n = 2k, the median is typically defined as (X_(k) + X_(k+1)) / 2. However, often X_(k) or X_(k+1) is considered