Random Variables

Definition of a Random Variable

In probability and statistics, an experiment whose outcome cannot be predicted with certainty is called a random experiment. When we perform a random experiment, we are often interested in some numerical outcome associated with it, rather than the raw outcomes themselves.

A Random Variable (RV) is a function that maps the outcomes of a random experiment to real numbers. It is typically denoted by capital letters like X, Y, Z.

For example, if you toss two coins, the possible outcomes are {HH, HT, TH, TT}. If X is the random variable representing the number of heads, then:

• X(HH) = 2
• X(HT) = 1
• X(TH) = 1
• X(TT) = 0

The values that a random variable can take are denoted by lowercase letters (e.g., x, y, z).

Types of Random Variables

Random variables are primarily classified into two types based on the nature of the values they can take:

Discrete Random Variables

A Discrete Random Variable is a random variable that can take on a finite or countably infinite number of distinct values. These values can often be counted.

Key characteristics:

• The possible values can be listed (e.g., 0, 1, 2, 3, ...).
• There are gaps between possible values.

Examples:

• The number of heads when tossing a coin 3 times (values: 0, 1, 2, 3).
• The number of defective items in a sample of 10 (values: 0, 1, 2, ..., 10).
• The number of cars passing a certain point on a road in an hour (values: 0, 1, 2, ...).
• The number of children in a family.

Continuous Random Variables

A Continuous Random Variable is a random variable that can take on any value within a given interval or continuum. Its possible values are uncountable.

Key characteristics:

• The possible values lie in an interval (e.g., [0, 1], (-∞, ∞)).
• There are no gaps between possible values within the interval.

Examples:

• The height of a student (e.g., any value between 1.50m and 1.90m).
• The weight of an apple (e.g., any value between 100g and 200g).
• The time it takes for a bus to arrive (e.g., any value between 0 and 30 minutes).
• The temperature in a city.

Illustrations and Properties of Random Variables

Understanding random variables involves recognizing their domains and ranges. The domain is the sample space of the random experiment, and the range is the set of real numbers assigned to these outcomes.

Illustration: Waiting Time for a Bus

Consider a bus that arrives every 15 minutes. If you arrive at the bus stop at a random time, your waiting time (X) is a random variable. The possible waiting times range from 0 to 15 minutes. Here, X is a continuous random variable because the waiting time can be any value in the interval [0, 15].

Illustration: Number of Successes

Suppose you are performing an experiment where you flip a coin until you get a head, but you only allow a maximum of 3 flips. Let X be the number of tails observed before the first head.

• Outcomes: H, TH, TTH, TTT (if H doesn't appear after 3 flips, consider 3 tails)
• Values of X:
• X(H) = 0
• X(TH) = 1
• X(TTH) = 2
• X(TTT) = 3

In this case, X is a discrete random variable, taking values {0, 1, 2, 3}.

General Properties:

• Random variables transform qualitative or non-numerical outcomes into numerical values, making them amenable to mathematical analysis.
• They simplify the description of complex random phenomena.
• The behavior of a random variable is entirely described by its probability distribution.

Probability Mass Function (p.m.f.)

Concept and Definition

The Probability Mass Function (p.m.f.) is used exclusively for discrete random variables. It specifies the probability that a discrete random variable takes on a particular value.

For a discrete random variable X, the Probability Mass Function (p.m.f.), denoted by p(x) or P(X=x), is a function that assigns a probability to each possible value x of X. That is, p(x) = P(X = x).

Properties of p.m.f.

A function p(x) can be a valid p.m.f. if it satisfies the following two properties:

1. Non-negativity: For every possible value x that the random variable X can take, the probability must be non-negative.

   p(x) ≥ 0 for all x ∈ R

2. Normalization: The sum of all probabilities for all possible values of X must be equal to 1.

   ∑_x p(x) = 1

Examples of p.m.f.

Example 1: Tossing Two Fair Coins

Let X be the number of heads when two fair coins are tossed. The possible values for X are {0, 1, 2}.

• P(X=0) = P(TT) = 1/4
• P(X=1) = P(HT, TH) = 2/4 = 1/2
• P(X=2) = P(HH) = 1/4

The p.m.f. can be tabulated as:

Verification:

• All p(x) ≥ 0.
• ∑ p(x) = 1/4 + 1/2 + 1/4 = 1. (Valid p.m.f.)

Example 2: Rolling a Fair Die

Let X be the outcome of rolling a single fair six-sided die. The possible values for X are {1, 2, 3, 4, 5, 6}.

The p.m.f. is:

p(x) = 1/6 for x ∈ {1, 2, 3, 4, 5, 6}

p(x) = 0 otherwise

Verification:

• All p(x) ≥ 0.
• ∑ p(x) = 1/6 + 1/6 + 1/6 + 1/6 + 1/6 + 1/6 = 6/6 = 1. (Valid p.m.f.)

Probability Density Function (p.d.f.)

Concept and Definition

The Probability Density Function (p.d.f.) is used for continuous random variables. Unlike p.m.f., the p.d.f. does not give the probability that a continuous random variable takes on a specific value, because for a continuous random variable, the probability of taking any single exact value is zero.

For a continuous random variable X, the Probability Density Function (p.d.f.), denoted by f(x), is a function such that the probability that X falls within an interval [a, b] is given by the integral of f(x) over that interval.

That is, P(a ≤ X ≤ b) = ∫_a^b f(x) dx.

The p.d.f. represents the relative likelihood for the random variable to take on a given value. The area under the p.d.f. curve over an interval gives the probability.

Properties of p.d.f.

A function f(x) can be a valid p.d.f. if it satisfies the following two properties:

1. Non-negativity: The p.d.f. must be non-negative for all possible values of X.

   f(x) ≥ 0 for all x ∈ R

2. Normalization: The total area under the entire p.d.f. curve must be equal to 1.

   ∫_-∞^∞ f(x) dx = 1

Important Note: For a continuous random variable, P(X = x) = 0 for any specific value x. This implies that P(a ≤ X ≤ b), P(a < X ≤ b), P(a ≤ X < b), and P(a < X < b) are all equal.

Examples of p.d.f.

Example 1: Uniform Distribution

Let X be a continuous random variable with the p.d.f.:

f(x) = 1/5 for 0 ≤ x ≤ 5

f(x) = 0 otherwise

This is a uniform distribution over the interval [0, 5].

Verification:

• f(x) ≥ 0 for all x.
• ∫_-∞^∞ f(x) dx = ∫₀⁵ (1/5) dx = (1/5) [x]₀⁵ = (1/5) (5 - 0) = 1. (Valid p.d.f.)

To find P(1 ≤ X ≤ 3):

P(1 ≤ X ≤ 3) = ∫₁³ (1/5) dx = (1/5) [x]₁³ = (1/5) (3 - 1) = 2/5

Example 2: Exponential Distribution

Let X be a continuous random variable with the p.d.f.:

f(x) = λe^-λx for x ≥ 0 (where λ > 0)

f(x) = 0 for x < 0

Verification:

• f(x) ≥ 0 for all x, since λ > 0 and e^-λx > 0.
• ∫_-∞^∞ f(x) dx = ∫₀^∞ λe^-λx dx = λ [- (1/λ)e^-λx]₀^∞ = -[e^-λx]₀^∞ = -(0 - 1) = 1. (Valid p.d.f.)

Cumulative Distribution Function (c.d.f.)

Concept and Definition

The Cumulative Distribution Function (c.d.f.), also known as the distribution function, is a fundamental concept applicable to both discrete and continuous random variables. It describes the probability that a random variable X will take a value less than or equal to a specific value x.

For a random variable X, the Cumulative Distribution Function (c.d.f.), denoted by F(x), is defined as F(x) = P(X ≤ x) for all x ∈ R.

Properties of c.d.f.

Regardless of whether the random variable is discrete or continuous, its c.d.f. F(x) must satisfy the following properties:

1. Range of Values: The value of the c.d.f. always lies between 0 and 1, inclusive.

   0 ≤ F(x) ≤ 1 for all x ∈ R

2. Non-decreasing: As x increases, F(x) either stays the same or increases. It never decreases.

   If x₁ < x₂, then F(x₁) ≤ F(x₂)

3. Limits at Extremes: As x approaches negative infinity, F(x) approaches 0. As x approaches positive infinity, F(x) approaches 1.

   lim_x→-∞ F(x) = 0

   lim_x→∞ F(x) = 1

4. Right-Continuity: The c.d.f. is always continuous from the right.

   lim_h→0+ F(x+h) = F(x)

Using c.d.f. to find probabilities:

• P(X > x) = 1 - F(x)
• P(a < X ≤ b) = F(b) - F(a)

c.d.f. for Discrete Random Variables

For a discrete random variable X with p.m.f. p(t), the c.d.f. is calculated by summing the probabilities for all values less than or equal to x:

F(x) = ∑_t≤x p(t)

The c.d.f. for a discrete random variable is a step function, meaning it increases in discrete jumps at each value the random variable can take.

Relationship between p.m.f. and c.d.f. for discrete RV:

p(x) = F(x) - F(x_previous)

Where x_previous is the value immediately preceding x for which the p.m.f. is non-zero.

c.d.f. for Continuous Random Variables

For a continuous random variable X with p.d.f. f(t), the c.d.f. is calculated by integrating the p.d.f. from -∞ up to x:

F(x) = ∫_-∞^x f(t) dt

The c.d.f. for a continuous random variable is a continuous function. The derivative of the c.d.f. gives the p.d.f. where the derivative exists.

Relationship between p.d.f. and c.d.f. for continuous RV:

f(x) = d/dx F(x)

Examples of c.d.f.

Example 1: c.d.f. for Discrete RV (Tossing Two Fair Coins)

From the p.m.f. example (X = number of heads):

The c.d.f. F(x) is:

• For x < 0: F(x) = 0
• For 0 ≤ x < 1: F(x) = P(X ≤ 0) = P(X=0) = 1/4
• For 1 ≤ x < 2: F(x) = P(X ≤ 1) = P(X=0) + P(X=1) = 1/4 + 1/2 = 3/4
• For x ≥ 2: F(x) = P(X ≤ 2) = P(X=0) + P(X=1) + P(X=2) = 1/4 + 1/2 + 1/4 = 1

In piecewise form:

F(x) =

• 0, if x < 0
• 1/4, if 0 ≤ x < 1
• 3/4, if 1 ≤ x < 2
• 1, if x ≥ 2

Example 2: c.d.f. for Continuous RV (Uniform Distribution)

Using the p.d.f. f(x) = 1/5 for 0 ≤ x ≤ 5 and 0 otherwise.

• For x < 0: F(x) = ∫_-∞^x 0 dt = 0
• For 0 ≤ x ≤ 5: F(x) = ∫_-∞⁰ 0 dt + ∫₀^x (1/5) dt = 0 + (1/5) [t]₀^x = x/5
• For x > 5: F(x) = ∫_-∞⁰ 0 dt + ∫₀⁵ (1/5) dt + ∫₅^x 0 dt = 0 + 1 + 0 = 1

In piecewise form:

F(x) =

• 0, if x < 0
• x/5, if 0 ≤ x ≤ 5
• 1, if x > 5

Using this c.d.f. to find P(1 ≤ X ≤ 3):

P(1 ≤ X ≤ 3) = F(3) - F(1) = (3/5) - (1/5) = 2/5