UNIT-III Bivariate Probability Distributions

Bivariate Probability Distributions

In many real-world scenarios, we are interested in the probabilities of two or more random variables occurring simultaneously. A bivariate probability distribution deals with the joint behavior of two random variables, say X and Y. It describes the probability of X taking a specific value (or range of values) AND Y taking a specific value (or range of values) at the same time.

Understanding bivariate distributions allows us to model relationships between two random phenomena, which is crucial in fields like statistics, engineering, and economics.

Discrete Bivariate Probability Distribution (Joint Probability Mass Function - Joint PMF)

For two discrete random variables X and Y, their joint probability mass function (joint PMF), denoted as p(x,y), gives the probability that X takes on a specific value x and Y takes on a specific value y simultaneously.

p(x,y) = P(X=x, Y=y)

Properties of a Joint PMF:

• Non-negativity: p(x,y) ≥ 0 for all possible values of x and y.
• Summation to One: ∑_x ∑_y p(x,y) = 1. (The sum of all joint probabilities must equal 1.)

Example (Discrete Joint PMF):

Consider two discrete random variables, X (number of heads in 2 coin flips) and Y (number of tails in 2 coin flips). The sample space for two flips is {HH, HT, TH, TT}.

• For (X=0, Y=2): Outcome TT, P(X=0, Y=2) = 1/4
• For (X=1, Y=1): Outcomes HT, TH, P(X=1, Y=1) = 2/4 = 1/2
• For (X=2, Y=0): Outcome HH, P(X=2, Y=0) = 1/4

The joint PMF table:

Continuous Bivariate Probability Distribution (Joint Probability Density Function - Joint PDF)

For two continuous random variables X and Y, their joint probability density function (joint PDF), denoted as f(x,y), is a function such that the probability that (X,Y) falls into a certain region R in the xy-plane is given by the double integral of f(x,y) over that region.

P((X,Y) ∈ R) = ∫∫_R f(x,y) dx dy

Properties of a Joint PDF:

• Non-negativity: f(x,y) ≥ 0 for all x and y.
• Total Probability: ∫_-∞^∞ ∫_-∞^∞ f(x,y) dx dy = 1. (The total volume under the surface defined by the PDF must equal 1.)

Marginal Distributions

From a joint probability distribution of two variables (X and Y), we can derive the individual probability distributions of each variable separately. These are called marginal distributions. They represent the probability distribution of a single random variable without considering the other variable's value.

Marginal Probability Mass Function (Marginal PMF)

For discrete random variables, the marginal PMF of X (P_X(x)) is found by summing the joint PMF over all possible values of Y. Similarly, for Y (P_Y(y)), we sum over all possible values of X.

P_X(x) = ∑_y p(x,y)
P_Y(y) = ∑_x p(x,y)

Example (from previous discrete joint PMF table):

To find the marginal PMF of X:

• P_X(0) = p(0,0) + p(0,1) + p(0,2) = 0 + 0 + 1/4 = 1/4
• P_X(1) = p(1,0) + p(1,1) + p(1,2) = 0 + 1/2 + 0 = 1/2
• P_X(2) = p(2,0) + p(2,1) + p(2,2) = 1/4 + 0 + 0 = 1/4

The marginal PMF for X is: P_X(0)=1/4, P_X(1)=1/2, P_X(2)=1/4.

To find the marginal PMF of Y:

• P_Y(0) = p(0,0) + p(1,0) + p(2,0) = 0 + 0 + 1/4 = 1/4
• P_Y(1) = p(0,1) + p(1,1) + p(2,1) = 0 + 1/2 + 0 = 1/2
• P_Y(2) = p(0,2) + p(1,2) + p(2,2) = 1/4 + 0 + 0 = 1/4

The marginal PMF for Y is: P_Y(0)=1/4, P_Y(1)=1/2, P_Y(2)=1/4.

Marginal Probability Density Function (Marginal PDF)

For continuous random variables, the marginal PDF of X (f_X(x)) is found by integrating the joint PDF over all possible values of Y. Similarly, for Y (f_Y(y)), we integrate over all possible values of X.

f_X(x) = ∫_-∞^∞ f(x,y) dy
f_Y(y) = ∫_-∞^∞ f(x,y) dx

Conditional Distributions

A conditional probability distribution describes the probability of one random variable taking on a certain value, given that another random variable has taken on a specific value. It helps us understand how the distribution of one variable changes based on the observation of another.

Conditional Probability Mass Function (Conditional PMF)

For discrete random variables, the conditional PMF of Y given X=x is denoted as p(y|x) or P(Y=y | X=x). It is defined as the joint PMF divided by the marginal PMF of X, provided P_X(x) > 0.

P(Y=y | X=x) = p(x,y) / P_X(x),   given P_X(x) > 0
P(X=x | Y=y) = p(x,y) / P_Y(y),   given P_Y(y) > 0

Example (using previous discrete data, find P(Y=y | X=1)):

We know P_X(1) = 1/2.

• P(Y=0 | X=1) = p(1,0) / P_X(1) = 0 / (1/2) = 0
• P(Y=1 | X=1) = p(1,1) / P_X(1) = (1/2) / (1/2) = 1
• P(Y=2 | X=1) = p(1,2) / P_X(1) = 0 / (1/2) = 0

This result makes intuitive sense: if X=1 (one head in 2 flips), Y must be 1 (one tail). The conditional distribution for Y given X=1 shows Y is certain to be 1.

Conditional Probability Density Function (Conditional PDF)

For continuous random variables, the conditional PDF of Y given X=x is denoted as f(y|x). It is defined as the joint PDF divided by the marginal PDF of X, provided f_X(x) > 0.

f(y | x) = f(x,y) / f_X(x),   given f_X(x) > 0
f(x | y) = f(x,y) / f_Y(y),   given f_Y(y) > 0

Important Note: The conditional PDF f(y|x) is itself a valid probability density function for Y; meaning its integral over all y will be 1.

Independence of Variates

The concept of independence between random variates (variables) is fundamental in probability and statistics. Two random variables are independent if the outcome of one does not affect the probability distribution of the other. (This is the general idea requested by the syllabus.)

Mathematically, independence means that their joint behavior can be described simply by the product of their individual (marginal) behaviors.

Definition for Discrete Random Variables

Two discrete random variables X and Y are said to be independent if and only if their joint PMF is equal to the product of their marginal PMFs for all possible pairs (x,y):

P(X=x, Y=y) = P_X(x) * P_Y(y)   for all x, y

An equivalent way to state this for discrete variables is that the conditional probability of Y given X is simply the marginal probability of Y (and vice versa):

• P(Y=y | X=x) = P_Y(y)
• P(X=x | Y=y) = P_X(x)

Definition for Continuous Random Variables

Two continuous random variables X and Y are said to be independent if and only if their joint PDF is equal to the product of their marginal PDFs for all x and y:

f(x,y) = f_X(x) * f_Y(y)   for all x, y

Similar to discrete variables, for continuous independent variables:

• f(y | x) = f_Y(y)
• f(x | y) = f_X(x)

Important Observations:

• Checking for Independence: To check if X and Y are independent, calculate their joint distribution and their marginal distributions. Then, verify if the product of the marginals equals the joint distribution for all possible values/ranges. If even one point fails this condition, the variables are not independent.
• Simplification: If X and Y are independent, many calculations (like expectations of products) become simpler.

Example:

Let's use our earlier example where X is the number of heads and Y is the number of tails in 2 coin flips. We found:

• p(0,2) = 1/4 (Joint PMF)
• P_X(0) = 1/4 (Marginal PMF for X=0)
• P_Y(2) = 1/4 (Marginal PMF for Y=2)

Now, let's check the independence condition for (x=0, y=2):

• Left side: P(X=0, Y=2) = 1/4
• Right side: P_X(0) * P_Y(2) = (1/4) * (1/4) = 1/16

Since 1/4 ≠ 1/16, the condition P(X=x, Y=y) = P_X(x) * P_Y(y) does not hold for all (x,y). Therefore, X and Y are not independent in this example.

This makes intuitive sense: if you know the number of heads (X), you directly know the number of tails (Y) because their sum is fixed (always 2 in two flips). Thus, they are clearly dependent variables.