Unit 4: Foundations of Probability
4.1 Basic Terminology
Random Experiment
An experiment or process whose outcome cannot be predicted with certainty, but all possible outcomes are known.
- Example 1: Tossing a coin.
- Example 2: Rolling a die.
- Example 3: Measuring the lifetime of a light bulb.
Sample Point
A single possible outcome of a random experiment.
- Example (Rolling a die): The number '4' is a sample point.
Sample Space (S)
The set of all possible outcomes (all sample points) of a random experiment.
- Example 1 (Tossing one coin): S = {Head, Tail}
- Example 2 (Rolling one die): S = {1, 2, 3, 4, 5, 6}
- Example 3 (Tossing two coins): S = {HH, HT, TH, TT}
4.2 Events and Algebra of Events
Event
An event is any subset of the sample space S. It is a collection of one or more sample points.
- Example (Rolling a die, S = {1, 2, 3, 4, 5, 6}):
- Let A be the event "getting an even number." Then A = {2, 4, 6}.
- Let B be the event "getting a number greater than 4." Then B = {5, 6}.
Types of Events
- Simple Event: An event with only one sample point (e.g., A = {1}).
- Compound Event: An event with more than one sample point (e.g., A = {2, 4, 6}).
- Sure Event (or Certain Event): The entire sample space S. (e.g., "getting a number from 1 to 6"). P(S) = 1.
- Impossible Event (or Null Event): The empty set (Ø). (e.g., "getting a 7"). P(Ø) = 0.
Algebra of Events (Set Operations)
Since events are sets, we can use set theory to combine them.
[Image of Venn diagrams illustrating A union B, A intersection B, and A complement]
| Operation |
Notation |
Meaning ("In Words") |
Example (A={1,2}, B={2,3}) |
| Union |
A ∪ B |
"Event A OR B or Both" |
{1, 2, 3} |
| Intersection |
A ∩ B |
"Event A AND B" |
{2} |
| Complement |
A' or Aᶜ or A-bar |
"Event NOT A" |
If S={1,2,3,4}, A' = {3, 4} |
Mutually Exclusive (or Disjoint) Events
Two events A and B are mutually exclusive if they cannot occur at the same time. This means their intersection is the impossible event (Ø).
A ∩ B = Ø
- Example: When rolling a die, the events A = {1, 3} (odd) and B = {2, 4} (even) are mutually exclusive. You cannot get a result that is in both.
Exhaustive Events
A set of events {A₁, A₂, ...} is exhaustive if their union is the entire sample space. At least one of the events must occur.
A₁ ∪ A₂ ∪ ... = S
- Example: The events A = {1, 2, 3} and B = {3, 4, 5, 6} are exhaustive.
- Example 2: A = {Odd}, B = {Even} are both mutually exclusive and exhaustive.
4.3 Definitions of Probability
4.4 Classical (or 'a priori') Definition
This definition applies when all outcomes in the sample space are equally likely (e.g., fair coin, fair die).
If a sample space S has 'N' finite, mutually exclusive, and equally likely outcomes, and an event 'A' has 'm' of those outcomes favorable to it, then the probability of A is:
P(A) = m / N
P(A) = (Number of outcomes favorable to A) / (Total number of possible outcomes)
- Example (Rolling a die): Find P(Even Number).
- S = {1, 2, 3, 4, 5, 6} => N = 6
- A = {2, 4, 6} => m = 3
- P(A) = 3 / 6 = 0.5
Merits and Demerits
- Merits: Very simple and intuitive. Easy to calculate for games of chance.
- Demerits (Limitations):
- "Equally likely" is circular: How do you know they are "equally likely" without already knowing they have equal probabilities?
- Fails if outcomes are not equally likely: Cannot be used for a biased coin.
- Fails for infinite sample spaces: Cannot be used if N is infinite (e.g., "pick a number between 0 and 1").
4.5 Relative Frequency (or Statistical / 'a posteriori') Definition
This definition is based on observation and experimentation.
If a random experiment is repeated 'n' times under identical conditions, and the event 'A' occurs 'm' times, then the probability of A is the limit of the relative frequency (m/n) as the number of trials 'n' becomes infinitely large.
P(A) = limn→∞ (m / n)
- Example: You toss a coin 10,000 times and get 5,034 heads. The relative frequency is 5034 / 10000 = 0.5034. We use this as an estimate of the true P(Head).
Merits and Demerits
- Merits:
- Based on real-world observation.
- Can be used when outcomes are not equally likely (e.g., can estimate P(Head) for a biased coin).
- Demerits (Limitations):
- Vague: We can never actually perform infinite trials. How large is "large enough"?
- Conditions may not be identical: Repeating an experiment perfectly is often impossible.
- Not a definition, but an estimate: It only gives an approximation of the "true" probability.
4.6 Axiomatic Definition (Kolmogorov's Axioms)
This is the modern, mathematical definition of probability. It doesn't say how to calculate probability; it just states the three rules (axioms) that a probability measure 'P' must follow.
Given a sample space S and a set of events, a probability P(A) is a function that assigns a real number to every event A, such that:
The Three Axioms of Probability
- Axiom 1: Non-negativity
P(A) ≥ 0
The probability of any event is a non-negative number.
- Axiom 2: Certainty (Normalization)
P(S) = 1
The probability of the entire sample space (a certain event) is 1.
- Axiom 3: Additivity
If A and B are mutually exclusive events (A ∩ B = Ø), then:
P(A ∪ B) = P(A) + P(B)
(This extends to any countable number of mutually exclusive events).
Merits and Demerits
- Merits:
- Mathematically rigorous: All other probability rules (e.g., P(A') = 1 - P(A)) can be proven from these three axioms.
- General: It includes both the classical and frequency definitions as special cases. It works for finite or infinite sample spaces.
- Demerits (Limitations):
- Abstract: It doesn't tell you how to assign the initial probabilities (P(A)). It just gives the rules they must follow.