1. Basic Concepts

Random Experiment

An experiment or process for which the outcome cannot be predicted with certainty, but all possible outcomes are known.

• Example: Tossing a coin, rolling a die.

Sample Point

A single possible outcome of a random experiment.

• Example: When rolling a die, "4" is a sample point.

Sample Space (S)

The set of all possible outcomes (sample points) of a random experiment.

• Example (Rolling a die): S = {1, 2, 3, 4, 5, 6}
• Example (Tossing two coins): S = {HH, HT, TH, TT}

Event (E)

A subset of the sample space. It is a collection of one or more sample points.

• Example (Rolling a die): Event A (Getting an even number): A = {2, 4, 6}

2. Algebra of Events

Events can be combined using set operations.

• A or B (A ∪ B): The event that *at least one* of A or B occurs.
• A and B (A ∩ B): The event that *both* A and B occur simultaneously.
• Not A (A' or A^c): The complement of A. The event that A does *not* occur.
• Mutually Exclusive Events: Two events that cannot occur at the same time. (A ∩ B = ∅, the empty set).
• Exhaustive Events: A set of events that covers the entire sample space. (A₁ ∪ A₂ ∪ ... = S).

3. Approaches to Probability

How we define and calculate probability.

1. Classical (or 'a priori') Definition

This definition assumes all outcomes are equally likely.

If a random experiment has 'n' mutually exclusive, exhaustive, and equally likely outcomes, and 'm' of these outcomes are favorable to an event A:

• Example: P(Rolling a 3 on a die) = 1 / 6.
• Limitation: Fails if outcomes are not equally likely (e.g., a biased coin).

2. Statistical (or Empirical / 'a posteriori') Definition

This definition is based on relative frequency from actual experiments.

If an experiment is repeated 'n' times and event A occurs 'f' times, the probability of A is the limit of this relative frequency as 'n' becomes infinitely large.

• Example: If we toss a coin 10,000 times and get 5,030 heads, we estimate P(Heads) ≈ 0.503.

3. Axiomatic Definition

This is the modern, mathematical definition. It states the *rules* (axioms) that probability must follow, without saying how to calculate it.

Axiom 1 (Non-negativity): For any event A, P(A) ≥ 0.

Axiom 2 (Certainty): The probability of the entire sample space is 1. P(S) = 1.

Axiom 3 (Additivity): If A and B are mutually exclusive events, then P(A ∪ B) = P(A) + P(B).

4. Theorems of Probability (Addition & Multiplication)

1. Addition Law of Probability

Used to find the probability of (A or B).

General Rule (for *any* two events):

(We subtract P(A ∩ B) because it was counted twice).

Special Rule (for *Mutually Exclusive* events):

Since P(A ∩ B) = 0, the formula simplifies to:

2. Multiplication Law of Probability

Used to find the probability of (A and B). It is derived from the definition of conditional probability.

5. Conditional Probability

Conditional Probability, P(A | B): The probability of event A occurring, given that event B has already occurred.

It "updates" the probability of A based on new information. We are restricting our sample space from S down to just B.

Formula:

Example:

Roll a die. S = {1, 2, 3, 4, 5, 6}.
Event A = "Get a 4". P(A) = 1/6.
Event B = "Get an even number" = {2, 4, 6}. P(B) = 3/6.
Event (A ∩ B) = "Get a 4 AND an even number" = {4}. P(A ∩ B) = 1/6.

P(A | B) = P(A ∩ B) / P(B) = (1/6) / (3/6) = 1/3.

6. Independent Events

Independent Events: Two events A and B are independent if the occurrence of one event does not affect the probability of the other event occurring.

Formal Definition:

A and B are independent if and only if:

This also means:

• P(A | B) = P(A)
• P(B | A) = P(B)

7. Bayes' Theorem and Its Applications

Bayes' Theorem allows us to "reverse" the conditional probability. If we know P(B | A), Bayes' Theorem helps us find P(A | B).

It is used to update our belief about a "cause" (A) given a new "effect" (B).

• P(A): Prior Probability (our initial belief about A)
• P(B | A): Likelihood (the chance of seeing effect B if cause A is true)
• P(A | B): Posterior Probability (our updated belief about A after seeing B)

The Theorem

For two events A and B:

More generally, if the sample space is partitioned by events A₁, A₂, ..., A_k:

Bayes' Theorem (Full Form):
P(A_i | B) = [ P(B | A_i)P(A_i) ] / [ Σ P(B | A_j)P(A_j) ]

The denominator is just P(B), calculated using the Law of Total Probability.

Application (Example):

A factory has two machines, A1 and A2.
- A1 produces 60% of items (P(A1) = 0.6).
- A2 produces 40% of items (P(A2) = 0.4).
- 2% of items from A1 are defective (P(D | A1) = 0.02).
- 1% of items from A2 are defective (P(D | A2) = 0.01).

Question: An item is selected at random and found to be defective (D). What is the probability it came from machine A1?

We want to find P(A1 | D).

Step 1: Numerator: P(D | A1) * P(A1)
(0.02) * (0.6) = 0.012

Step 2: Denominator (Total Probability of D):
P(D) = P(D | A1)P(A1) + P(D | A2)P(A2)
P(D) = (0.02)(0.6) + (0.01)(0.4)
P(D) = 0.012 + 0.004 = 0.016

Step 3: Divide.
P(A1 | D) = 0.012 / 0.016 = 12 / 16 = 0.75

Result: There is a 75% chance the defective item came from machine A1.