Unit I: Errors and Finite Differences

Exact and Approximate Numbers

In numerical analysis, numbers are broadly classified into two categories based on their origin and precision: exact numbers and approximate numbers.

Definitions and Concepts

• Exact Numbers: Numbers that are obtained by counting, definitions, or exact theoretical derivations. They have no uncertainty. Examples include numbers resulting from counting objects (e.g., exactly 5 apples) or exact conversion factors (e.g., 1 meter = 100 centimeters).
• Approximate Numbers: Numbers that represent the result of a measurement, physical observation, or mathematical approximation. They carry a degree of uncertainty or error. Examples include measurements like π ≈ 3.14159 or the measured length of a table as 1.25 meters.

Exam-Oriented Notes

When solving numerical problems, always identify whether a given value is exact or approximate, as approximate numbers dictate the precision limit of your final computed result.

Significant Digits

Significant digits (or significant figures) are the digits in a number that contribute to its precision. They include all certain digits plus one estimated or uncertain digit.

Rules for Identifying Significant Digits

• All non-zero digits are always significant (e.g., 432 has 3 significant digits).
• Any zeros between significant digits are significant (e.g., 50.05 has 4 significant digits).
• Leading zeros (zeros to the left of the first non-zero digit) are not significant; they merely indicate the position of the decimal point (e.g., 0.0045 has 2 significant digits).
• Trailing zeros to the right of a decimal point are significant (e.g., 3.50 has 3 significant digits).
• Trailing zeros in a whole number without a decimal point are generally ambiguous, but in standard numerical methods coursework, they are treated as non-significant unless scientific notation is used.

Rounding Off and Truncation Errors

Rounding off is the process of omitting unwanted digits while adjusting the retained digits to keep the value as close as possible to the original number. Truncation is the process of simply dropping all digits after a specified decimal place without regard to the magnitude of the discarded digits.

Rules for Rounding Off:

• If the first discarded digit is less than 5, leave the preceding retained digit unchanged (rounding down).
• If the first discarded digit is greater than 5, or is 5 followed by non-zero digits, increase the preceding retained digit by 1 (rounding up).
• If the first discarded digit is exactly 5 followed by zeros (or nothing), standard convention often rounds to the nearest even digit to avoid systematic bias.

Common Mistake: Confusing truncation with rounding. Truncation always introduces a one-sided bias by strictly cutting off values, whereas proper rounding minimizes the absolute error.

Errors: Round Off, Truncation, Absolute, and Relative Errors

Understanding types of errors is crucial for evaluating the reliability of numerical computations.

Types of Errors

• Round-off Error: Arises from the computer or calculator's inability to represent numbers with infinite precision (due to fixed-point or floating-point word lengths).
• Truncation Error: Arises from using an approximation in place of an exact mathematical procedure (e.g., approximating an infinite series like e^x by taking only its first few terms).

Mathematical Definitions of Error

Practical Example

If the exact value of a quantity is x = 10.00 and its approximate value is x̄ = 9.85:

• Absolute Error: E_a = |10.00 - 9.85| = 0.15
• Relative Error: E_r = 0.15 / 10.00 = 0.015
• Percentage Error: E_p = 0.015 × 100% = 1.5%

Error Propagation Under Arithmetic Operations

When approximate numbers with inherent errors are combined using arithmetic operations, those errors propagate through the calculation. Let x̄ and ȳ be approximate values with absolute errors Δx and Δy respectively.

1. Addition and Subtraction

When two numbers are added or subtracted, the maximum absolute error of the result is the sum of the absolute errors of the individual numbers.

For z = x ± y:

Maximum Absolute Error: Δz = Δx + Δy

For subtraction, even if the numbers are very close, their absolute errors add up, which can lead to a catastrophic loss of significance (subtracting nearly equal numbers).

2. Multiplication

When numbers are multiplied, the relative error of the product is approximately the sum of the relative errors of the individual numbers.

For z = x × y:

Relative Error: (Δz / z) ≈ (Δx / x) + (Δy / y)

3. Division

When numbers are divided, the relative error of the quotient is approximately the sum of the relative errors of the numerator and the denominator.

For z = x / y:

Relative Error: (Δz / z) ≈ (Δx / x) + (Δy / y)

Important Observation: In multiplication and division, relative errors are additive. In addition and subtraction, absolute errors are additive.

Finite Difference Operators and Their Properties

Finite differences form the backbone of numerical interpolation, differentiation, and the solution of differential equations. Let y_i = f(x_i) denote the values of a function at equally spaced intervals x_i = x₀ + i h, where h is the step size.

Basic Finite Difference Operators

• Forward Difference Operator (Δ): Defined as:
  Δ f(x) = f(x + h) - f(x)
  In subscript notation: Δ y_i = y_i+1 - y_i
• Backward Difference Operator (∇): Defined as:
  ∇ f(x) = f(x) - f(x - h)
  In subscript notation: ∇ y_i = y_i - y_i-1
• Shift Operator (E): Defined as an operator that shifts the function value by one step size:
  E f(x) = f(x + h)
  In subscript notation: E y_i = y_i+1
• Central Difference Operator (δ): Defined as:
  δ f(x) = f(x + (h/2)) - f(x - (h/2))
  In subscript notation: δ y_i = y_i+(1/2) - y_i-(1/2)
• Averaging Operator (μ): Defined as:
  μ f(x) = (1/2) [f(x + (h/2)) + f(x - (h/2))]

Relationships and Properties Among Operators

Operators obey algebraic laws and can be expressed in terms of one another. The most fundamental relationships include:

• Expressing Shift Operator via Forward Difference:
  E y_i = y_i+1 = y_i + Δ y_i = (1 + Δ) y_i
  Therefore, E = 1 + Δ
• Expressing Forward Difference via Shift Operator:
  Δ = E - 1
• Relationship with Backward Difference:
  ∇ y_i = y_i - y_i-1 = y_i - E^-1 y_i
  Therefore, ∇ = 1 - E^-1
• Connection between Forward and Backward Differences:
  Since E = 1 + Δ and E^-1 = 1 - ∇, higher-order relations can be systematically derived.

Linearity Property

All standard finite difference operators are linear operators. For any constants a and b and functions f(x) and g(x):

Δ [a f(x) + b g(x)] = a Δ f(x) + b Δ g(x)

This linearity property holds true for ∇, E, δ, and μ as well, greatly simplifying algebraic manipulations in numerical methods.