Unit 1: Linear Vector Spaces, Matrices, and Tensors: A Comprehensive Study Guide

Linear Vector Space

Vectors in n-dimension

A vector in n-dimension (or n-dimensional vector) is an ordered set of n real or complex numbers. It can be represented as a row vector [v₁, v₂, ..., v_n] or a column vector:

v = ⎛⎜⎝v₁v₂⋮v_n⎞⎟⎠

where v_i are the components of the vector. The set of all such vectors forms an n-dimensional vector space, often denoted as ℝⁿ for real components or ℂⁿ for complex components.

Linear independence

A set of vectors {v₁, v₂, ..., v_k} is said to be linearly independent if the only solution to the equation c₁v₁ + c₂v₂ + ... + c_kv_k = 0 is c₁ = c₂ = ... = c_k = 0.

If there exists a non-trivial solution (where at least one c_i ≠ 0), then the vectors are linearly dependent, meaning at least one vector can be expressed as a linear combination of the others.

Basis and Dimension

A basis for a vector space V is a set of linearly independent vectors that span V. This means every vector in V can be uniquely expressed as a linear combination of the basis vectors.

The dimension of a vector space is the number of vectors in any basis for that space. For ℝⁿ or ℂⁿ, the standard basis consists of n vectors, e.g., for ℝ³: {(1,0,0), (0,1,0), (0,0,1)}, so its dimension is 3.

Scalar product

The scalar product (also known as dot product or inner product) is an operation that takes two vectors and returns a scalar quantity. For two n-dimensional real vectors u = [u₁, ..., u_n] and v = [v₁, ..., v_n], the scalar product is:

u ⋅ v = u₁v₁ + u₂v₂ + ... + u_nv_n = ∑_i=1ⁿ u_iv_i

For complex vectors, if u = [u₁, ..., u_n] and v = [v₁, ..., v_n], the scalar product is often defined as:

u ⋅ v = u₁v̄₁ + u₂v̄₂ + ... + u_nv̄_n = ∑_i=1ⁿ u_iv̄_i

where v̄_i denotes the complex conjugate of v_i.

Norm and Orthogonality

The norm (or length/magnitude) of a vector v is derived from the scalar product. For a real vector v, it is defined as:

||v|| = √ (v ⋅ v) = √ (v₁² + v₂² + ... + v_n²)

For a complex vector v, it is defined as:

||v|| = √ (v ⋅ v) = √ (∑_i=1ⁿ v_iv̄_i) = √ (∑_i=1ⁿ |v_i|²)

Two vectors u and v are said to be orthogonal if their scalar product is zero:

u ⋅ v = 0

If two non-zero vectors are orthogonal, they are also linearly independent.

Schwarz inequality

The Schwarz inequality (also known as Cauchy-Schwarz inequality) states that for any two vectors u and v in an inner product space, the absolute value of their inner product is less than or equal to the product of their norms:

|u ⋅ v| ≤ ||u|| ||v||

Equality holds if and only if u and v are linearly dependent (one is a scalar multiple of the other).

Gram-Schmidt orthogonalization technique

The Gram-Schmidt orthogonalization process is an algorithm for constructing an orthonormal basis from any basis of an inner product space. Given a basis {v₁, v₂, ..., v_k}, it produces an orthonormal basis {u₁, u₂, ..., u_k} using the following steps:

1. Choose u₁ = v₁ / ||v₁||.
2. For k > 1, compute the orthogonal projection of v_k onto the subspace spanned by {u₁, ..., u_k-1}:
   w_k = v_k - ∑_j=1^k-1 (v_k ⋅ u_j)u_j
3. Normalize w_k to get u_k:
   u_k = w_k / ||w_k||

This process generates an orthogonal set of vectors, which are then normalized to form an orthonormal set.

Infinite dimensional space

An infinite dimensional space is a vector space that does not have a finite basis. This means no finite set of vectors can span the entire space.

Examples include the space of all continuous functions on an interval [a, b], or the space of all polynomials.

Hilbert space

A Hilbert space is a complete inner product space. It is a vector space equipped with an inner product that allows distances and angles to be defined, and it is 'complete' in the sense that all Cauchy sequences converge to a point within the space.

Hilbert spaces are fundamental in functional analysis and quantum mechanics, providing a suitable mathematical framework for wave functions.

Matrices and Linear Operators

Linear operators and their Matrix representation

A linear operator (or linear transformation) T is a function from one vector space V to another vector space W (T: V → W) that satisfies two properties for all vectors u, v in V and all scalars c:

1. Additivity: T(u + v) = T(u) + T(v)
2. Homogeneity: T(cu) = cT(u)

Every linear operator between finite-dimensional vector spaces can be represented by a matrix. If V has basis {v₁, ..., v_n} and W has basis {w₁, ..., w_m}, then for each v_j, T(v_j) can be written as a linear combination of the w_i's:

T(v_j) = ∑_i=1^m a_ijw_i

The matrix A = [a_ij] is the matrix representation of T with respect to these bases. The j-th column of A consists of the coefficients of T(v_j) in terms of the basis of W.

Eigenvalues and Eigenvectors of a matrix

For a square matrix A, an eigenvector v is a non-zero vector that, when multiplied by A, only changes by a scalar factor. The scalar factor λ is called the eigenvalue associated with that eigenvector.

This relationship is expressed by the equation:

Av = λv

To find eigenvalues, we solve the characteristic equation:

det(A - λI) = 0

where I is the identity matrix. For each eigenvalue λ, the corresponding eigenvectors v are found by solving (A - λI)v = 0.

Cayley-Hamilton theorem

The Cayley-Hamilton theorem states that every square matrix satisfies its own characteristic equation. If p(λ) = det(A - λI) = a_nλⁿ + a_n-1λ^n-1 + ... + a₁λ + a₀ is the characteristic polynomial of a matrix A, then:

p(A) = a_nAⁿ + a_n-1A^n-1 + ... + a₁A + a₀I = 0

This theorem is useful for finding powers of a matrix, its inverse, or expressing a matrix polynomial in terms of lower powers of A.

Orthogonal, Unitary and Hermitian matrices

These are special types of square matrices with important properties:

Tensors

Definition of Tensor

A tensor is a mathematical object that generalizes scalars (rank 0 tensors), vectors (rank 1 tensors), and linear operators (rank 2 tensors) to higher dimensions. It transforms in a specific way under coordinate transformations, allowing physical laws to be expressed in a form that is independent of the coordinate system chosen.

Tensors are typically represented by components in a coordinate system, and their transformation properties define their type.

Covariant and Contravariant tensor

Tensors are classified by how their components transform under a change of coordinates:

• Contravariant Tensor: A tensor whose components transform inversely to the transformation of the basis vectors. This is typically indicated by superscript indices (e.g., Aⁱ, T^ij). Vectors are often considered contravariant tensors (rank 1). If xⁱ are coordinates, and new coordinates are x'^j, then the transformation rule for a contravariant vector Aⁱ is:
  A'^j = (∂x'^j/∂xⁱ) Aⁱ (sum over i)
• Covariant Tensor: A tensor whose components transform in the same way as the transformation of the basis vectors (or gradient of a scalar field). This is indicated by subscript indices (e.g., A_i, T_ij). Gradient vectors are examples of covariant tensors. The transformation rule for a covariant vector A_i is:
  A'_j = (∂xⁱ/∂x'^j) A_i (sum over i)

Tensors can also be of mixed type, having both contravariant and covariant indices (e.g., Tⁱ_j).

Fundamental operation with tensors

Common operations with tensors include:

• Addition/Subtraction: Tensors of the same rank and type can be added or subtracted component-wise.
• Scalar Multiplication: Multiplying a tensor by a scalar scales all its components.
• Contraction: Reduces the rank of a tensor by 2 by setting one contravariant and one covariant index equal and summing over it (e.g., Tⁱ_i).
• Outer Product (Tensor Product): Creates a higher-rank tensor from two tensors by multiplying their components (e.g., if Aⁱ and B_j, then Cⁱ_j = AⁱB_j).
• Inner Product: A combination of an outer product and a contraction, resulting in a lower-rank tensor or a scalar.
• Raising and Lowering Indices: Using the metric tensor to change contravariant indices to covariant ones and vice versa (e.g., A_i = g_ijA^j, Aⁱ = g^ijA_j).

Metric tensor

The metric tensor, denoted g_ij (covariant) or g^ij (contravariant), defines the inner product in a vector space and measures distances and angles. It captures the geometry of the space.

In a given coordinate system, the infinitesimal squared distance ds² between two nearby points is given by:

ds² = g_ij dxⁱ dx^j (sum over i, j)

The components g_ij form a symmetric matrix. Its inverse, g^ij, is used to raise indices.

Covariant differentiation and Christoffel symbols

In curved spaces or non-Cartesian coordinate systems, the standard partial derivative of a tensor component does not transform as a tensor. This is because the basis vectors themselves change from point to point.

Covariant differentiation is a generalization of differentiation that yields a tensor. It accounts for the change in the coordinate system's basis vectors as one moves through space.

For a contravariant vector A^μ, the covariant derivative is:

∇_νA^μ = ∂_νA^μ + Γ^μ_ανA^α

For a covariant vector A_μ, the covariant derivative is:

∇_νA_μ = ∂_νA_μ - Γ^α_μνA_α

The Christoffel symbols (Γ^μ_αν) are coefficients that describe how the basis vectors change in a given coordinate system. They are not tensors themselves but arise from the derivatives of the metric tensor.

For a Riemannian manifold, Christoffel symbols of the second kind are given by:

Γ^k_ij = (1/2)g^kl(∂_ig_jl + ∂_jg_il - ∂_lg_ij)

where g^kl is the inverse of the metric tensor g_kl, and ∂ denotes partial differentiation.