Skip to main content
Home / Orthogonality / Projection Onto A Subspace

Orthogonality

Orthogonal projections, bases, and distances

💡 Need Help?

  • • Enter fractions like 1/2
  • • Use decimals like 0.5
  • • Click "Load Example" to try a preset
  • • Click any step to copy LaTeX
Calculating...

Projection Onto A Subspace Calculator

The projection of a vector onto a subspace finds the closest point in that subspace to the given vector. This is fundamental for least squares approximation, regression analysis, and many optimization problems.

Basis Vectors:
Dimension: D

Vector b (to project onto subspace)

Basis Vectors for Subspace (span of these vectors)

The projection finds the closest point in the subspace spanned by the basis vectors to vector b. Enter fractions like 1/2 or decimals like 0.5.

📑 Quick Navigation

What is Projection onto a Subspace?

Projection onto a subspace is a fundamental operation in linear algebra that finds the "closest" vector in a subspace to a given vector. If you have a subspace W (like a line, plane, or higher-dimensional space) and a vector b outside it, the projection of b onto W is the vector in W that minimizes the distance to b.

Core Concept: The projection of vector b onto subspace W (spanned by columns of matrix A) is the unique vector p in W such that:

  • p is the closest point in W to b
  • The error vector b - p is perpendicular to every vector in W
  • No other vector in W is closer to b than p
$$ \boxed{\text{proj}_W(\mathbf{b}) = \text{closest vector in } W \text{ to } \mathbf{b}} $$

💡 Intuition: Imagine shining a light perpendicular to a plane. The shadow cast by vector b onto that plane is its projection. The line from b to its shadow is perpendicular to the plane!

The Projection Formula

Let A be a matrix whose columns form a basis for subspace W. Then the projection of vector b onto W is given by:

$$ \boxed{\mathbf{p} = A(A^T A)^{-1} A^T \mathbf{b}} $$

Understanding Each Component

A - Matrix with basis vectors as columns

A^T A - Gram matrix (always invertible if columns are independent)

(A^T A)^{-1} A^T - Pseudoinverse of A (denoted A⁺)

P = A(A^T A)^{-1} A^T - Projection matrix (projects ANY vector onto W)

Important: The columns of A must be linearly independent. If they're not, use our Linear Independence Checker first!

Special Case: Projection onto a Line (1D Subspace)

If the subspace is a line spanned by a single vector a, the formula simplifies dramatically:

$$ \mathbf{p} = \frac{\mathbf{a}\mathbf{a}^T}{\mathbf{a}^T\mathbf{a}} \mathbf{b} = \frac{\mathbf{a} \cdot \mathbf{b}}{\mathbf{a} \cdot \mathbf{a}} \mathbf{a} $$

This is the vector projection formula you may have seen before!

The Projection Matrix

The matrix P = A(A^T A)^{-1} A^T has these important properties:

  • P² = P (idempotent - projecting twice is same as once)
  • P^T = P (symmetric)
  • P projects onto the column space of A
  • I - P projects onto the orthogonal complement (the error)
$$ \mathbf{b} = \underbrace{P\mathbf{b}}_{\text{projection}} + \underbrace{(I-P)\mathbf{b}}_{\text{error (perpendicular)}} $$

Step-by-Step Calculation

To project vector b onto the subspace spanned by columns of matrix A:

Step 1: Verify Columns are Independent

Check that the columns of A are linearly independent. If not, you need a basis first.

Step 2: Compute A^T A

Multiply the transpose of A by A. This gives a square matrix of size (number of columns) × (number of columns).

Step 3: Compute A^T b

Multiply the transpose of A by the vector b.

Step 4: Solve (A^T A) x = A^T b

Solve this linear system for x. The solution gives the coefficients of the projection in the basis of A's columns.

$$ \mathbf{p} = A\mathbf{x} $$

Step 5: Compute the Projection

Multiply A by x to get the projection vector p.

💡 Alternative Approach: If the columns of A are orthonormal, the formula simplifies to p = A A^T b (no inverse needed!). Use Gram-Schmidt to orthonormalize first!

Complete Examples

Example 1: Project onto a Line (1D Subspace)

Problem: Project b = (3,4) onto the line spanned by a = (1,0) (the x-axis).

Step 1: This is a 1D subspace, so use the simplified formula:

$$ \mathbf{p} = \frac{\mathbf{a} \cdot \mathbf{b}}{\mathbf{a} \cdot \mathbf{a}} \mathbf{a} $$

Step 2: Compute dot product: a·b = 1·3 + 0·4 = 3

Step 3: Compute a·a = 1² + 0² = 1

Step 4: p = (3/1)·(1,0) = (3,0)

Result: The projection of (3,4) onto the x-axis is (3,0). The error vector is (0,4), which is perpendicular to the x-axis.

Example 2: Project onto a Plane (2D Subspace in ℝ³)

Problem: Project b = (1,2,3) onto the plane spanned by v₁ = (1,0,0) and v₂ = (0,1,0) (the xy-plane).

Step 1: Form matrix A

$$ A = \begin{bmatrix} 1 & 0 \\ 0 & 1 \\ 0 & 0 \end{bmatrix} $$

Step 2: Compute A^T A

$$ A^T A = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \end{bmatrix} \begin{bmatrix} 1 & 0 \\ 0 & 1 \\ 0 & 0 \end{bmatrix} = \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix} $$

Step 3: Compute A^T b

$$ A^T \mathbf{b} = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \end{bmatrix} \begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix} = \begin{bmatrix} 1 \\ 2 \end{bmatrix} $$

Step 4: Solve (A^T A)x = A^T b

$$ \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \end{bmatrix} = \begin{bmatrix} 1 \\ 2 \end{bmatrix} \implies x_1 = 1, x_2 = 2 $$

Step 5: Compute projection p = A x

$$ \mathbf{p} = \begin{bmatrix} 1 & 0 \\ 0 & 1 \\ 0 & 0 \end{bmatrix} \begin{bmatrix} 1 \\ 2 \end{bmatrix} = \begin{bmatrix} 1 \\ 2 \\ 0 \end{bmatrix} $$

Result: The projection of (1,2,3) onto the xy-plane is (1,2,0). The z-coordinate is set to zero because the plane is horizontal.

Example 3: Project onto a 2D Subspace (Not Coordinate Axes)

Problem: Project b = (1,1,1) onto the subspace spanned by v₁ = (1,0,1) and v₂ = (0,1,1).

Step 1: Form matrix A

$$ A = \begin{bmatrix} 1 & 0 \\ 0 & 1 \\ 1 & 1 \end{bmatrix} $$

Step 2: Compute A^T A

$$ A^T A = \begin{bmatrix} 1 & 0 & 1 \\ 0 & 1 & 1 \end{bmatrix} \begin{bmatrix} 1 & 0 \\ 0 & 1 \\ 1 & 1 \end{bmatrix} = \begin{bmatrix} 2 & 1 \\ 1 & 2 \end{bmatrix} $$

Step 3: Compute A^T b

$$ A^T \mathbf{b} = \begin{bmatrix} 1 & 0 & 1 \\ 0 & 1 & 1 \end{bmatrix} \begin{bmatrix} 1 \\ 1 \\ 1 \end{bmatrix} = \begin{bmatrix} 2 \\ 2 \end{bmatrix} $$

Step 4: Solve (A^T A)x = A^T b

$$ \begin{bmatrix} 2 & 1 \\ 1 & 2 \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \end{bmatrix} = \begin{bmatrix} 2 \\ 2 \end{bmatrix} $$

Solving this system (using Cramer's Rule or Gaussian elimination):

$$ x_1 = \frac{2}{3}, \quad x_2 = \frac{2}{3} $$

Step 5: Compute projection p = A x

$$ \mathbf{p} = \frac{2}{3}\begin{bmatrix} 1 \\ 0 \\ 1 \end{bmatrix} + \frac{2}{3}\begin{bmatrix} 0 \\ 1 \\ 1 \end{bmatrix} = \begin{bmatrix} \frac{2}{3} \\ \frac{2}{3} \\ \frac{4}{3} \end{bmatrix} $$

Result: The projection is p = (2/3, 2/3, 4/3). You can verify that b - p = (1/3, 1/3, -1/3) is perpendicular to both v₁ and v₂ (their dot products are 0).

Special Cases

✅ b is already in W

If b is already in the subspace, then p = b. The projection is the vector itself!

$$ \mathbf{b} \in W \implies \text{proj}_W(\mathbf{b}) = \mathbf{b} $$

✅ b is perpendicular to W

If b is orthogonal to every vector in W, then p = 0. The projection is zero.

$$ \mathbf{b} \perp W \implies \text{proj}_W(\mathbf{b}) = \mathbf{0} $$

⚠️ Columns of A are dependent

If columns are linearly dependent, the projection formula using (A^T A)^{-1} fails because the inverse doesn't exist.

Solution: Find an independent basis for W first!

✅ Columns are orthonormal

If columns form an orthonormal basis (orthogonal and unit length), the formula simplifies:

$$ \mathbf{p} = A A^T \mathbf{b} $$

No matrix inverse needed!

Geometric Interpretation

In ℝ² (2D)

  • Line projection: Shadow of a point onto a line
  • Plane projection: In ℝ², the only nontrivial subspaces are lines and the whole plane

Projecting (x,y) onto the x-axis gives (x,0) — the vertical component disappears.

In ℝ³ (3D)

  • Line projection: Point onto a line → point on that line
  • Plane projection: Point onto a plane → point in that plane
  • The error vector is always perpendicular to the plane/line

Orthogonal Decomposition: Any vector b can be uniquely decomposed as:

$$ \mathbf{b} = \mathbf{p} + \mathbf{e} $$

where p ∈ W (the projection) and e ∈ W^⊥ (the error, perpendicular to everything in W).

Real-World Applications

📊 Data Science & Machine Learning

  • PCA (Principal Component Analysis): Projects data onto principal components
  • Linear Regression: Projects response vector onto column space of features
  • Dimensionality Reduction: Projects high-dimensional data onto lower-dimensional subspace

📡 Signal Processing

  • Noise Reduction: Project noisy signal onto clean subspace
  • Filter Design: Project desired response onto filter space
  • Compressed Sensing: Reconstruct signals via projection

🔬 Physics & Engineering

  • Least Squares Fitting: Fit curves to experimental data
  • Finite Element Analysis: Project forces onto basis functions
  • Quantum Mechanics: Project states onto measurement subspaces

🎮 Computer Graphics

  • 3D to 2D Projection: Rendering 3D scenes on 2D screen
  • Shadow Calculation: Project objects onto surfaces
  • Collision Detection: Project shapes onto separating axes

Relationship to Least Squares

Projection is intimately connected to the least squares problem. When solving Ax = b with no exact solution, the least squares solution finds the x that minimizes ||Ax - b||².

$$ \min_{\mathbf{x}} \|A\mathbf{x} - \mathbf{b}\|^2 \implies \mathbf{x} = (A^T A)^{-1} A^T \mathbf{b} $$

The projection of b onto the column space of A is exactly p = A x.

Key Insight: The least squares solution finds the projection! The vector p is the closest point in the column space to b, and the error b - p is perpendicular to all columns.

📐 Try it: Use our Least Squares Calculator to see projection in action!

Frequently Asked Questions

Q: What's the difference between projection onto a line vs. plane?

A: The formula is the same! A can have 1 column (line), 2 columns (plane in ℝ³), or more columns (higher dimensions). The only difference is the dimension of the subspace.

Q: Why do we need columns to be independent?

A: If columns are dependent, A^T A is singular (no inverse). This means the subspace has redundant directions, and you need a basis first.

Q: Is projection the same as least squares?

A: Yes! The least squares solution x gives the coefficients, and Ax is the projection of b onto the column space.

Q: What if the subspace is 0-dimensional (only zero vector)?

A: The projection is always 0. There's only one vector in the subspace: the zero vector.

Q: Is projection always unique?

A: Yes! For any vector b and any subspace W, the orthogonal projection onto W is unique.

Q: What's the projection matrix?

A: P = A(A^T A)^{-1} A^T. It projects ANY vector onto the column space of A. Properties: P² = P (idempotent) and P^T = P (symmetric).

Practice Problems

Beginner

  1. Project b = (2,5) onto the line spanned by a = (1,0).
  2. Project b = (4,3) onto the line spanned by a = (0,1).
  3. What is the projection of any vector onto the subspace consisting of only the zero vector?

Intermediate

  1. Project b = (1,2,3) onto the plane spanned by v₁ = (1,0,0) and v₂ = (0,1,0).
  2. Project b = (2,2,2) onto the subspace spanned by v₁ = (1,0,1) and v₂ = (0,1,1).
  3. Find the projection matrix P for the subspace spanned by a = (1,1) in ℝ².

Advanced

  1. Show that for any projection matrix, P² = P and P^T = P.
  2. Prove that I - P is also a projection matrix (onto the orthogonal complement).
  3. If columns are orthonormal, why does P = A A^T?
Click to reveal solutions

1. p = (2,0)

2. p = (0,3)

3. p = (0,0) (always zero)

4. p = (1,2,0)

5. p = (4/3, 4/3, 8/3)

6. P = (1/2)[[1,1],[1,1]]

7-9. See the derivation in the guide above!

Summary

🎯 Key Takeaways

  • Projection formula: p = A(A^T A)^{-1} A^T b
  • Orthogonal decomposition: b = p + e with p ∈ W, e ⟂ W
  • Projection matrix: P = A(A^T A)^{-1} A^T has P² = P and P^T = P
  • Columns must be independent for the formula to work
  • Least squares solution: x = (A^T A)^{-1} A^T b gives coefficients for projection

💡 Pro Tip: Always check that your basis vectors are linearly independent before projecting. Use our Linear Independence Checker to verify!

Try It Yourself!

Use the calculator above to project vectors onto subspaces:

  1. Enter basis vectors as columns of matrix A
  2. Enter target vector b
  3. Click "Calculate" to see:
    • The projection p
    • The error vector e = b - p
    • Verification that e is orthogonal to all basis vectors
    • The projection matrix P
    • Step-by-step calculation

Test these examples:

  • Line in ℝ²: A = [1,0]ᵀ, b = (3,4) → p = (3,0)
  • Plane in ℝ³: A = [[1,0],[0,1],[0,0]], b = (1,2,3) → p = (1,2,0)
  • Diagonal line in ℝ²: A = [1,1]ᵀ, b = (2,1) → p = (1.5,1.5)
  • b already in subspace: A = [1,0]ᵀ, b = (4,0) → p = (4,0)

📐 Pro Tip: If your projection doesn't make geometric sense, check if your basis vectors are linearly independent and correctly represent the intended subspace!

Related Orthogonality Operations

Gram-Schmidt
Orthogonalize vectors
Orthonormal Basis
Unit orthogonal basis
Decomposition
Parallel/perpendicular components
Projection
Project onto subspace