Positive Definite Matrix: Complete Guide with Examples

Quick Navigation

• What is a Positive Definite Matrix?
• How to Check if a Matrix is Positive Definite
• Properties of Positive Definite Matrices
• Positive Definite vs Positive Semidefinite
• Real-World Applications
• Step-by-Step Examples
• Common Mistakes to Avoid
• Frequently Asked Questions
• Related Topics

---

What is a Positive Definite Matrix?

Definition

A symmetric matrix $A$ is called positive definite if the quadratic form $x^T A x$ is strictly positive for every non-zero vector $x$:

$$ \boxed{x^T A x > 0 \quad \text{for all } x \neq 0} $$

Simple Analogy

Think of a positive definite matrix as the matrix equivalent of a positive number:

Numbers	Matrices
$a > 0$	$A \succ 0$ (positive definite)
$a \geq 0$	$A \succeq 0$ (positive semidefinite)
$a$ can be positive or negative	$A$ is indefinite

Geometric Interpretation

For a 2×2 positive definite matrix, the equation $x^T A x = 1$ represents an ellipse centered at the origin:

• The ellipse has finite area (non-zero determinant).
• The axes of the ellipse align with the eigenvectors of $A$.
• The lengths of the semi-axes are $1/\sqrt{\lambda_i}$.

$$ \lambda_1 x_1^2 + \lambda_2 x_2^2 = 1 \quad \text{(Equation of an ellipse)} $$

Simple Examples

✅ Positive Definite: $$ A = \left[\begin{array}{cc} 4 & 1 \\ 1 & 3 \end{array}\right],\quad B = \left[\begin{array}{cc} 2 & 0 \\ 0 & 5 \end{array}\right],\quad C = \left[\begin{array}{cc} 1 & -1 \\ -1 & 4 \end{array}\right] $$

❌ Not Positive Definite: $$ D = \left[\begin{array}{cc} 1 & 0 \\ 0 & -1 \end{array}\right],\quad E = \left[\begin{array}{cc} 1 & 2 \\ 2 & 1 \end{array}\right],\quad F = \left[\begin{array}{cc} 0 & 0 \\ 0 & 0 \end{array}\right] $$

---

How to Check if a Matrix is Positive Definite

Method 1: Eigenvalue Test (Most Reliable)

A symmetric matrix is positive definite if and only if all its eigenvalues are strictly positive:

$$ \lambda_i > 0 \quad \text{for all } i = 1, \dots, n $$

Procedure:

1. Compute the characteristic polynomial: $\det(A - \lambda I) = 0$.
2. Find all eigenvalues $\lambda_1, \dots, \lambda_n$.
3. Verify that every $\lambda_i > 0$.

Example: $$ A = \left[\begin{array}{cc} 4 & 1 \\ 1 & 3 \end{array}\right] $$ Eigenvalues: $\lambda_1 \approx 4.618$, $\lambda_2 \approx 2.382$. Both are $> 0$ ✓ Positive definite!

Method 2: Cholesky Decomposition (Fastest for Computation)

A matrix is positive definite if and only if it has a unique Cholesky decomposition:

$$ A = L L^T $$

where $L$ is a lower triangular matrix with strictly positive diagonal entries. If the algorithm fails (e.g., tries to take the square root of a negative number), the matrix is not positive definite.

Example: $$ A = \left[\begin{array}{cc} 4 & 1 \\ 1 & 3 \end{array}\right] = \left[\begin{array}{cc} 2 & 0 \\ 0.5 & 1.658 \end{array}\right] \left[\begin{array}{cc} 2 & 0.5 \\ 0 & 1.658 \end{array}\right] $$ Cholesky exists! ✓ Positive definite.

Method 3: Sylvester's Criterion (Determinant Test)

A symmetric matrix is positive definite if and only if all leading principal minors are positive:

$$ \det(A_1) > 0,\ \det(A_2) > 0,\ \dots,\ \det(A_n) > 0 $$

where $A_k$ is the top-left $k \times k$ submatrix of $A$.

Example for 3×3 matrix: $$ A = \left[\begin{array}{ccc} 2 & -1 & 0 \\ -1 & 2 & -1 \\ 0 & -1 & 2 \end{array}\right] $$

Check determinants:

• $\det(A_1) = 2 > 0$ ✓
• $\det(A_2) = (2)(2) - (-1)(-1) = 3 > 0$ ✓
• $\det(A_3) = \det(A) = 4 > 0$ ✓

All minors positive → Positive definite!

Method 4: Diagonal Dominance (Sufficient Condition)

If a symmetric matrix is strictly diagonally dominant and has positive diagonal entries, it is guaranteed to be positive definite:

$$ |a_{ii}| > \sum_{j \neq i} |a_{ij}| \quad \text{for all } i $$

Example: $$ A = \left[\begin{array}{ccc} 5 & 1 & 1 \\ 1 & 5 & 1 \\ 1 & 1 & 5 \end{array}\right] $$

• Row 1: $5 > 1 + 1 = 2$ ✓
• Row 2: $5 > 1 + 1 = 2$ ✓
• Row 3: $5 > 1 + 1 = 2$ ✓

Diagonally dominant → Positive definite!

---

Properties of Positive Definite Matrices

1. 🔑 Invertible (Non-Singular)

All positive definite matrices are invertible because no eigenvalue is zero:

$$ \det(A) = \prod_{i=1}^n \lambda_i > 0 $$

2. 📐 Inverse is Also Positive Definite

If $A$ is positive definite, then its inverse $A^{-1}$ is also positive definite:

$$ A \succ 0 \Rightarrow A^{-1} \succ 0 $$

3. 🔄 Unique Square Root Exists

Positive definite matrices have a unique positive definite square root $S$:

$$ A = S^2 \quad \text{where } S = Q \sqrt{D} Q^T $$

4. 📈 All Diagonal Entries Are Positive

Since $e_i^T A e_i = a_{ii}$ (where $e_i$ is the standard basis vector):

$$ a_{ii} > 0 \quad \text{for all } i $$

5. 🔗 Sum of Positive Definite Matrices

The sum of two positive definite matrices is always positive definite:

$$ A \succ 0,\ B \succ 0 \Rightarrow A + B \succ 0 $$

6. ⚡ Cholesky Decomposition

Every positive definite matrix has a unique Cholesky decomposition $A = LL^T$, which is computationally efficient for solving linear systems.

7. 🎯 Convex Quadratic Forms

The quadratic function $f(x) = \frac{1}{2} x^T A x - b^T x$ is strictly convex and has a unique global minimum if $A$ is positive definite.

---

Positive Definite vs Positive Semidefinite

Property	Positive Definite ($A \succ 0$)	Positive Semidefinite ($A \succeq 0$)
Quadratic form	$x^T A x > 0$ (strict)	$x^T A x \geq 0$
Eigenvalues	All $\lambda_i > 0$	All $\lambda_i \geq 0$
Invertible	Yes (non-singular)	No (singular if any $\lambda_i=0$)
Determinant	$> 0$	$\geq 0$
Cholesky	Exists (unique, positive diag)	Exists (diag may have zeros)
Geometry	Ellipse (non-degenerate)	Ellipse, line, or point
Optimization	Unique strict minimum	May have flat directions

Example Comparison

Positive Definite: $$ A = \left[\begin{array}{cc} 2 & 1 \\ 1 & 2 \end{array}\right],\quad \lambda_1 = 3,\ \lambda_2 = 1 $$

Positive Semidefinite: $$ B = \left[\begin{array}{cc} 1 & 1 \\ 1 & 1 \end{array}\right],\quad \lambda_1 = 2,\ \lambda_2 = 0 $$

---

Real-World Applications

📊 Statistics: Covariance Matrices

Covariance matrices are always positive semidefinite. When variables are not linearly dependent, they are positive definite:

$$ \Sigma = \left[\begin{array}{cccc} \sigma_1^2 & \sigma_{12} & \cdots & \sigma_{1n} \\ \sigma_{12} & \sigma_2^2 & \cdots & \sigma_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ \sigma_{1n} & \sigma_{2n} & \cdots & \sigma_n^2 \end{array}\right] \succeq 0 $$

Interpretation: Positive definiteness implies no perfect linear correlations between variables.

📈 Optimization: Hessian at Minimum

In unconstrained optimization, the Hessian matrix $H$ at a local minimum is positive semidefinite. For a strict local minimum, $H$ must be positive definite:

$$ H(x^) \succ 0 \Rightarrow x^ \text{ is a strict local minimum} $$

🔧 Engineering: Stiffness Matrices

In finite element analysis, the stiffness matrix $K$ is positive definite for properly constrained structures, ensuring a unique solution to $Ku=f$:

$$ K \succ 0 \Rightarrow \text{Unique displacement solution} $$

🤖 Machine Learning: Kernel Matrices

Valid kernel functions (like in Support Vector Machines) produce positive definite Gram matrices:

$$ K_{ij} = k(x_i, x_j) \succeq 0 $$

Examples:

• RBF kernel: $k(x,y) = \exp(-\gamma \|x-y\|^2)$
• Polynomial kernel: $k(x,y) = (x \cdot y + c)^d$

🔬 Physics: Mass Matrices

In structural dynamics, mass matrices $M$ are positive definite, ensuring kinetic energy is always positive for non-zero velocity:

$$ T = \frac{1}{2} \dot{u}^T M \dot{u} > 0 $$

---

Step-by-Step Examples

Example 1: 2×2 Positive Definite Matrix

Problem: Check if $A = \left[\begin{array}{cc} 4 & 1 \\ 1 & 3 \end{array}\right]$ is positive definite.

Solution using Eigenvalues:

Step 1: Compute characteristic equation: $$ \det(A - \lambda I) = \det\left[\begin{array}{cc} 4-\lambda & 1 \\ 1 & 3-\lambda \end{array}\right] = (4-\lambda)(3-\lambda) - 1 = 0 $$

Step 2: Expand: $$ 12 - 7\lambda + \lambda^2 - 1 = \lambda^2 - 7\lambda + 11 = 0 $$

Step 3: Solve using quadratic formula: $$ \lambda = \frac{7 \pm \sqrt{49 - 44}}{2} = \frac{7 \pm \sqrt{5}}{2} $$

Step 4: Approximate values: $$ \lambda_1 \approx 4.618,\quad \lambda_2 \approx 2.382 $$

Step 5: Conclusion: Both eigenvalues $> 0$ → Positive definite ✓

Example 2: 2×2 Matrix That is NOT Positive Definite

Problem: Check if $A = \left[\begin{array}{cc} 1 & 2 \\ 2 & 1 \end{array}\right]$ is positive definite.

Solution using Eigenvalues:

Step 1: Characteristic equation: $$ \det\left[\begin{array}{cc} 1-\lambda & 2 \\ 2 & 1-\lambda \end{array}\right] = (1-\lambda)^2 - 4 = 0 $$

Step 2: Solve: $$ (1-\lambda)^2 = 4 \Rightarrow 1-\lambda = \pm 2 \Rightarrow \lambda = -1 \text{ or } \lambda = 3 $$

Step 3: Conclusion: One eigenvalue is negative ($-1$) → Not positive definite (Indefinite)

Example 3: 3×3 Positive Definite (Sylvester's Criterion)

Problem: Check if $A = \left[\begin{array}{ccc} 2 & -1 & 0 \\ -1 & 2 & -1 \\ 0 & -1 & 2 \end{array}\right]$ is positive definite.

Solution:

Step 1: First leading minor (1×1): $$ \det(A_1) = 2 > 0 \quad \text{✓} $$

Step 2: Second leading minor (2×2): $$ \det(A_2) = \det\left[\begin{array}{cc} 2 & -1 \\ -1 & 2 \end{array}\right] = 4 - 1 = 3 > 0 \quad \text{✓} $$

Step 3: Third leading minor (3×3): $$ \det(A) = 2(3) - (-1)(-2) + 0 = 6 - 2 = 4 > 0 \quad \text{✓} $$

Conclusion: All leading principal minors are positive → Positive definite ✓

Example 4: Using Cholesky Decomposition

Problem: Find the Cholesky decomposition of $A = \left[\begin{array}{cc} 4 & 2 \\ 2 & 5 \end{array}\right]$.

Solution:

Step 1: First diagonal entry: $$ l_{11} = \sqrt{a_{11}} = \sqrt{4} = 2 $$

Step 2: First column, second row: $$ l_{21} = \frac{a_{21}}{l_{11}} = \frac{2}{2} = 1 $$

Step 3: Second diagonal entry: $$ l_{22} = \sqrt{a_{22} - l_{21}^2} = \sqrt{5 - 1^2} = \sqrt{4} = 2 $$

Step 4: Construct $L$: $$ L = \left[\begin{array}{cc} 2 & 0 \\ 1 & 2 \end{array}\right] $$

Step 5: Verify $A = L L^T$: $$ L L^T = \left[\begin{array}{cc} 2 & 0 \\ 1 & 2 \end{array}\right] \left[\begin{array}{cc} 2 & 1 \\ 0 & 2 \end{array}\right] = \left[\begin{array}{cc} 4 & 2 \\ 2 & 5 \end{array}\right] = A $$

Conclusion: Cholesky exists → Positive definite ✓

---

Common Mistakes to Avoid

❌ Mistake 1: Checking Only Diagonal Entries

Wrong: "All diagonal entries are positive, so the matrix is positive definite."

Counterexample: $$ A = \left[\begin{array}{cc} 1 & 10 \\ 10 & 1 \end{array}\right] $$ Diagonal entries are positive (1, 1), but eigenvalues are $11$ and $-9$. Since one is negative, it is not positive definite.

Right: You must check eigenvalues or determinants, not just diagonals.

❌ Mistake 2: Forgetting to Check Symmetry

Wrong: Applying positive definiteness tests to non-symmetric matrices.

Right: Positive definiteness is defined for symmetric matrices. If a matrix is not symmetric, check its symmetric part $\frac{A+A^T}{2}$.

❌ Mistake 3: Confusing Positive Definite with Positive Semidefinite

Wrong: "All eigenvalues $\geq 0$ means positive definite."

Right: Positive definite requires strictly $> 0$. If any eigenvalue is 0, it is only positive semidefinite.

❌ Mistake 4: Numerical Rounding Errors

Wrong: Declaring a matrix not positive definite because an eigenvalue is $-10^{-15}$.

Right: Use a small tolerance $\epsilon$ (e.g., $10^{-10}$). If $\lambda_i > -\epsilon$, treat it as non-negative.

---

Frequently Asked Questions

Q: Is the identity matrix positive definite?

A: Yes! $I$ has eigenvalues all equal to 1 ($>0$). Also, $x^T I x = \|x\|^2 > 0$ for all $x \neq 0$.

Q: Is a zero matrix positive definite?

A: No. The zero matrix is positive semidefinite (eigenvalues are 0), but not positive definite.

Q: Can a positive definite matrix have negative entries?

A: Yes! Off-diagonal entries can be negative. Example: $\left[\begin{array}{cc} 2 & -1 \\ -1 & 2 \end{array}\right]$ is positive definite.

Q: Is the sum of positive definite matrices positive definite?

A: Yes! If $A \succ 0$ and $B \succ 0$, then $A + B \succ 0$.

Q: What is the relationship between positive definiteness and convexity?

A: A quadratic function $f(x) = \frac{1}{2}x^T A x - b^T x$ is strictly convex if and only if $A$ is positive definite. This guarantees a unique global minimum.

---

Related Topics

• Symmetric Matrix Checker
• Eigenvalue Calculator
• Cholesky Decomposition Solver
• Matrix Diagonalization