When Matrices Cannot Be Inverted

A matrix is non-invertible (singular) when it has no inverse matrix A⁻¹

Determinant = 0

The most fundamental condition: det(A) = 0

Linear Dependence

Rows or columns are linearly dependent

Rank Deficient

rank(A) < n for an n×n matrix

Zero Eigenvalue

At least one eigenvalue equals zero

Common Examples:

Zero Row/Column:

[1 2]
[0 0]

Proportional Rows:

[2 4]
[1 2]

Zero Matrix:

[0 0]
[0 0]

💡 Key Insight

A matrix is non-invertible when it "loses information" - meaning multiple inputs can produce the same output, making the transformation non-reversible.

🔗 Connection to Linear Regression

Classical Method: β̂ = (X^TX)^-1X^Ty

When X^TX is non-invertible, this formula fails!

❌ When Classical Fails:

Multicollinearity (high correlation) → Near-zero eigenvalues
More features than samples (p > n) → Rank deficient matrix
Linear dependence of predictors → det(X^TX) = 0
Zero variance predictors (constant values) → Singular matrix

✅ ML Solutions:

Ridge Regression
Lasso Regression
Principal Component Regression
Elastic Net