AI Engineer Exhaustive
MediumLinear Algebra

Implement Matrix Normalization

Linear Algebra · Data Processing

Medium

Problem

Normalize a matrix by an L1, L2, or maximum norm. Compute the selected norm over columns when axis=0, rows when axis=1, or the complete matrix when axis=None.

\lVert x \rVert_1 = \sum_i |x_i|

\lVert x \rVert_2 = \sqrt{\sum_i x_i^2}

\lVert x \rVert_{\infty} = \max_i |x_i|

Divide every value by the norm of its selected slice. A zero-norm slice remains zero. Return a NumPy array with the same shape as the matrix.

Theory

Matrix normalization transforms the values in a matrix so that rows, columns, or the entire matrix meet certain criteria such as summing to one or having unit length. It ensures that different samples or features are comparable regardless of their original scales or magnitudes.

Why Normalize Matrices?

Probability interpretation: When rows (or columns) sum to 1, entries can be interpreted as probabilities or proportions.

Eliminating scale effects: Samples with large absolute values do not dominate comparisons or distance calculations.

Algorithm requirements: Many algorithms (softmax, attention mechanisms, Markov chains) require normalized inputs.

Numerical stability: Normalization keeps values in reasonable ranges, preventing overflow/underflow.

Types of Matrix Normalization

Row Normalization (L1 - Sum to One)

Each row is divided by its sum so that all row entries add up to 1:

x'_{ij} = \frac{x_{ij}}{\sum_{k} x_{ik}}

Use cases:

Example:

Row Normalization (L2 - Unit Length)

Each row is divided by its Euclidean norm (L2 norm):

x'_{ij} = \frac{x_{ij}}{\sqrt{\sum_{k} x_{ik}^2}}

After normalization, each row has length 1: \sqrt{\sum_k (x'_{ik})^2} = 1

Use cases:

Example:

Column Normalization

Same operations applied column-wise instead of row-wise:

L1 (sum to one):

x'_{ij} = \frac{x_{ij}}{\sum_{k} x_{kj}}

L2 (unit length):

x'_{ij} = \frac{x_{ij}}{\sqrt{\sum_{k} x_{kj}^2}}

Use cases:

Global Normalization

Normalize using statistics computed over the entire matrix:

Divide by global sum:

x'_{ij} = \frac{x_{ij}}{\sum_{m,n} x_{mn}}

Divide by global maximum:

x'_{ij} = \frac{x_{ij}}{\max_{m,n}(x_{mn})}

Mathematical Properties

L1 normalization preserves:

L2 normalization preserves:

Neither preserves:

Worked Example: L1 Row Normalization

Original matrix:

Step 1 - Compute row sums:

Step 2 - Divide each element by its row sum:

Verification: Each row sums to 1.0

Worked Example: L2 Row Normalization

Original matrix:

Step 1 - Compute L2 norms:

Step 2 - Divide each element by its row norm:

Verification: \sqrt{0.6^2 + 0.8^2} = 1.0 and \sqrt{0^2 + 1^2} = 1.0

Handling Edge Cases

Zero rows/columns: If a row sums to zero (or has zero L2 norm), division is undefined.

Options:

Negative values: L1 normalization can produce negative probabilities if inputs are negative. Consider using absolute values or only applying to non-negative data.

Very small denominators: Can cause numerical instability. Add a small epsilon: x'_{ij} = \frac{x_{ij}}{\text{norm} + \epsilon}

Normalization vs Standardization

Normalization: Scales values to a specific range or constraint

Standardization: Centers and scales by statistics

The terms are sometimes used interchangeably, but they refer to different operations.

Axis Selection

In matrix operations, "axis" determines the direction:

For row normalization (each sample normalized independently):

For column normalization (each feature normalized independently):

Where Matrix Normalization Shows Up

Examples

Example 1

Input
matrix = [[3, 4], [1, 0]], axis = 1, norm_type = "l2"
Output
[[0.6, 0.8], [1.0, 0.0]]
Explanation
The row norms are 5 and 1, so each row is divided by its own norm.

Example 2

Input
matrix = [[1, 2], [3, 4]], axis = 0, norm_type = "l1"
Output
[[0.25, 0.333333], [0.75, 0.666667]]

Example 3

Input
matrix = [[2, 8, 4], [1, 3, 9]], axis = 1, norm_type = "max"
Output
[[0.25, 1.0, 0.5], [0.111111, 0.333333, 1.0]]

Hints

  1. Use keepdims=True when reducing so the norm broadcasts back over the matrix.
  2. Replace zero norms with 1.0 through np.where before dividing.

Requirements

Constraints

Starter Code

import numpy as np

def matrix_normalization(matrix: list, axis=None, norm_type: str = "l2") -> np.ndarray:
    """
    Returns a NumPy array with the same shape as matrix.
    """
    # Write code here
    pass

Test Cases

CaseMatches
L2 Row—public
L1 Col—public
Max RowExample 3public