AI Engineer Exhaustive
EasyData Processing

Implement z-Score Standardization

Data Processing

Easy

Problem

Standardize numeric data to zero mean and unit population variance along a selected axis:

z = \frac{x - \mu}{\sigma}

Here, x is an input value, \mu is the mean of its selected slice, \sigma is the population standard deviation of that slice, and z is the standardized value. Use column-wise slices when axis=0 and row-wise slices when axis=1. If a slice has standard deviation at most eps, return zeros for that slice. Return a NumPy array.

Theory

Z-score standardization transforms data to have zero mean and unit standard deviation. Each value is converted to the number of standard deviations it lies from the mean. This is one of the most fundamental preprocessing techniques in statistics and machine learning, making features comparable regardless of their original scales.

Why Standardize?

Equal feature contribution: Features with larger scales would otherwise dominate distance calculations and gradient updates.

Algorithm requirements: Many algorithms assume or perform better with standardized data (PCA, SVM, regularized regression).

Interpretable scores: Z-scores directly indicate how unusual a value is - a z-score of 2 means the value is 2 standard deviations above the mean.

Numerical stability: Keeps values in reasonable ranges, preventing overflow in calculations.

The Z-Score Formula

For a value x from a distribution with mean \mu and standard deviation \sigma:

z = \frac{x - \mu}{\sigma}

For a sample:

z = \frac{x - \bar{x}}{s}

Where:

Properties of Z-Scores

Mean of zero: After transformation, the mean of z-scores is exactly 0

E[Z] = E\left[\frac{X - \mu}{\sigma}\right] = \frac{E[X] - \mu}{\sigma} = 0

Unit variance: Standard deviation of z-scores is exactly 1

\text{Var}(Z) = \text{Var}\left(\frac{X - \mu}{\sigma}\right) = \frac{\text{Var}(X)}{\sigma^2} = 1

Shape preservation: Standardization is a linear transformation; it does not change the shape of the distribution.

Worked Example

Data: [10, 20, 30, 40, 50]

Step 1 - Calculate mean:

\bar{x} = \frac{10 + 20 + 30 + 40 + 50}{5} = 30

Step 2 - Calculate standard deviation:

s = \sqrt{\frac{(10-30)^2 + (20-30)^2 + (30-30)^2 + (40-30)^2 + (50-30)^2}{5-1}}

= \sqrt{\frac{400 + 100 + 0 + 100 + 400}{4}} = \sqrt{250} \approx 15.81

Step 3 - Calculate z-scores:

z_1 = \frac{10 - 30}{15.81} = -1.26

z_2 = \frac{20 - 30}{15.81} = -0.63

z_3 = \frac{30 - 30}{15.81} = 0.00

z_4 = \frac{40 - 30}{15.81} = 0.63

z_5 = \frac{50 - 30}{15.81} = 1.26

Result: [-1.26, -0.63, 0.00, 0.63, 1.26]

Verification: Mean ≈ 0, Standard deviation ≈ 1

Sample vs Population Standard Deviation

Population standard deviation (divide by N):

\sigma = \sqrt{\frac{1}{N} \sum_{i=1}^{N} (x_i - \mu)^2}

Sample standard deviation (divide by N-1, Bessel's correction):

s = \sqrt{\frac{1}{N-1} \sum_{i=1}^{N} (x_i - \bar{x})^2}

Which to use?

Interpretation of Z-Scores

Z = 0: Value equals the mean

Z > 0: Value is above the mean

Z < 0: Value is below the mean

For normal distributions:

Handling Zero Standard Deviation

When all values are identical:

\sigma = 0 \Rightarrow z = \frac{x - \mu}{0} = \text{undefined}

Solutions:

Column-wise Standardization

For a dataset with multiple features, standardize each column independently:

For each column:

  1. Compute column mean
  2. Compute column standard deviation
  3. Apply z-score formula to each element

Result: Each feature has mean 0 and std 1, but different features may still have different ranges.

Train-Test Split Considerations

Critical rule: Compute mean and std from training data only

Apply to training data:

z_{train} = \frac{x_{train} - \mu_{train}}{\sigma_{train}}

Apply to test data using training statistics:

z_{test} = \frac{x_{test} - \mu_{train}}{\sigma_{train}}

Why? Using test statistics causes data leakage - the model would have information about the test distribution.

Inverse Transform

To convert z-scores back to original scale:

x = z \cdot \sigma + \mu

Use case: After making predictions in standardized space, convert back to interpretable units.

Standardization vs Normalization

Standardization (Z-score):

Normalization (Min-Max):

Impact on Different Algorithms

Benefits from standardization:

Indifferent to standardization:

Numerical Stability

Catastrophic cancellation: Computing (x - \bar{x}) when x and \bar{x} are large and similar can lose precision.

Two-pass algorithm: First pass computes mean, second pass computes variance (more stable)

Welford's algorithm: Single-pass, numerically stable online algorithm for computing mean and variance

Where Z-Score Standardization Shows Up

Examples

Example 1

Input
X = [[1, 2], [3, 6], [5, 10]], axis = 0, eps = 1e-8
Output
[[-1.224745, -1.224745], [0.0, 0.0], [1.224745, 1.224745]]
Explanation
Both columns have the same relative spacing, so their standardized values match.

Hints

  1. Use np.mean(..., keepdims=True) and np.std(..., keepdims=True) along axis.
  2. Use np.where(std > eps, std, 1.0) before dividing.

Requirements

Constraints

Starter Code

import numpy as np

def zscore_standardize(X: list, axis: int = 0, eps: float = 1e-12) -> np.ndarray:
    """
    Returns population Z-scores as a NumPy array matching the shape of X.
    """
    # Write code here
    pass

Test Cases

CaseMatches
Column-wiseExample 1public