Winsorization

Feature Engineering · Data Processing

Medium

Problem

Theory

Winsorization is a technique that limits extreme values by capping them at specified percentiles. Rather than removing outliers, winsorization replaces them with the nearest non-extreme value. Named after biostatistician Charles P. Winsor, this approach preserves sample size while reducing the influence of extreme observations.

Why Winsorize?

Preserve sample size: Unlike trimming (which removes data), winsorization keeps all observations, maintaining statistical power.

Reduce outlier influence: Extreme values are capped, preventing them from dominating means, variances, and model parameters.

Maintain data structure: The number of data points remains unchanged; only extreme values are modified.

Robust statistics: Winsorized means and variances are more resistant to outliers than standard statistics.

The Winsorization Process

For winsorization at the p-th percentile on both tails:

Step 1: Compute the lower bound (p-th percentile) and upper bound ((100-p)-th percentile)

Step 2: Replace values below the lower bound with the lower bound

Step 3: Replace values above the upper bound with the upper bound

Mathematically:

x_{winsorized} = \begin{cases} L & \text{if } x < L \\ x & \text{if } L \leq x \leq U \\ U & \text{if } x > U \end{cases}

Where:

• L = lower bound (p-th percentile)
• U = upper bound ((100-p)-th percentile)

Common Winsorization Levels

5% winsorization:

• Lower 5% of values capped at 5th percentile
• Upper 5% capped at 95th percentile

1% winsorization:

• Only the most extreme 1% on each tail is capped
• Less aggressive, preserves more original data

10% winsorization:

• More aggressive outlier handling
• 20% of values total are potentially modified

Worked Example

Data: [1, 2, 3, 4, 5, 6, 7, 8, 9, 100]

Note: 100 is an extreme outlier.

10% winsorization (lower 10% and upper 10%):

Step 1 - Compute percentiles:

• 10th percentile: approximately 1.9
• 90th percentile: approximately 18.1

Step 2 - Apply caps:

• Value 1: Below 1.9 → cap at 1.9
• Values 2-9: Within bounds → unchanged
• Value 100: Above 18.1 → cap at 18.1

Winsorized data: [1.9, 2, 3, 4, 5, 6, 7, 8, 9, 18.1]

Effect on mean:

• Original mean: 14.5 (inflated by outlier)
• Winsorized mean: 6.4 (more representative)

Asymmetric Winsorization

Different percentiles for lower and upper bounds:

Example: 0% lower, 5% upper

• No capping of low values
• Only extreme high values capped at 95th percentile

Use case: When outliers are expected only in one direction (e.g., response times can be extremely high but not negative)

Winsorization vs Trimming

Winsorization (capping):

• Replaces extreme values with boundary values
• Preserves sample size
• All data points contribute to statistics

Trimming (truncation):

• Removes extreme values entirely
• Reduces sample size
• Excluded data points contribute nothing

Example with 10% on each tail, n=100:

• Winsorization: 100 values remain (10 on each tail are capped)
• Trimming: 80 values remain (10 on each tail are removed)

Winsorized Mean

The mean of winsorized data:

\bar{x}_w = \frac{1}{n} \sum_{i=1}^{n} x_{i,winsorized}

Properties:

• More robust than arithmetic mean
• Breakdown point depends on winsorization level
• Converges to regular mean as winsorization approaches 0%

Winsorized Standard Deviation

Applied to winsorized data:

s_w = \sqrt{\frac{1}{n-1} \sum_{i=1}^{n} (x_{i,winsorized} - \bar{x}_w)^2}

Note: Some formulations adjust degrees of freedom for winsorization. The standard formula treats winsorized values as real observations.

Choosing Winsorization Level

Factors to consider:

• Expected outlier proportion in data
• Sensitivity requirements of analysis
• Domain knowledge about valid data ranges

Common choices:

• 1%: Light winsorization for mild outliers
• 5%: Standard choice for moderate outliers
• 10%: Aggressive for highly contaminated data

Too aggressive: May distort the true distribution Too conservative: May not adequately address outliers

Column-wise Application

For multi-feature datasets, winsorize each column independently:

Process:

1. For each column, compute percentile bounds
2. Apply winsorization using column-specific bounds
3. Different columns may have different bound values

Result: Each feature is winsorized according to its own distribution

Relationship to IQR-Based Methods

IQR outlier detection:

• Lower: Q1 - 1.5 × IQR
• Upper: Q3 + 1.5 × IQR

Percentile-based winsorization:

• More flexible (can choose any percentile)
• Does not assume specific distribution shape

Equivalence: For normal distributions, 1.5 × IQR roughly corresponds to certain percentiles, but the methods are distinct.

Order Statistics Perspective

Winsorization replaces extreme order statistics:

For sorted data x_{(1)} \leq x_{(2)} \leq ... \leq x_{(n)}:

k-winsorization (k values on each tail):

• Replace x_{(1)}, ..., x_{(k)} with x_{(k+1)}
• Replace x_{(n-k+1)}, ..., x_{(n)} with x_{(n-k)}

This is equivalent to percentile-based winsorization with p = 100k/n.

Numerical Considerations

Percentile calculation: Various interpolation methods exist (linear, lower, higher, nearest). Choice affects boundary values slightly.

Ties at boundaries: If many values equal the boundary, they remain unchanged.

Empty tails: If percentile falls within repeated values, cap may not change any data.

Where Winsorization Shows Up

• Financial Analysis: Capping extreme returns in portfolio analysis

• Quality Control: Handling measurement errors in manufacturing data

• Survey Research: Managing extreme responses that may be errors

• Insurance: Limiting claim amounts for actuarial calculations

• Medical Statistics: Handling outlier patient measurements

• Economic Data: GDP per capita, income data with extreme values

• Preprocessing for ML: Reducing outlier influence before model training

• Robust Regression: Preparing data for models sensitive to outliers

• Image Processing: Capping pixel intensities for contrast adjustment

• Scientific Research: Managing experimental measurement errors

Examples

Hints

1. Use (len(values) - 1) * percentile / 100 for the fractional sorted index.
2. Clip each original value with max(lower_bound, min(upper_bound, value)).

Requirements

• Compute both percentile bounds using linear interpolation.
• Clip values below the lower bound to that bound.
• Clip values above the upper bound to that bound.
• Preserve the original value order and input length.

Constraints

• values is nonempty.
• 0 <= lower_pct <= upper_pct <= 100.
• Time limit: 300 ms.

Starter Code

def winsorize(values: list, lower_pct: float, upper_pct: float) -> list:
    """
    Returns values clipped to the interpolated percentile bounds.
    """
    # Write code here
    pass

Test Cases