Double Exponential Smoothing

Time Series

Medium

Problem

Theory

Double exponential smoothing (Holt's linear method) extends simple exponential smoothing to handle data with trends. It maintains two components: level and trend.

\ell_t = \alpha y_t + (1-\alpha)(\ell_{t-1} + b_{t-1})

b_t = \beta(\ell_t - \ell_{t-1}) + (1-\beta)b_{t-1}

where \ell_t is the level, b_t is the trend, \alpha is the level smoothing parameter, and \beta is the trend smoothing parameter.

The Formulas

Level equation:

\ell_t = \alpha y_t + (1-\alpha)(\ell_{t-1} + b_{t-1})

Updates estimated level, combining observed value with forecasted level from previous period.

Trend equation:

b_t = \beta(\ell_t - \ell_{t-1}) + (1-\beta)b_{t-1}

Updates estimated trend, combining current slope with previous trend estimate.

Forecast:

\hat{y}_{t+h} = \ell_t + h \cdot b_t

Level plus trend extrapolated h steps ahead.

Parameters

Level smoothing (\alpha):

Range: 0 < \alpha < 1

• High \alpha: Responsive to recent observations
• Low \alpha: Smoother, more influenced by history

Trend smoothing (\beta):

Range: 0 < \beta < 1

• High \beta: Trend adapts quickly
• Low \beta: Trend changes slowly

Typical values: \alpha \in [0.1, 0.3], \beta \in [0.05, 0.2]

Optimization: Choose \alpha and \beta to minimize forecast error (MSE, MAE).

Initialization

Level (\ell_0):

Common: \ell_0 = y_1

Better: Average of first few observations.

Trend (b_0):

Simple: b_0 = y_2 - y_1

Better: Linear regression slope on first few observations.

Example: For first 4 values [10, 12, 15, 17]:

\ell_0 = 10

b_0 = \frac{17-10}{3} = 2.33

Worked Example

Data: [20, 22, 25, 27, 30]

Parameters: \alpha = 0.3, \beta = 0.1

Initialization: \ell_0 = 20, b_0 = 2

Period 1 (t=1, y_1=20):

\ell_1 = 0.3(20) + 0.7(20 + 2) = 6 + 15.4 = 21.4

b_1 = 0.1(21.4 - 20) + 0.9(2) = 0.14 + 1.8 = 1.94

Period 2 (t=2, y_2=22):

\ell_2 = 0.3(22) + 0.7(21.4 + 1.94) = 6.6 + 16.338 = 22.938

b_2 = 0.1(22.938 - 21.4) + 0.9(1.94) = 0.1538 + 1.746 = 1.900

Period 3 (t=3, y_3=25):

\ell_3 = 0.3(25) + 0.7(22.938 + 1.900) = 7.5 + 17.386 = 24.886

b_3 = 0.1(24.886 - 22.938) + 0.9(1.900) = 0.1948 + 1.71 = 1.905

Forecast for t=4:

\hat{y}_4 = \ell_3 + 1 \cdot b_3 = 24.886 + 1.905 = 26.791

Actual y_4 = 27, error = $$

Interpretation of Components

Level (\ell_t):

Current baseline value of the series.

Trend (b_t):

Rate of change per period. Positive for increasing, negative for decreasing.

Combined forecast:

\hat{y}_{t+h} = \ell_t + h \cdot b_t

Linear extrapolation from current level with current trend.

Multi-Step Forecasting

One-step ahead:

\hat{y}_{t+1} = \ell_t + b_t

Two-steps ahead:

\hat{y}_{t+2} = \ell_t + 2b_t

h-steps ahead:

\hat{y}_{t+h} = \ell_t + hb_t

Linear forecast function: Straight line with slope b_t.

Limitation: Assumes constant trend. May overestimate for distant horizons.

Damped Trend

Problem: Linear trend forecasts often overshoot.

Solution: Add damping parameter \phi:

\hat{y}_{t+h} = \ell_t + (\phi + \phi^2 + ... + \phi^h)b_t

where 0 < \phi < 1.

Simplified:

\hat{y}_{t+h} = \ell_t + \frac{\phi(1-\phi^h)}{1-\phi}b_t

Effect: Trend flattens as horizon increases. More conservative forecasts.

When \phi=1: Reduces to standard double exponential smoothing.

Comparison to Simple Exponential Smoothing

Simple (SES):

\ell_t = \alpha y_t + (1-\alpha)\ell_{t-1}

Flat forecast: \hat{y}_{t+h} = \ell_t

Double (DES):

Adds trend component. Forecast has slope.

Use simple when: No trend present.

Use double when: Clear upward or downward trend.

Test: If trend is significant, DES will outperform SES.

Connection to ARIMA

Double exponential smoothing is equivalent to ARIMA(0,2,2):

DES forecast function:

\hat{y}_{t+h} = y_t + h \cdot \hat{b}_t

ARIMA(0,2,2) forecast function:

Same form after appropriate parameterization.

Advantage: ARIMA provides statistical framework (confidence intervals, diagnostics).

Advantage DES: Simpler interpretation and implementation.

Adaptive Smoothing

Problem: Fixed \alpha and \beta may not adapt to changing dynamics.

Adaptive methods:

Adjust parameters based on recent forecast errors.

Example: Increase \alpha when errors are large, decrease when errors are small.

Trigg-Leach method:

\alpha_t = |\frac{\text{Smoothed error}}{\text{MAD}}|

Trade-off: Added complexity vs improved adaptability.

Forecast Error Metrics

Mean Absolute Error (MAE):

\text{MAE} = \frac{1}{T} \sum_{t=1}^{T} |y_t - \hat{y}_t|

Mean Squared Error (MSE):

\text{MSE} = \frac{1}{T} \sum_{t=1}^{T} (y_t - \hat{y}_t)^2

Root Mean Squared Error (RMSE):

\text{RMSE} = \sqrt{\text{MSE}}

Optimization: Choose \alpha and \beta to minimize chosen metric.

Parameter Optimization

Grid search:

Try all combinations of \alpha and \beta in increments (e.g., 0.05).

Evaluate forecast error for each combination.

Select parameters with lowest error.

Numerical optimization:

Use gradient-based methods (Nelder-Mead, L-BFGS).

Minimize error function directly.

Cross-validation:

Split data into training and validation sets.

Optimize on training, evaluate on validation.

Prevents overfitting to specific sample.

Handling Level Shifts

Problem: Sudden permanent change in level.

Example: Price increase after policy change.

DES response:

Level \ell_t adapts gradually.

Trend b_t temporarily spikes then returns to baseline.

Better approach: Intervention analysis or structural break models.

Robust DES: Use robust estimation (median instead of mean) to reduce outlier impact.

Seasonal Extension

Triple exponential smoothing (Holt-Winters):

Adds seasonal component to level and trend.

Additive seasonal:

\hat{y}_{t+h} = \ell_t + hb_t + s_{t+h-m}

Multiplicative seasonal:

\hat{y}_{t+h} = (\ell_t + hb_t)s_{t+h-m}

where s_t is seasonal component and m is seasonal period.

Confidence Intervals

Prediction intervals for DES:

\hat{y}_{t+h} \pm z_{\alpha/2} \sigma \sqrt{h}

where \sigma is estimated standard deviation of one-step-ahead errors.

Limitation: Assumes constant variance and normally distributed errors.

Better approach: Use state space formulation for exact prediction intervals.

State Space Representation

State equations:

\begin{bmatrix} \ell_t \\ b_t \end{bmatrix} = \begin{bmatrix} 1 & 1 \\ 0 & 1 \end{bmatrix} \begin{bmatrix} \ell_{t-1} \\ b_{t-1} \end{bmatrix} + \begin{bmatrix} \alpha \\ \beta \end{bmatrix} e_t

Observation equation:

y_t = \begin{bmatrix} 1 & 0 \end{bmatrix} \begin{bmatrix} \ell_t \\ b_t \end{bmatrix} + e_t

Advantage: Kalman filter can be applied. Maximum likelihood estimation. Rigorous prediction intervals.

Dealing with Negative Forecasts

Problem: DES can produce negative forecasts if trend is strongly negative.

Issue: Unacceptable for positive variables (prices, quantities).

Solutions:

1. Log transformation: Model \ln(y_t), exponentiate forecasts
2. Constrain forecasts: \hat{y}_{t+h} = \max(\hat{y}_{t+h}, 0)
3. Use multiplicative methods instead of additive

Computational Efficiency

Per-observation cost: O(1)

Update level and trend with simple arithmetic.

Total cost: O(T) for series of length T.

Comparison:

• ARIMA: O(T^3) for estimation
• Regression: O(T^2)
• Neural networks: O(T \cdot p) per epoch

Advantage: DES is extremely fast. Suitable for real-time applications and large-scale forecasting.

Bias and Consistency

Bias: DES forecasts can be biased if true model differs from assumed form.

Consistency: As T \to \infty, estimates converge to optimal values under correct specification.

Practical: Often used as quick baseline. More complex models tested against DES benchmark.

Trend Reversal Detection

Monitoring b_t:

Track sign and magnitude of trend component.

Sign change: b_t switches from positive to negative indicates potential reversal.

Magnitude decrease: |b_t| shrinking suggests trend weakening.

Application: Early warning system for trend changes in business metrics.

Rolling vs Expanding Window

Expanding window:

Use all available data for each forecast.

Updates incorporate entire history.

Rolling window:

Use only most recent w observations.

\ell_t = \alpha y_t + (1-\alpha)(\ell_{t-1} + b_{t-1})

computed only on last w points.

Trade-off: Rolling adapts to regime changes, expanding uses more information.

Residual Analysis

One-step-ahead errors:

e_t = y_t - \hat{y}_t = y_t - (\ell_{t-1} + b_{t-1})

Diagnostic checks:

1. Plot residuals over time (should be random)
2. ACF of residuals (should show no autocorrelation)
3. Histogram of residuals (should be approximately normal)
4. Heteroscedasticity test (variance should be constant)

Good fit: Residuals are white noise.

Poor fit: Patterns in residuals indicate model inadequacy.

Outlier Handling

Impact: Outliers disproportionately affect level and trend estimates.

Detection: Identify |e_t| > k \sigma where k=3 (3-sigma rule).

Treatment:

1. Replace outlier with forecast: y_t^* = \hat{y}_t
2. Use robust smoothing: Replace mean with median in equations
3. Winsorize: Cap extreme values

Trade-off: Over-correction removes legitimate signals. Under-correction allows contamination.

Comparison to Linear Regression

Linear regression on time:

\hat{y}_t = \hat{\beta}_0 + \hat{\beta}_1 t

Fixed slope throughout.

Double exponential smoothing:

Slope b_t evolves over time.

DES advantage: Adapts to changing trends. Captures evolving dynamics.

Regression advantage: Simple, interpretable, provides confidence intervals directly.

Practical Applications

Sales forecasting:

Track baseline sales level and growth rate.

Inventory management:

Predict demand to optimize stock levels.

Financial metrics:

Revenue, costs, customer acquisition trends.

Web analytics:

Traffic growth, conversion rate trends.

Energy demand:

Load forecasting with trend component.

Examples

Hints

1. Store the old level until both update equations have been evaluated.
2. Append the initial level before looping from index 1.

Requirements

• Initialize level from the first value.
• Initialize trend as the second value minus the first.
• Update level before using the new level to update trend.
• Return one level value for every input observation.

Constraints

• series contains at least two numeric values.
• 0 < alpha <= 1.
• 0 < beta <= 1.
• Time limit: 300 ms.

Starter Code

def double_exponential_smoothing(series: list, alpha: float, beta: float) -> list:
    """
    Returns the smoothed level at every time step.
    """
    # Write code here
    pass

Test Cases