AI Engineer Exhaustive
EasyDDPM

Noise Schedule

Denoising Diffusion Probabilistic Models

Easy

Problem

Implement the linear variance schedule used in the original DDPM experiments. For T diffusion steps, create equally spaced values from \beta_1 through \beta_T, including both endpoints.

\beta_t = \beta_1 + \frac{t-1}{T-1}(\beta_T-\beta_1), \qquad t=1,\ldots,T.

Here, T is the number of diffusion steps and \beta_t is the variance added at step t. When T=1, return [\beta_1]. Return the schedule as a list of Python floats rounded to six decimals.

Theory

The noise schedule is the sequence of variance values \beta_1, \beta_2, \dots, \beta_T that controls how much Gaussian noise is injected at each timestep of the forward diffusion process. In Denoising Diffusion Probabilistic Models (Ho, Gulrajani, and Abbeel, 2020), this schedule is the backbone of the generative framework. Every downstream quantity -- noisy training samples, signal-to-noise ratio, and network prediction targets -- derives from this sequence of scalars.

A poorly chosen schedule either wastes steps where the data is already destroyed, or fails to fully corrupt the input into pure noise by the final timestep.

What It Is / What It Does

The noise schedule defines the forward process q(x_t | x_{t-1}) at every timestep. At each step t, Gaussian noise scaled by \beta_t is added and the signal is scaled by \sqrt{1 - \beta_t}. Over T steps, this transforms any data distribution into an isotropic Gaussian.

The schedule is a fixed hyperparameter, not learned. From \beta_t, two derived quantities govern all computations:

Key Equations

The forward process at each step:

q(x_t | x_{t-1}) = \mathcal{N}(x_t; \sqrt{1 - \beta_t}\, x_{t-1},\; \beta_t \mathbf{I})

The marginal q(x_t | x_0) has a closed form via \bar{\alpha}_t:

q(x_t | x_0) = \mathcal{N}(x_t; \sqrt{\bar{\alpha}_t}\, x_0,\; (1 - \bar{\alpha}_t) \mathbf{I})

Any noisy sample at step t can be written as:

x_t = \sqrt{\bar{\alpha}_t}\, x_0 + \sqrt{1 - \bar{\alpha}_t}\, \epsilon, \quad \epsilon \sim \mathcal{N}(0, \mathbf{I})

The derived quantities:

The linear schedule defines \beta_t by interpolation:

\beta_t = \beta_1 + \frac{t - 1}{T - 1}(\beta_T - \beta_1)

where \beta_1 = 10^{-4}, \beta_T = 0.02, and T = 1000.

The cosine schedule from Nichol and Dhariwal (2021) defines \bar{\alpha}_t directly:

\bar{\alpha}_t = \frac{f(t)}{f(0)}, \quad f(t) = \cos\left(\frac{t/T + s}{1 + s} \cdot \frac{\pi}{2}\right)^2

where s = 0.008 is a small offset preventing \beta_t from being too small near t = 0.

Linear Schedule

Ho et al. chose the simplest possible schedule: linear interpolation of \beta_t from \beta_1 = 10^{-4} to \beta_T = 0.02 over T = 1000 steps.

With these values, \bar{\alpha}_t starts near 1 and decays monotonically. Early on, each \alpha_t \approx 0.9999 so the product barely budges. In the middle, \bar{\alpha}_t drops steadily. Near the end, it approaches zero rapidly.

The linear schedule has a known asymmetry: \bar{\alpha}_t spends many early timesteps near 1 and late timesteps near 0. The useful transition zone is concentrated in the middle third. The model wastes capacity on near-trivial early denoising and near-impossible late denoising.

Cosine Schedule

Nichol and Dhariwal (2021) identified the linear schedule's inefficiency: too many timesteps are spent where \bar{\alpha}_t is near 1 or near 0. Their solution: define \bar{\alpha}_t directly via a cosine function, then extract \beta_t.

\bar{\alpha}_t = \frac{f(t)}{f(0)}, \quad f(t) = \cos\left(\frac{t/T + s}{1 + s} \cdot \frac{\pi}{2}\right)^2

Key properties:

Once \bar{\alpha}_t is defined, \beta_t is recovered as:

\beta_t = 1 - \frac{\bar{\alpha}_t}{\bar{\alpha}_{t-1}}

Nichol and Dhariwal clip \beta_t to a maximum of 0.999 to prevent numerical instability near the end of the schedule. The practical benefit: the SNR decreases more gradually, more timesteps are spent in the informative middle range, and sample quality improves.

What \bar{\alpha}_t Represents

\bar{\alpha}_t captures total signal retention from x_0 to x_t. It is the most important quantity in the diffusion framework.

The formal signal-to-noise ratio (SNR) at timestep t:

\text{SNR}(t) = \frac{\bar{\alpha}_t}{1 - \bar{\alpha}_t}

The DDPM loss can be rewritten in terms of SNR, and the timestep weighting implicitly controls focus on different noise levels.

The coefficients satisfy \bar{\alpha}_t + (1 - \bar{\alpha}_t) = 1, so total variance is preserved at every timestep. This variance-preserving property makes the forward process well-behaved.

Paper Context

Ho et al. (2020) made deliberately simple choices to show diffusion models could compete with GANs.

The simplified training loss:

L_{\text{simple}} = \mathbb{E}_{t, x_0, \epsilon}\left[\|\epsilon - \epsilon_\theta(x_t, t)\|^2\right]

where t \sim \text{Uniform}\{1, \dots, T\}. The schedule enters through x_t = \sqrt{\bar{\alpha}_t}\, x_0 + \sqrt{1 - \bar{\alpha}_t}\, \epsilon. Uniform t sampling with the linear schedule over-trains on easy and hard regions, under-trains on the informative middle. Nichol and Dhariwal later showed this is suboptimal.

Despite its simplicity, the linear schedule yielded an FID of 3.17 on CIFAR-10, establishing diffusion models as competitive.

Numerical Example

Linear schedule with \beta_1 = 10^{-4}, \beta_T = 0.02, T = 1000. The step size:

\Delta\beta = \frac{\beta_T - \beta_1}{T - 1} = \frac{0.02 - 0.0001}{999} \approx 1.99 \times 10^{-5}

Early timesteps (t = 1, 2, 3):

After 3 steps, \bar{\alpha}_3 \approx 0.9996. In the sampling equation, the noise coefficient is only \sqrt{0.0004} = 0.02. The noisy image is visually indistinguishable from the original.

Late timesteps (t = 999, 1000):

The cumulative product at t = 1000 is computed via log-sum:

\log \bar{\alpha}_{1000} = \sum_{s=1}^{1000} \log(1 - \beta_s) \approx -10.0

This gives \bar{\alpha}_{1000} \approx e^{-10.0} \approx 4.5 \times 10^{-5}. The sampling equation becomes x_{1000} \approx 0.0067\, x_0 + 0.9999\, \epsilon, which is nearly pure noise.

The decay profile across timesteps:

This shows the asymmetry: the first 250 steps only destroy 6% of the signal, while steps 750-1000 are spent where the signal is already 95%+ destroyed. The cosine schedule redistributes timesteps to spend more time in \bar{\alpha}_t \in [0.1, 0.9].

Modern Context

The linear schedule was a starting point. Subsequent work produced many alternatives.

The trend is toward either simpler schedules (flow matching removes them entirely) or adaptive ones (continuous-time solvers choose step sizes). Ho et al.'s linear schedule remains a baseline and pedagogical reference.

Pitfalls

\bar{\alpha}_T Not Reaching Near Zero

If \bar{\alpha}_T is not close to zero, q(x_T | x_0) differs from \mathcal{N}(0, \mathbf{I}). The reverse process starts from this prior, so any mismatch causes systematic bias. With Ho et al.'s schedule, \bar{\alpha}_{1000} \approx 4.5 \times 10^{-5}, small enough to avoid this.

Numerical Underflow in Cumulative Products

Computing \prod_{s=1}^{t} (1 - \beta_s) as a running product in float32 can underflow to zero for large $$, making the loss gradient degenerate. Fix: compute \log \bar{\alpha}_t = \sum \log(1 - \beta_s) in log-space and exponentiate only when needed, or use float64.

Off-by-One in Timestep Indexing

Some implementations index from 0 to T-1, others from 1 to $$. The formula assumes 1-based indexing. Using 0-based without adjusting shifts the entire schedule. Always verify \beta at the first index equals \beta_1 and at the last equals \beta_T.

Confusing \beta_t with \alpha_t

\beta_t is noise variance; \alpha_t = 1 - \beta_t is signal retention. Passing \beta_t where \alpha_t is expected inverts the noise level -- the model sees pure noise at t = 1 and clean data at t = T. Training loss explodes immediately.

Wrong T Value

If training uses T = 1000 but sampling uses T = 500 without recomputing the schedule, every \bar{\alpha}_t is wrong. Fix: retrain with the new T or use DDIM-style timestep skipping (subset of the original 1000 steps).

\beta_t Too Large

Large \beta_t values (e.g., > 0.1) break the Gaussian approximation in the reverse process. The cosine schedule clips \beta_t at 0.999 for this reason.

Examples

Example 1

Input
T=5, beta_1=0.0001, beta_T=0.02
Output
[0.0001,0.005075,0.01005,0.015025,0.02]
Explanation
Five equally spaced variance values include both supplied endpoints.

Example 2

Input
T=4, beta_1=0.001, beta_T=0.01
Output
[0.001,0.004,0.007,0.01]

Example 3

Input
T=1, beta_1=0.005, beta_T=0.005
Output
[0.005]

Hints

  1. np.linspace includes both endpoints by default.
  2. Convert the rounded array to a Python list.

Requirements

Constraints

Starter Code

import numpy as np

def linear_beta_schedule(T: int,
                         beta_1: float = 0.0001,
                         beta_T: float = 0.02) -> list[float]:
    """
    Returns T linearly spaced beta values.
    """
    pass

Test Cases

CaseMatches
Linear schedule 1—public
Linear schedule 2—public
Linear schedule 3—public