VGG Configuration

Very Deep Convolutional Networks

Easy

Problem

Theory

A VGG configuration is a flat list of integers and the character 'M' that fully specifies the convolutional feature-extraction backbone of a VGGNet. Simonyan and Zisserman (2014) introduced this compact representation in "Very Deep Convolutional Networks for Large-Scale Image Recognition," defining four primary variants (VGG11, VGG13, VGG16, VGG19) that differ only in the number of convolutional layers per spatial block.

What It Is

VGGNet's architecture can be described entirely by a single ordered list. Each element is either an integer or the string 'M'. An integer specifies a convolutional layer whose output has that many channels (all convolutions use 3 \times 3 kernels, stride 1, padding 1, followed by ReLU). The character 'M' denotes a $ \times 2$ max-pooling layer with stride 2, which halves the spatial resolution.

Reading the list from left to right reproduces the entire feature extractor. There is no ambiguity about kernel sizes, strides, or padding because VGGNet uses the same values everywhere: every conv is 3 \times 3 with stride 1 and padding 1, every pool is $ \times 2$ with stride 2. The only things that change between layers are the number of output channels and whether the operation is a convolution or a pooling.

This design philosophy was deliberate. The paper's central thesis is that network depth is the critical variable for performance on image classification tasks. By fixing all hyperparameters except depth (and the channel count that naturally grows with depth), the authors isolated depth as the single experimental variable.

Key Equations

Because every convolution uses padding 1 with a 3 \times 3 kernel at stride 1, spatial dimensions are preserved through convolutions. Spatial downsampling happens exclusively at the 'M' (max pool) layers:

H_{out} = \lfloor H_{in} / 2 \rfloor, \quad W_{out} = \lfloor W_{in} / 2 \rfloor

Starting from a 224 \times 224 input, each of the five pool layers halves the resolution:

224 \xrightarrow{M} 112 \xrightarrow{M} 56 \xrightarrow{M} 28 \xrightarrow{M} 14 \xrightarrow{M} 7

The channel progression doubles after each pool, starting at 64 and capping at 512:

64 \xrightarrow{M} 128 \xrightarrow{M} 256 \xrightarrow{M} 512 \xrightarrow{M} 512

The total number of convolutional layers and weight layers across the four variants:

• VGG11: 8 conv + 3 FC = 11 weight layers
• VGG13: 10 conv + 3 FC = 13 weight layers
• VGG16: 13 conv + 3 FC = 16 weight layers
• VGG19: 16 conv + 3 FC = 19 weight layers

The "weight layers" in the name count only layers with learnable parameters: convolutions and fully connected layers. Max-pool and ReLU layers have no parameters and are not counted.

The Four Variants

The paper presents configurations A through E in Table 1. Each variant organizes its convolutional layers into five spatial blocks separated by max-pool operations. The variants differ only in how many conv layers appear in each block:

VGG11 (Configuration A)

Block structure: 1-1-2-2-2 (convs per block). The shallowest variant with 8 conv layers total. One conv at 64 channels, one at 128, two at 256, two at 512, two at 512.

VGG13 (Configuration B)

Block structure: 2-2-2-2-2 (convs per block). The uniform variant with exactly two convolutions in every block, giving 10 conv layers total.

VGG16 (Configuration D)

Block structure: 2-2-3-3-3 (convs per block). The most widely used VGGNet variant with 13 conv layers. It adds a third conv layer to each of the last three blocks compared to VGG13. VGG16 became the de facto feature extractor for tasks like object detection (Faster R-CNN) and style transfer.

VGG19 (Configuration E)

Block structure: 2-2-4-4-4 (convs per block). The deepest variant with 16 conv layers. The paper found that going beyond 19 weight layers did not improve accuracy on ImageNet, suggesting a saturation point for this architecture (which residual connections later overcame).

Note that VGG16 is "Configuration D" in the paper, not "Configuration C." Configuration C uses 1 \times 1 convolutions in some positions and is rarely used in practice. The standard VGG16 with all $ \times 3$ convolutions is Configuration D.

The Config List Format

The config list is a Python-style flat list where each element is either an integer or the string 'M'. Reading from left to right:

• An integer (e.g., 64, 128, 256, 512) means "create a convolutional layer with this many output channels." The kernel is always 3 \times 3, stride is always 1, padding is always 1. A ReLU activation follows each conv.
• The character 'M' means "insert a 2 \times 2 max-pooling layer with stride 2." This halves the spatial dimensions and marks the boundary between spatial blocks.

For VGG16, the config list is:

[64, 64, \text{'M'}, 128, 128, \text{'M'}, 256, 256, 256, \text{'M'}, 512, 512, 512, \text{'M'}, 512, 512, 512, \text{'M'}]

This list has 13 integers (conv layers) and 5 'M' entries (pool layers), totaling 18 elements. The integers group naturally by their values: two 64s, two 128s, three 256s, three 512s, three 512s. Each group forms a spatial block, and the 'M' between groups marks the resolution reduction.

To build the network from this list, iterate through it: for each integer, create a Conv2d with the appropriate input channels (3 for the first layer, or the previous conv's output channels) and the integer as output channels, followed by ReLU. For each 'M', create a MaxPool2d. This is exactly how the torchvision implementation of VGGNet works.

Channel Doubling Pattern

The channel counts follow a geometric progression, doubling after each pool:

64 \to 128 \to 256 \to 512 \to 512

The doubling starts at 64 channels and continues until 512. The fourth and fifth blocks both use 512 rather than continuing to 1024. This cap exists for practical reasons:

• Memory: By the fourth block, spatial dimensions are 14 \times 14. With 512 channels, the activation tensor is $ \times 14 \times 512 = 100{,}352$ values per image. Doubling to 1024 would double memory for activations, gradients, and parameters.
• Parameters: Each conv layer in the fifth block has 512 \times 512 \times 3 \times 3 = 2{,}359{,}296 parameters. At 1024 channels, that becomes $$ per layer. The FC layer after the final pool (7 \times 7 \times 512 = 25{,}088 inputs to 4096 outputs) already has \sim parameters.

The doubling compensates for spatial shrinking. After a 2 \times 2 pool, spatial area drops by 4\times. Doubling channels means the total "feature units" (positions times channels) drops by only 2\times per block, keeping computational load roughly balanced.

This pattern of doubling channels while halving spatial resolution was not invented by VGGNet (LeNet and AlexNet used similar ideas), but VGGNet made it systematic. This exact pattern became the template for nearly every CNN that followed, including ResNet and DenseNet.

Paper Context

Simonyan and Zisserman submitted "Very Deep Convolutional Networks for Large-Scale Image Recognition" to ICLR 2015, with the arXiv preprint appearing in September 2014. The paper's core contribution is a systematic study of how network depth affects classification accuracy on ImageNet.

The paper presents its architectures in Table 1 as configurations A through E:

• Configuration A (VGG11): 11 weight layers, the baseline.
• Configuration A-LRN: Same as A with Local Response Normalization. The paper found LRN did not help, contradicting AlexNet's design.
• Configuration B (VGG13): 13 weight layers.
• Configuration C: 16 weight layers with 1 \times 1 convolutions in blocks 3-5. A stepping stone to test extra nonlinearity without adding receptive field.
• Configuration D (VGG16): 16 weight layers, all 3 \times 3. Outperforms C, showing spatial context matters more than 1 \times 1 nonlinearity.
• Configuration E (VGG19): 19 weight layers, the deepest. Slightly better than D with diminishing returns.

The paper's key finding: "The configurations improve from A to E by increasing the depth: from 11 to 19 weight layers." Error rates decreased monotonically, with the largest gains early (A to B, B to D) and marginal improvement from D to E.

VGGNet placed second in ILSVRC-2014 classification (behind GoogLeNet) but won the localization task. Despite not winning classification outright, VGG16 became far more widely adopted because its uniform, simple architecture was easy to implement, modify, and use for transfer learning.

Numerical Example

Consider VGG16. The full config list has 18 elements:

[64, 64, \text{'M'}, 128, 128, \text{'M'}, 256, 256, 256, \text{'M'}, 512, 512, 512, \text{'M'}, 512, 512, 512, \text{'M'}]

Walking through the list with input shape 224 \times 224 \times 3:

Block 1 (64 channels):

• Conv2d(3 \to 64) + ReLU, then Conv2d($ \to 64$) + ReLU. Spatial dims preserved: 224 \times 224.
• 'M': MaxPool2d(2 \times 2, stride 2). Output: $ \times 112 \times 64$.

Block 2 (128 channels):

• Conv2d(64 \to 128) + ReLU, then Conv2d($ \to 128$) + ReLU. Spatial: 112 \times 112.
• 'M': pool to 56 \times 56 \times 128.

Block 3 (256 channels):

• Three convolutions: 128 \to 256, 256 \to 256, 256 \to 256, each with ReLU. Spatial: 56 \times 56.
• 'M': pool to 28 \times 28 \times 256.

Block 4 (512 channels):

• Three convolutions: 256 \to 512, 512 \to 512, 512 \to 512. Spatial: 28 \times 28.
• 'M': pool to 14 \times 14 \times 512.

Block 5 (512 channels):

• Three convolutions: 512 \to 512, 512 \to 512, 512 \to 512. Spatial: 14 \times 14.
• 'M': pool to 7 \times 7 \times 512.

After the feature extractor, the output 7 \times 7 \times 512 = 25{,}088 values are flattened and fed to three fully connected layers: FC(25088, 4096) + ReLU + Dropout, FC(4096, 4096) + ReLU + Dropout, FC(4096, 1000). The FC layers are identical across all variants; only the config list changes.

Counting from the list: 13 integers = 13 conv layers, 5 'M' entries = 5 pool layers. Adding 3 FC layers gives 13 + 3 = 16 weight layers, confirming "VGG16."

The Depth vs Width Tradeoff

VGGNet made a deliberate choice: depth over width. Instead of large kernels (5 \times 5, 7 \times 7, 11 \times 11 as in AlexNet), VGGNet exclusively uses 3 \times 3 convolutions and compensates by stacking many layers.

The paper explicitly argues for this. Two stacked 3 \times 3 conv layers have an effective receptive field equivalent to one 5 \times 5 conv. Three stacked 3 \times 3 layers match one 7 \times 7 conv. The advantage of stacking small convolutions is twofold:

• More nonlinearity: Two 3 \times 3 layers have two ReLU activations, while one 5 \times 5 layer has only one. More nonlinear activations let the network represent more complex decision boundaries.
• Fewer parameters: Two 3 \times 3 layers have 2 \times (3^2 \times C^2) = 18C^2 parameters vs. one 5 \times 5 layer's 25C^2. Three 3 \times 3 layers have 27C^2 vs. one 7 \times 7 layer's 49C^2.

By keeping all convolutions at 3 \times 3 and channel counts moderate (64 to 512), VGGNet avoids the combinatorial explosion of hyperparameters in earlier architectures. The only variable is depth, which the paper systematically increases from 11 to 19 layers.

This simplicity has a practical benefit: the config list is trivially parameterizable. Changing the architecture means changing a single list of integers and 'M' characters. No kernel sizes, strides, or padding values need to be specified per layer.

Pitfalls

• Wrong number of convolutions per block. The block structures must be memorized precisely: VGG11 is 1-1-2-2-2, VGG13 is 2-2-2-2-2, VGG16 is 2-2-3-3-3, VGG19 is 2-2-4-4-4. A common mistake is writing VGG16 as 2-2-2-3-3 or 3-3-3-2-2. Count the integers between each pair of 'M' entries to verify.

• Forgetting the 'M' entries. The config list must include all five 'M' entries, one after each spatial block. Without them, the network has no pooling and spatial dimensions never decrease. A list with only integers would produce a network where every layer operates at 224 \times 224.

• Wrong channel counts. The channel sequence is 64, 128, 256, 512, 512. A common error is 64, 128, 256, 512, 1024 (continuing the doubling) or mixing up which blocks use 256 vs 512. Within a block, all conv layers use the same output channel count.

• Case sensitivity in variant names. The problem accepts variant names case-insensitively, so "VGG16", "vgg16", and "Vgg16" should all produce the same config list. Failing to normalize the input string before lookup is a common implementation bug.

• Confusing paper configuration letters with variant numbers. VGG16 is Configuration D, not Configuration C. Configuration C uses 1 \times 1 convolutions and is a different architecture. Similarly, VGG11 is A, VGG13 is B, and VGG19 is E.

• Including FC layers in the config list. The config list describes only the convolutional feature extractor. The three fully connected layers (4096, 4096, 1000) are the same for all variants and are not part of the config list.

• Off-by-one in total layer counting. The "16" in VGG16 counts weight layers (conv + FC), not total layers. It does not count ReLU, pooling, dropout, or softmax. VGG16 has 13 conv + 3 FC = 16 weight layers, but the actual forward pass involves many more operations.

Examples

Hints

1. Normalize the variant with str.lower().
2. Store the four canonical configurations in a dictionary.

Requirements

• Use ordinary Python lists and strings.
• Support VGG11, VGG13, VGG16, and VGG19.
• Treat variant names case-insensitively.
• Return a new Python list in canonical layer order.

Constraints

• variant names one of the four supported VGG architectures, ignoring case.

Starter Code

def make_vgg_config(variant: str) -> list:
    """
    Returns the canonical VGG layer configuration as a new list.
    """
    pass

Test Cases