Course contentsShow
← Machine Learning and Deep Learning
Lesson 865 of 3,53820. Convolutional Neural Networks: FundamentalsFree lesson

Grouped Convolution

Splitting input channels into groups to reduce computation and parameters.

Grouped Convolution

What you'll learn: How splitting input channels into independent groups reduces computation and parameters in convolutional layers.

The Core Idea

In standard convolution, every output channel connects to all input channels. Grouped convolution splits both input and output channels into separate groups, where each group's filters only process their assigned input channels. This dramatically reduces parameters and computation.

Think of it like organizing a large kitchen: instead of every chef (output channel) needing ingredients from every pantry (input channel), you divide the kitchen into stations. The dessert station only accesses the baking ingredients, the grill station only uses meat and vegetables. Each station works independently but in parallel.

How It Works

With groups=g:

  • Input channels are divided into g equal groups
  • Output channels are also divided into g equal groups
  • Each group's filters only convolve with their corresponding input group
  • Results are concatenated along the channel dimension

Example: A layer with 64 input channels, 128 output channels, and groups=4:

  • Each group processes 64/4 = 16 input channels
  • Each group produces 128/4 = 32 output channels
  • Four independent convolutions run in parallel

Parameter Reduction

For a 3×3 convolution:

  • Standard: 64 × 128 × 3 × 3 = 73,728 parameters
  • Grouped (g=4): 16 × 32 × 3 × 3 × 4 = 18,432 parameters

That's 4× fewer parameters while maintaining representational capacity through parallel processing pathways.

When to Use It

Grouped convolution offers a tunable trade-off between efficiency and mixing information across channels. It's especially valuable in resource-constrained models like MobileNets and was popularized by AlexNet for multi-GPU training.

Key Takeaway: Grouped convolution splits channels into independent processing groups, reducing parameters and computation by limiting cross-channel connectivity while enabling efficient parallel pathways.