Course contentsShow
← Machine Learning and Deep Learning
Lesson 596 of 3,53815. Neural Network FoundationsFree lesson

Network Architecture Terminology: Depth and Width

Understanding network depth (number of layers), width (neurons per layer), and architecture notation.

Network Architecture Terminology: Depth and Width

What You'll Learn

You'll learn how to describe neural network architectures using two fundamental dimensions: depth (how many layers) and width (how many neurons per layer).

The Core Idea

When we describe a neural network's architecture, we use precise terminology to communicate its structure:

Depth refers to the number of layers in your network. Count all layers that process information: input layer, hidden layers, and output layer. A network with 1 input layer, 3 hidden layers, and 1 output layer has a depth of 5. The term "deep learning" literally comes from using networks with many layers—high depth.

Width refers to how many neurons exist in a single layer. Different layers can have different widths. For example, your first hidden layer might have 128 neurons, while your second has 64.

Architecture Notation

We commonly describe networks using notation like:

  • 784-128-64-10: Input layer (784 neurons), two hidden layers (128 and 64 neurons), output layer (10 neurons)
  • This network has depth = 4 and varying width per layer

Think of depth like floors in a building—more floors mean taller (deeper). Width is like the size of each floor—more rooms (neurons) mean wider processing capacity at that level.

Why It Matters

Depth and width control your network's capacity to learn complex patterns. Deeper networks can learn hierarchical features (edges → shapes → objects), while wider layers can capture more diverse features at each level. The universal approximation theorem tells us even shallow networks work, but deeper architectures often learn more efficiently.

Key Takeaway: Network depth counts the total layers, width measures neurons per layer, and together they define your architecture's learning capacity and computational cost.