What is a Computational Graph?
You'll learn: How computational graphs visually represent the mathematical operations inside neural networks as a flow diagram.
The Core Idea
A computational graph is a directed acyclic graph (DAG) that maps out all the mathematical operations in your neural network. Think of it as a flowchart where:
- Nodes represent operations (like addition, multiplication, or activation functions)
- Edges represent the flow of data (tensors or scalars) between operations
- The graph flows in one direction without loops (that's what "acyclic" means)
Imagine a recipe: ingredients flow through steps (chop, mix, bake) to produce a final dish. Similarly, your input data flows through operations (matrix multiply, add bias, apply activation) to produce predictions.
Why It Matters
Computational graphs make both forward propagation and backpropagation crystal clear:
- Forward pass: Follow the graph from inputs to outputs, computing values at each node
- Backward pass: Traverse the graph in reverse, applying the chain rule to compute gradients
For example, if you compute z = (x * w) + b then a = sigmoid(z), the graph shows:
- Input
xand weightw→ multiply operation → intermediate result - Add bias
b→ another intermediate result - Apply
sigmoid→ final activationa
During backpropagation, gradients flow backward through these same nodes in reverse order.
The DAG Property
"Directed acyclic" means information flows one way without circular dependencies. This structure guarantees you can always compute outputs from inputs and systematically calculate gradients layer by layer.
Key Takeaway: Computational graphs are directed acyclic graphs that visualize data flow and operations in neural networks, making forward propagation and backpropagation mechanically systematic rather than mysterious.