BiteSizedChunks.comLearn one small thing at a time.

Course contentsShow

Machine Learning and Deep Learning

889LeNet-5: The First Successful CNN
890AlexNet: The Deep Learning Revolution
891AlexNet's Key Innovations
892VGGNet: Depth Through Simplicity
893VGG Block Pattern and Design Principles
894GoogLeNet and the Inception Module
895Inception Module: Multi-Path Architecture
8961×1 Convolutions for Dimensionality Reduction
897Global Average Pooling vs Fully Connected
898Auxiliary Classifiers for Training Deep Networks
899Comparing Early Architectures: Trade-offs
900Architectural Evolution: From AlexNet to ResNet
901The Degradation Problem in Deep Networks
902Skip Connections: The Core Idea
903Residual Learning Formulation
904The Residual Block Architecture
905ResNet-34 and ResNet-50 Architecture
906Bottleneck Residual Blocks
907Gradient Flow Through Skip Connections
908Identity vs Projection Shortcuts
909Pre-Activation Residual Blocks
910ResNet Family: 18, 34, 50, 101, 152
911Wide Residual Networks (WRN)
912ResNeXt: Aggregated Residual Transformations
913Residual Networks in Practice
914Why Residual Networks Revolutionized Deep Learning
915Network Efficiency Metrics and Design Principles
916Depthwise Separable Convolutions
917MobileNetV1: Efficient Architecture for Mobile
918MobileNetV2: Inverted Residuals and Linear Bottlenecks
919MobileNetV3: Neural Architecture Search and Optimizations
920EfficientNet: Compound Scaling
921EfficientNet Architecture and MBConv Blocks
922Neural Architecture Search for CNNs
923ShuffleNet: Channel Shuffle Operations
924SqueezeNet: Fire Modules and Compression
925GhostNet: Cheap Operations for Redundant Features
926ConvNeXt: Modernizing CNNs with Transformer Principles
927RegNet: Design Space Analysis
928Hardware-Aware Architecture Design
929Dynamic Networks and Early Exit
930Comparing Efficiency vs Accuracy Trade-offs

Machine Learning and Deep Learning

889LeNet-5: The First Successful CNN
890AlexNet: The Deep Learning Revolution
891AlexNet's Key Innovations
892VGGNet: Depth Through Simplicity
893VGG Block Pattern and Design Principles
894GoogLeNet and the Inception Module
895Inception Module: Multi-Path Architecture
8961×1 Convolutions for Dimensionality Reduction
897Global Average Pooling vs Fully Connected
898Auxiliary Classifiers for Training Deep Networks
899Comparing Early Architectures: Trade-offs
900Architectural Evolution: From AlexNet to ResNet
901The Degradation Problem in Deep Networks
902Skip Connections: The Core Idea
903Residual Learning Formulation
904The Residual Block Architecture
905ResNet-34 and ResNet-50 Architecture
906Bottleneck Residual Blocks
907Gradient Flow Through Skip Connections
908Identity vs Projection Shortcuts
909Pre-Activation Residual Blocks
910ResNet Family: 18, 34, 50, 101, 152
911Wide Residual Networks (WRN)
912ResNeXt: Aggregated Residual Transformations
913Residual Networks in Practice
914Why Residual Networks Revolutionized Deep Learning
915Network Efficiency Metrics and Design Principles
916Depthwise Separable Convolutions
917MobileNetV1: Efficient Architecture for Mobile
918MobileNetV2: Inverted Residuals and Linear Bottlenecks
919MobileNetV3: Neural Architecture Search and Optimizations
920EfficientNet: Compound Scaling
921EfficientNet Architecture and MBConv Blocks
922Neural Architecture Search for CNNs
923ShuffleNet: Channel Shuffle Operations
924SqueezeNet: Fire Modules and Compression
925GhostNet: Cheap Operations for Redundant Features
926ConvNeXt: Modernizing CNNs with Transformer Principles
927RegNet: Design Space Analysis
928Hardware-Aware Architecture Design
929Dynamic Networks and Early Exit
930Comparing Efficiency vs Accuracy Trade-offs

← Machine Learning and Deep Learning

Lesson 903 of 3,538·21. CNN ArchitecturesPro lesson

Residual Learning Formulation

Learning F(x) = H(x) - x instead of H(x) directly, and why this is easier to optimize.

This lesson is for subscribers

You've completed the free preview. Subscribe to unlock every lesson in every course.

See pricing Back to course