Course contentsShow
← Machine Learning and Deep Learning
Lesson 943 of 3,53822. Computer Vision TasksFree lesson

Model Selection for Transfer Learning

Choosing pretrained architectures based on computational constraints, accuracy needs, and domain similarity.

Model Selection for Transfer Learning

What you'll learn: How to pick the right pretrained CNN architecture based on your computational budget, accuracy goals, and how similar your dataset is to the model's original training domain.

The Right Tool for the Job

Imagine choosing between a bicycle, a car, or a truck for a delivery. Each has different costs, speeds, and cargo capacities—and the right choice depends on your specific needs. Similarly, selecting a pretrained model for transfer learning requires balancing three key factors.

Three Decision Criteria

1. Computational Constraints

Are you deploying on a smartphone, embedded device, or cloud server? MobileNets and EfficientNet-B0 shine on resource-limited devices, while ResNet-50 or EfficientNet-B4 work well when you have more compute. Check parameter count and inference time—not just accuracy leaderboards.

2. Accuracy Requirements

A medical diagnosis system demands higher accuracy than a photo filter app. For tight accuracy needs, consider deeper models like ResNet-101, EfficientNet-B7, or ConvNeXt. For moderate requirements, ResNet-34 or MobileNetV3 may suffice.

3. Domain Similarity

This is crucial: if your task resembles ImageNet (general objects, similar visual patterns), almost any pretrained model transfers well. But if your domain differs significantly—like medical X-rays or satellite imagery—you might benefit from models pretrained on more similar data, or expect to fine-tune more layers. Greater domain shift means you'll need more training data and potentially a deeper fine-tuning strategy.

Making the Trade-Off

Start with a mid-range model like ResNet-50 as your baseline. If it's too slow, try MobileNetV2. If accuracy is insufficient and you have compute headroom, scale up to EfficientNet or larger ResNets. Always validate that the feature hierarchy learned on the source domain (like ImageNet) provides useful patterns for your target task.

Key Takeaway: Choose your pretrained architecture by matching computational budget to deployment constraints, accuracy needs to model capacity, and verifying domain similarity to ensure transferable features—there's no universal best model, only best-for-your-task models.