Course contentsShow
← AI Engineering
Lesson 1557 of 1,88638. Bias, Fairness, and AlignmentFree lesson

Sources of Bias: Model Architecture and Objectives

Learn how model design choices, optimization objectives, and inductive biases introduce systematic skews.

Sources of Bias: Model Architecture and Objectives

You'll learn how the fundamental design decisions you make when building AI models—architecture choices, loss functions, and optimization goals—can bake in systematic biases before training even begins.

The Architecture Isn't Neutral

When you design or select a model architecture, you're making assumptions about what patterns matter. A convolutional neural network assumes spatial locality is important—great for images, but this "inductive bias" might miss global context. Transformer architectures prioritize attention patterns that can inadvertently amplify whatever correlations appear most frequently in training data, even harmful ones.

Think of it like building a house: if you design it with only right-angle walls, you've constrained the shapes of rooms you can create. Similarly, architectural choices like layer depth, attention mechanisms, or tokenization schemes constrain what your model can and cannot learn.

Objectives Shape Behavior

Your loss function is your model's north star—it optimizes ruthlessly for whatever you measure. If you optimize purely for accuracy on a dataset where one demographic is overrepresented, the model learns to excel at predictions for that group while underperforming on others.

Consider a language model trained with next-token prediction: it learns to reproduce statistical patterns, including stereotypes. If technical documentation in your corpus consistently uses "he" for engineers, that objective function rewards perpetuating that pattern.

The Optimization Process

Even how you optimize matters. Gradient descent can get stuck in local optima that favor majority patterns. Regularization techniques might inadvertently suppress features important to minority groups if those features appear "noisy" statistically.

Key Takeaway: Bias enters your AI system through the very structure you choose, the goals you set, and how you optimize—these design decisions encode assumptions about what matters before a single data point is processed.