Course contentsShow
← Machine Learning and Deep Learning
Lesson 3246 of 3,53871. Interpretability: Neural Network MethodsFree lesson

Choosing a Baseline

Selecting meaningful reference inputs: zeros, blurred images, neutral text, and the impact on attributions.

Choosing a Baseline

What you'll learn: How the choice of baseline (reference input) fundamentally shapes attribution results in neural network interpretability methods.

Why Baselines Matter

When using path-based attribution methods like Integrated Gradients, you measure the difference between your actual input and a baseline (or reference) input. The baseline represents "absence of signal" or a neutral starting point. Think of it like asking: "What changed between this baseline and my input, and how did those changes affect the prediction?"

Your baseline choice directly impacts what the attributions tell you. If you choose poorly, your explanations may highlight irrelevant features or miss important ones entirely.

Common Baseline Choices

All-Zeros Baseline

Simple and common for many tasks. Works well when zero genuinely represents "nothing" (e.g., pixel intensity in images). However, for a network trained on natural images, an all-black image might be out-of-distribution and produce misleading gradients.

Blurred Images

For computer vision, a blurred version of the input preserves overall color and texture distributions while removing fine details. This keeps the baseline in-distribution, often yielding more meaningful attributions about which sharp features matter.

Neutral Text

For NLP, you might use a padding token, an empty string representation, or even the mean embedding vector. Each choice answers a slightly different question about what "matters" in your input.

Domain-Specific Neutrals

In some domains, you know what "neutral" means: a gray pixel in certain medical imaging, silence in audio, or a specific baseline measurement in tabular data.

The Impact

A good baseline should be semantically neutral and in-distribution. Poor choices lead to attributions that reflect the path from an unrealistic starting point rather than meaningful feature contributions.

Key Takeaway: The baseline isn't a technical detail—it defines what your attributions measure. Choose baselines that represent meaningful absence of signal in your domain, and consider testing multiple baselines to ensure robust interpretation.