Feature-Based NER with CRFs
What you'll learn: How Conditional Random Fields leverage hand-crafted features and sequence patterns to identify named entities in text.
The Core Idea
Before neural networks dominated NER, Conditional Random Fields (CRFs) were the gold standard. Unlike simple token-by-token classifiers, CRFs understand that NER tags aren't independent—they follow patterns. For example, if you see "B-PER" (beginning of a person name), the next tag is more likely to be "I-PER" (inside person) than "B-LOC" (beginning of location).
How CRFs Work for NER
CRFs combine two types of information:
- Emission scores: How likely is this token to have this tag, based on hand-crafted features?
- Transition scores: How likely is this tag sequence based on learned patterns?
Hand-Crafted Features
Unlike BERT which learns features automatically, CRF-based NER requires you to design features manually:
- Word properties: Is it capitalized? All uppercase? Contains digits?
- Context windows: What are the words before and after?
- Linguistic features: Part-of-speech tags, prefixes, suffixes
- Gazetteers: Does it appear in a list of known names or places?
Sequence Modeling
CRFs learn transition probabilities between tags. They know that "B-PER I-PER O B-LOC" is more probable than "B-PER B-PER B-PER I-LOC" because tags follow logical patterns (you can't start a location entity immediately after starting another entity without an "O" in between).
The Trade-off
CRFs require feature engineering expertise but are interpretable and work well with limited data. They were state-of-the-art until neural approaches like BiLSTM-CRFs and transformer fine-tuning surpassed them.
Key Takeaway: CRFs use hand-crafted features and learned transition probabilities to model NER as a structured sequence labeling problem, capturing dependencies between consecutive tags.