Course contentsShow
← Machine Learning and Deep Learning
Lesson 1157 of 3,53826. Pretrained Language Models: BERT FamilyFree lesson

Loading Pretrained BERT with Hugging Face

Practical steps to load and inspect a pretrained BERT model using the transformers library.

Loading Pretrained BERT with Hugging Face

You'll learn how to load a ready-to-use pretrained BERT model in just a few lines of code using the transformers library.

The Core Idea

Think of Hugging Face as a library of pretrained models. Instead of training BERT from scratch (which requires massive compute and data), you can simply download a model that's already been pretrained on BookCorpus and Wikipedia. The transformers library provides a standardized interface to access these models effortlessly.

Loading BERT: The Essential Steps

To load a pretrained BERT model, you need two components:

  1. The tokenizer — converts text into tokens using WordPiece tokenization
  2. The model — the actual neural network with pretrained weights

Here's how to load BERT Base (the smaller configuration):

from transformers import BertTokenizer, BertModel

# Load pretrained tokenizer
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')

# Load pretrained model
model = BertModel.from_pretrained('bert-base-uncased')

# Inspect the model
print(f"Number of parameters: {model.num_parameters():,}")
print(f"Model configuration: {model.config}")

The 'bert-base-uncased' identifier tells Hugging Face which specific BERT variant to download. Other options include 'bert-large-uncased', 'bert-base-cased', etc.

What You Get

When you load the model, you receive all 12 encoder layers (for Base), pretrained attention weights, and the embedding layers (token, segment, and position embeddings) — everything learned during pretraining with Masked Language Modeling and Next Sentence Prediction.

You can immediately use this model for feature extraction or fine-tune it on your specific downstream task.

Key Takeaway: Loading pretrained BERT requires just two lines: BertTokenizer.from_pretrained() for the tokenizer and BertModel.from_pretrained() for the model weights, giving you instant access to powerful contextual representations without training from scratch.