Character Encodings: ASCII, UTF-8, and Latin-1
You'll learn how different character encoding systems interpret bytes as text and when each encoding is used.
When you read a text file, your computer doesn't see letters—it sees numbers (bytes). Character encodings are the rulebooks that translate those numbers into readable characters. Choosing the wrong encoding is like using a Spanish dictionary to read French: you'll get gibberish or errors.
Think of encodings as different alphabets for the digital world:
ASCII: The Original 128 Characters
ASCII (American Standard Code for Information Interchange) uses 7 bits to represent 128 characters: English letters, digits, punctuation, and control characters. It's simple and universal but limited—no accented letters, no emoji, no non-Latin scripts.
Latin-1 (ISO-8859-1): Western European Extension
Latin-1 extends ASCII to 256 characters using 8 bits, adding accented characters like é, ñ, and ü. Perfect for Western European languages, but still can't handle Cyrillic, Arabic, or Asian scripts.
UTF-8: The Universal Standard
UTF-8 can represent over a million characters using variable-length encoding (1-4 bytes per character). It's backward-compatible with ASCII and handles virtually every language and symbol. It's now the web standard and default for most modern systems.
When This Matters in Data Science
When loading CSV or text files with pandas, you may encounter UnicodeDecodeError if Python guesses the wrong encoding. Specify it explicitly:
df = pd.read_csv('data.csv', encoding='utf-8')
df = pd.read_csv('old_data.csv', encoding='latin-1')
Legacy systems often used Latin-1, while modern data typically uses UTF-8. Text scraped from websites or old databases might mix encodings, causing mojibake (corrupted characters like "café" appearing as "café").
Key Takeaway: Character encodings translate bytes to text; UTF-8 is the modern standard supporting all languages, while ASCII and Latin-1 are limited but still encountered in legacy data—always specify encoding when reading text files to avoid corruption.