Initial Data Profiling: Shape, Types, and Memory
You'll learn how to perform the essential first inspection of any dataset by checking its structure, data types, and resource footprint.
The Foundation of Every EDA
Before diving into distributions or relationships, you need to understand what you're working with. Think of it like meeting someone new—you first notice basic facts (height, age, appearance) before learning their personality. Initial data profiling answers: How big is this dataset? What kind of information does it contain? Will it fit comfortably in memory?
The Three Core Checks
1. Shape (Dimensions)
Reveals the number of rows and columns. This tells you the scale of your analysis and whether you're dealing with a few records or millions. Use .shape in pandas to get (rows, columns).
2. Data Types
Shows what kind of data each column holds (integers, floats, strings, dates). Misidentified types are common—numbers stored as text, dates as strings. Use .dtypes to inspect column types and .info() for a comprehensive view.
3. Memory Usage
Indicates how much RAM the dataset consumes. Critical for large datasets that might overwhelm your system. Use .memory_usage() or the memory_usage parameter in .info() to see per-column and total consumption.
Why This Matters
This quick profiling prevents surprises downstream. You might discover your "small" dataset is actually 10GB in memory, or that numeric columns were read as text (blocking calculations). It's your reality check before investing hours in analysis on malformed or unworkable data.
Key Takeaway: Initial profiling with shape, types, and memory gives you the structural blueprint of your dataset—essential context before any meaningful exploration can begin.