Reshaping Data: Pivot and Melt
What You'll Learn
You'll learn how to transform data between wide (pivot) and long (melt) formats to prepare datasets for different machine learning tasks and analyses.
Why Reshape Data?
Data doesn't always arrive in the format you need. Sometimes you have wide data (many columns, fewer rows) when you need long data (fewer columns, many rows), or vice versa. Think of it like rearranging a bookshelf: the books don't change, just their organization.
Wide vs Long Format
Wide format: Each subject has one row with multiple measurement columns.
# Wide: each student has one row
# Name | Math | Science | History
# Alice | 90 | 85 | 88
Long format: Each measurement gets its own row.
# Long: each grade gets its own row
# Name | Subject | Score
# Alice | Math | 90
# Alice | Science | 85
# Alice | History | 88
Pivot: Long to Wide
Use pivot_table() to spread data out horizontally:
import pandas as pd
# Long format data
df_long = pd.DataFrame({
'Name': ['Alice', 'Alice', 'Bob', 'Bob'],
'Subject': ['Math', 'Science', 'Math', 'Science'],
'Score': [90, 85, 78, 82]
})
# Pivot to wide format
df_wide = df_long.pivot_table(index='Name', columns='Subject', values='Score')
Melt: Wide to Long
Use melt() to collapse columns into rows:
# Wide format data
df_wide = pd.DataFrame({
'Name': ['Alice', 'Bob'],
'Math': [90, 78],
'Science': [85, 82]
})
# Melt to long format
df_long = df_wide.melt(id_vars='Name', var_name='Subject', value_name='Score')
When to Use Each
- Pivot: Create feature matrices for ML models, make data human-readable
- Melt: Prepare data for grouping operations, visualizations, or certain model inputs
Key Takeaway: Pivoting widens data by turning unique values into columns, while melting lengthens data by converting columns into rows—essential transformations for preparing ML datasets.