Course contentsShow
← Data Science
Lesson 1801 of 2,14542. Working with Big DataFree lesson

Column Selection and Usecols

Load only necessary columns with usecols parameter to minimize memory usage and improve read performance.

Column Selection and Usecols

What You'll Learn

You'll learn how to load only the columns you actually need from large CSV files using the usecols parameter, dramatically reducing memory usage and speeding up data loading.

Why Load Only What You Need

When working with big datasets, reading the entire file into memory can be wasteful—especially when you only need a handful of columns for your analysis. Think of it like ordering a pizza: if you only want pepperoni and mushrooms, why pay for (and carry home) all 15 toppings?

The usecols parameter in pd.read_csv() lets you specify exactly which columns to load, ignoring the rest entirely. This happens during the read operation, so unwanted columns never enter memory at all.

How to Use Usecols

You can specify columns in three ways:

By column name (list of strings):

df = pd.read_csv('large_file.csv', usecols=['customer_id', 'purchase_amount', 'date'])

By column index (list of integers):

df = pd.read_csv('large_file.csv', usecols=[0, 3, 7])  # Load 1st, 4th, and 8th columns

By callable function:

df = pd.read_csv('large_file.csv', usecols=lambda col: col.startswith('sales_'))

Combining with Chunked Reading

You can combine usecols with chunked reading for even greater efficiency:

chunk_iterator = pd.read_csv('huge_file.csv', 
                              usecols=['id', 'value'], 
                              chunksize=50000)

This reads only two columns in manageable chunks—a powerful combination when memory is tight.

Key Takeaway: Use the usecols parameter in read_csv() to load only necessary columns, reducing memory footprint and improving I/O performance before data even enters memory.