Selecting and Filtering Data
What You'll Learn
You'll learn how to extract specific rows and columns from Pandas DataFrames using loc, iloc, and boolean indexing—essential skills for preparing features in machine learning.
Why This Matters for ML
In machine learning, you rarely use all the data in its raw form. You need to select specific features (columns), filter examples that meet certain criteria (rows), or extract subsets for training and validation. Think of it like preparing ingredients: you wouldn't throw the entire pantry into a recipe—you select exactly what you need.
Three Core Selection Methods
loc: Label-Based Selection
loc selects data by labels (row and column names):
# Select specific columns by name
features = df.loc[:, ['age', 'income', 'education']]
# Select rows where index label is 0-10 and specific columns
subset = df.loc[0:10, ['age', 'income']]
iloc: Position-Based Selection
iloc selects by integer positions (like array indexing):
# Select first 100 rows, first 5 columns
training_subset = df.iloc[0:100, 0:5]
# Select every 10th row
sampled_data = df.iloc[::10, :]
Boolean Indexing: Condition-Based Filtering
Filter rows based on conditions (remember boolean masking from NumPy?):
# Select only high-income examples
high_income = df[df['income'] > 50000]
# Combine multiple conditions
young_high_earners = df[(df['age'] < 30) & (df['income'] > 50000)]
# Select specific columns from filtered rows
features = df[df['income'] > 50000][['age', 'education']]
Practical ML Use Case
You might filter to create balanced datasets, remove outliers, or select only relevant features before training—all common feature engineering tasks.
Key Takeaway: Use loc for label-based selection, iloc for position-based selection, and boolean indexing to filter rows by conditions—these are your primary tools for extracting ML-ready data subsets.