Course contentsShow
← Python
Lesson 948 of 1,06015. Data Science FundamentalsFree lesson

Applying Conditional Transformations Across Columns

Use vectorized conditional logic to create new features based on complex rules involving multiple existing columns in your dataframe.

Applying Conditional Transformations Across Columns

In this lesson, you will learn how to create new features by applying complex, rule-based logic that evaluates multiple columns simultaneously using vectorized operations.

Moving Beyond Simple Column Math

Up until now, you have mostly performed operations that act on entire columns at once, such as df['A'] + df['B']. However, real-world data science often requires "conditional logic"—where the value in a new column depends on a combination of different existing columns. While you could write a loop to check each row, that is inefficient. Instead, we use vectorized conditional logic to evaluate all rows at once.

The most powerful tool for this is numpy.where(). It functions like an "if-else" statement that operates across an entire array simultaneously. It takes three arguments: a condition, the value if true, and the value if false.

Implementing Complex Rules

Imagine you have a DataFrame representing sales data with Revenue and Cost columns. You want to categorize each row as "Profitable" if revenue exceeds cost, or "Loss" otherwise. You can do this without iterating:

import pandas as pd
import numpy as np

# Sample data
df = pd.DataFrame({
    'Revenue': [100, 200, 50],
    'Cost': [80, 250, 40]
})

# Create a new feature based on the relationship between columns
df['Performance'] = np.where(df['Revenue'] > df['Cost'], 'Profitable', 'Loss')

For more complex rules—such as needing to check three or more conditions—you can nest np.where() statements or use np.select(). The np.select() function is particularly elegant; it takes a list of conditions and a corresponding list of results:

conditions = [
    (df['Revenue'] > df['Cost']),
    (df['Revenue'] == df['Cost']),
    (df['Revenue'] < df['Cost'])
]

choices = ['Profitable', 'Break-even', 'Loss']

df['Status'] = np.select(conditions, choices, default='Unknown')

By using these vectorized functions, you keep your code clean, readable, and highly performant, ensuring that your feature engineering scales even as your datasets grow into millions of rows.

Key Takeaway: Use np.where or np.select to create new features based on multi-column logic, replacing slow row-by-row iteration with fast, vectorized operations.