Course contentsShow
← Python
Lesson 970 of 1,06015. Data Science FundamentalsFree lesson

Shifting and Lagging Data for Time-Dependent Features

Use shift operations to calculate period-over-period changes and prepare datasets for predictive time-series modeling tasks.

In this lesson, you will learn how to create predictive features by offsetting your time-series data to compare current values against their historical counterparts.

The Logic of Lags

In time-series modeling, the "past" is the most powerful predictor of the "future." When you have a column of sequential data—like daily revenue or stock prices—you cannot simply feed the entire column into a model. Instead, you need to provide the model with historical context for each row.

A lag is a displacement of the data series. By shifting a column down by one period, the value at index t becomes the value that previously existed at t-1. This allows you to perform calculations like "period-over-period" changes (e.g., revenue - revenue.shift(1)) or prepare input features (where the model sees the value from yesterday to predict today).

Implementation in Python

Using the shift() method in pandas is the standard way to create these temporal features. A positive integer inside shift() pushes data forward (creating a lag), while a negative integer pulls data backward (creating a "lead").

import pandas as pd

# Assume 'df' has a DatetimeIndex
# Calculate the one-day lag
df['revenue_lag1'] = df['revenue'].shift(1)

# Calculate the daily change (current - previous)
df['revenue_diff'] = df['revenue'] - df['revenue'].shift(1)

# Calculate the percentage change relative to 7 days ago
df['weekly_pct_change'] = (df['revenue'] - df['revenue'].shift(7)) / df['revenue'].shift(7)

Why This Matters for Modeling

When you shift data, you create NaN values at the beginning of your dataset because there is no historical data available for those early time points. In a production pipeline, this is a critical reminder that you cannot make a prediction for day one if your model relies on the value from day zero.

By creating these shifted columns, you transform a single temporal variable into a set of features that capture momentum, seasonality, and trend. This effectively allows your model to "see" the history of the target variable directly within the row structure, turning a static snapshot into a dynamic time-dependent input.

Key Takeaway: Shifting data aligns past observations with current rows, enabling the calculation of temporal changes and creating the necessary historical context for predictive time-series models.