Feature Engineering

Temporal Lag Creation and Target Generation

Data Science

What this does: Transforms raw ride data into meaningful patterns — such as demand trends from past hours and time-of-day effects — that the model can learn from, directly improving forecast accuracy.

Feature engineering focuses on capturing temporal dependencies in the time-series data using lag features, temporal extractions, and feature interactions.

​

Pipeline Overview

The feature_engineering pipeline transforms clean data into a supervised learning dataset through 4 chained Kedro nodes.

​

Key Transformations

​

1. Target Variable

We forecast the next hour’s demand.

• Feature: target
• Logic: df['cnt'].shift(-1)
• Impact: The model predicts $t+1$ using data available at $t$.

​

2. Lag Features

We create “lags” to provide historical context for the model.

Show Why do we use Lags?

Bike demand is highly auto-correlated.

• 1 Hour Ago: Immediate trend.
• 22–23 Hours Ago: Daily seasonality (same time yesterday).

​

Visualizing Temporal Dependencies

How the model “looks back” in time to predict the future:

​

Lag Configuration

All lag values are defined in conf/base/parameters.yml:

Feature	Lags Created	Rationale
cnt (demand)	1, 2, 22, 23	Recent demand & Daily pattern
hour	1, 2, 3	Time of day context
temp	1, 2, 3	Temperature trends
weather_condition	1, 2, 3	Weather condition shifts
humidity	1, 2, 3	Humidity trends
day_of_week	1, 2, 3	Weekly cycle context

​

Transformation Example

Visualizing how Lag 1 shifts data to provide historical context for the model.

Time (t)	Demand (cnt)	Lag_1 (t-1)	Target (t+1)
08:00	100	NaN	150
09:00	150	100	120
10:00	120	150	200
11:00	200	120	…

​

3. Data Splitting

The dataset is split chronologically (not randomly) to preserve temporal order:

feature_engineering/nodes.py

train_size = int(len(model_input) * params["train_fraction"])  # 0.8
X_train = model_input.iloc[:train_size].drop(columns=["target"])
X_test = model_input.iloc[train_size:].drop(columns=["target"])

​

4. Data Cleaning

NaN Handling: We first apply Backward Fill (bfill) to preserve initial rows, then drop any remaining NaN values to ensure clean input for the model.

---

​

Related Pages

Data Catalog

All engineered datasets registered in the catalog with schemas and paths.

Training & Optimization

How engineered features are used to train CatBoost with Optuna optimization.