> ## Documentation Index
> Fetch the complete documentation index at: https://freelance-4888ed08.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Feature Engineering

> Temporal Lag Creation and Target Generation

Data Science

> [!NOTE]
>
> **What this does:** Transforms raw ride data into meaningful patterns — such as demand trends from past hours and time-of-day effects — that the model can learn from, directly improving forecast accuracy.

Feature engineering focuses on capturing temporal dependencies in the time-series data using lag features, temporal extractions, and feature interactions.

## Pipeline Overview

The `feature_engineering` pipeline transforms clean data into a supervised learning dataset through 4 chained Kedro (Python framework for reproducible, maintainable data science pipelines.) nodes.

```mermaid theme={null}
graph TD
    %% Styling
    classDef process fill:#E0E7FF,stroke:#4338ca,color:#1e1b4b,stroke-width:2px;
    classDef storage fill:#DCFCE7,stroke:#15803d,color:#14532d,stroke-width:2px;
    classDef validate fill:#FEF3C7,stroke:#b45309,color:#78350f,stroke-width:2px;

    Clean[(preprocessed_bike_data)]:::storage --> Temporal[extract_temporal_features]:::process
    Temporal --> Interact[create_feature_interactions]:::process
    Interact --> Validate[validate_features]:::validate
    Validate --> Split[split_features]:::process
    Split --> XTrain[(X_train)]:::storage
    Split --> XTest[(X_test)]:::storage
    Split --> YTrain[(y_train)]:::storage
    Split --> YTest[(y_test)]:::storage

    %% Link Styling
    linkStyle default stroke:#6366f1,stroke-width:2px;
```

## Key Transformations

### 1. Target Variable

We forecast the **next hour's** demand.

* **Feature**: `target`
* **Logic**: `df['cnt'].shift(-1)`
* **Impact**: The model predicts $t+1$ using data available at $t$.

### 2. Lag Features

We create "lags" to provide historical context for the model.

**Why do we use Lags?**

Bike demand is highly auto-correlated.

* **1 Hour Ago**: Immediate trend.
* **22–23 Hours Ago**: Daily seasonality (same time yesterday).

### Visualizing Temporal Dependencies

How the model "looks back" in time to predict the future:

```mermaid theme={null}
graph LR
    subgraph Past [Historical Context]
        direction TB
        L1["Lag 1 (t-1)<br/><i>Previous Hour</i>"]:::feature
        L2["Lag 2 (t-2)<br/><i>Two Hours Ago</i>"]:::feature
        L22["Lag 22 (t-22)<br/><i>Near-Yesterday</i>"]:::feature
        L23["Lag 23 (t-23)<br/><i>Yesterday</i>"]:::feature
    end

    subgraph Present [Current State]
        direction TB
        Now["Model Input<br/>Vector at t"]:::current
    end

    subgraph Future [Prediction Target]
        direction TB
        Target["Target (t+1)<br/><i>Next Hour</i>"]:::target
    end

    L1 --> Now
    L2 --> Now
    L22 --> Now
    L23 --> Now
    Now -.->|Training Signal| Target

    %% Styling
    classDef feature fill:#E0E7FF,stroke:#4338ca,color:#1e1b4b;
    classDef current fill:#F3F4F6,stroke:#4B5563,color:#111827,stroke-dasharray: 5 5;
    classDef target fill:#DCFCE7,stroke:#15803d,color:#14532d;
```

### Lag Configuration

All lag values are defined in `conf/base/parameters.yml`:

| Feature | Lags Created | Rationale |
| :- | :- | :- |
| `cnt` (demand) | 1, 2, 22, 23 | Recent demand & Daily pattern |
| `hour` | 1, 2, 3 | Time of day context |
| `temp` | 1, 2, 3 | Temperature trends |
| `weather_condition` | 1, 2, 3 | Weather condition shifts |
| `humidity` | 1, 2, 3 | Humidity trends |
| `day_of_week` | 1, 2, 3 | Weekly cycle context |

### Transformation Example

Visualizing how **Lag 1** shifts data to provide historical context for the model.

| Time (t) | Demand (cnt) | Lag\_1 (t-1) | Target (t+1) |
| :- | :- | :- | :- |
| **08:00** | 100 | NaN | 150 |
| **09:00** | 150 | **100** | 120 |
| **10:00** | 120 | **150** | 200 |
| **11:00** | 200 | **120** | ... |

### 3. Data Splitting

The dataset is split **chronologically** (not randomly) to preserve temporal order:

```python feature_engineering/nodes.py theme={null}
train_size = int(len(model_input) * params["train_fraction"])  # 0.8
X_train = model_input.iloc[:train_size].drop(columns=["target"])
X_test = model_input.iloc[train_size:].drop(columns=["target"])
```

### 4. Data Cleaning

> [!NOTE]
>
> **NaN Handling**: We first apply **Backward Fill** (`bfill`) to preserve initial rows, then **drop** any remaining `NaN` values to ensure clean input for the model.

***

## Related Pages

> [card] [Data Catalog](https://freelance-4888ed08.mintlify.site/configuration/data-catalog.md)
>
> All engineered datasets registered in the catalog with schemas and paths.

> [card] [Training & Optimization](https://freelance-4888ed08.mintlify.site/ml-solution/training-optimization.md)
>
> How engineered features are used to train CatBoost with Optuna optimization.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.