> ## Documentation Index
> Fetch the complete documentation index at: https://freelance-4888ed08.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Experiments & Lab

> From Jupyter Notebook prototypes to Production Pipelines

Data Science

> [!NOTE]
>
> **What this does:** Provides a laboratory environment for rapid prototyping and exploratory data analysis — code developed here graduates into the production pipeline, with full experiment tracking via MLflow.

The "Lab" environment consists of Jupyter Notebooks used for Exploratory Data Analysis (EDA) and rapid prototyping. Code developed here is the precursor to the robust `src/` pipelines.

## Notebook Inventory

Located in `app-ml/notebooks/`:

| Notebook | Purpose | Key Outputs |
| :- | :- | :- |
| `EDA.ipynb` | Initial data investigation | Correlation matrices, Seasonality plots, Missing value analysis |
| `Modeling.ipynb` | Algorithms prototyping | Baseline Random Forest performance, CatBoost feature importance |

## Workflow: Lab to Production

We follow a strict promotion process to ensure experimental code is safely productionized.

```mermaid theme={null}
graph TD
    %% Styling
    classDef experimental fill:#E0E7FF,stroke:#4338ca,color:#1e1b4b,stroke-width:2px,stroke-dasharray: 5 5;
    classDef prod fill:#E0E7FF,stroke:#4338ca,color:#1e1b4b,stroke-width:2px;
    
    subgraph "Experimentation"
        direction TB
        NB[Jupyter Lab<br/><i>Prototyping</i>]:::experimental
    end
    
    subgraph "Productionization"
        direction TB
        Node[Kedro Nodes<br/><i>Modular Logic</i>]:::prod
        Pipe[Kedro Pipeline<br/><i>Assembly</i>]:::prod
        DAG[Airflow DAG<br/><i>Orchestration</i>]:::prod
    end
    
    NB -->|Refactor| Node
    Node -->|Assemble| Pipe
    Pipe -->|Orchestrate| DAG
    
    %% Link Styling
    linkStyle default stroke:#6366f1,stroke-width:2px;
```

### 1. Exploratory Data Analysis (`EDA.ipynb`)

Data Scientists use this notebook to understand the dataset distribution. Key insights derived here informed the pipeline design:

* **Seasonality**: Confirmed strong daily and weekly cycles, leading to the creation of lag features `[1, 2, 22, 23]`.
* **Correlations**: Identified `temp` and `atemp` as highly collinear, leading to the decision to use only `temp`.

### 2. Modeling Prototypes (`Modeling.ipynb`)

Used to compare algorithms before settling on CatBoost.

* Tested: Linear Regression, Random Forest, XGBoost, CatBoost.
* **Result**: CatBoost showed superior handling of categorical variables (`season`, `weathersit`) without extensive one-hot encoding.

## Running Notebooks Locally

To run the notebooks within the correct environment:

**1. Activate Environment**

```bash theme={null}
micromamba activate demand-forecasting
```

**2. Launch Jupyter**

```bash theme={null}
cd app-ml/notebooks
jupyter lab
```

**3. Kernel Selection**

Ensure the kernel is set to **demand-forecasting** to access project dependencies like `kedro`, `optuna`, and `pandas`.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.