> ## Documentation Index
> Fetch the complete documentation index at: https://freelance-4888ed08.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Inference & Serving

> FastAPI prediction API with Argon2 auth, rate limiting, A/B testing, and Prometheus observability

Engineering

> [!NOTE]
>
> **What this does:** Generates real-time demand predictions via a secure, rate-limited API — enabling business applications to request forecasts on demand with enterprise-grade authentication and monitoring.

The system provides a robust **Real-Time Inference API** powered by **FastAPI** with production-grade security, observability, and traffic management.

## Architecture

The serving layer (`src/demand_forecasting/serving`) exposes a REST API for demand forecasting.

```mermaid theme={null}
graph TD
    %% Styling
    classDef client fill:#FEF3C7,stroke:#b45309,color:#78350f,stroke-width:2px;
    classDef security fill:#FEE2E2,stroke:#b91c1c,color:#7f1d1d,stroke-width:2px;
    classDef server fill:#E0E7FF,stroke:#4338ca,color:#1e1b4b,stroke-width:2px;
    classDef model fill:#DCFCE7,stroke:#15803d,color:#14532d,stroke-width:2px;
    classDef monitoring fill:#F3E8FF,stroke:#7c3aed,color:#4c1d95,stroke-width:2px;

    Client[Client App]:::client -->|POST /predict| MW[Security Middleware<br/><i>Headers + Rate Limit + CORS</i>]:::security
    MW --> Auth{API Key Auth<br/><i>Argon2id Verification</i>}:::security
    Auth -->|Valid| Router[Model Server]:::server
    Auth -->|Invalid| Reject[403 Forbidden]:::security
    Router --> ModelA[Production Model]:::model
    Router -.->|If A/B Enabled| ModelB[Challenger Model]:::model
    Router --> Metrics[Prometheus Metrics]:::monitoring
    Router --> Drift[Evidently Drift Check]:::monitoring

    %% Link Styling
    linkStyle default stroke:#6366f1,stroke-width:2px;
```

## Security Middleware Stack

The API applies four layers of middleware, in order of priority:

**1. SecurityHeadersMiddleware**

Adds OWASP-recommended headers: `X-Frame-Options: DENY`, `X-Content-Type-Options: nosniff`, `Content-Security-Policy`, `Referrer-Policy`, `Permissions-Policy`. In production, adds `Strict-Transport-Security` (HSTS).

**2. RateLimitMiddleware**

IP-based rate limiting (default: 60 requests/minute). Returns `429 Too Many Requests` with `Retry-After` and `X-RateLimit-*` headers. Configurable via `RATE_LIMIT_PER_MINUTE` environment variable.

**3. TrustedHostMiddleware**

Prevents Host header attacks. Allowed hosts are environment-specific — production hosts must be explicitly configured via `ALLOWED_HOSTS`.

**4. CORS**

Environment-aware origin allowlisting. Only `GET`, `POST`, `OPTIONS` methods allowed. No wildcard origins in any environment.

## Authentication

API key authentication using **Argon2id** hashing with constant-time verification:

* **Key Storage**: SHA-256 → Argon2id hashed in memory (loaded from `VALID_API_KEYS` env var)
* **Brute Force Protection**: Max 10 failed attempts per hour per IP
* **Rehashing**: Automatic rehash if Argon2 parameters are updated

> [!WARNING]
>
> All `/predict` and `/predict/ab` endpoints require the `X-API-Key` header. Unauthenticated requests receive a `401 Unauthorized` response.

## Inference Strategies

**A/B Testing & Traffic Splitting**

Simultaneously serve two models to compare real-world performance.

* **Weighted Routing**: Configurable split (e.g., `AB_SPLIT=0.7`).
* **Per-Request Routing**: Each request is independently routed based on the split ratio.
* **Enable**: Set `ENABLE_AB_TESTING=true` and `MODEL_B_URI` environment variables.

**Shadow Mode (Challenger)**

New models receive live traffic for validation without returning results to the user.

* **Safe Validation**: Calculate metrics on live distributions before promotion.

**Prediction Confidence**

API returns a **Confidence Score** (0.50 to 0.95).

* **Default**: 0.75 for standard regression predictions.
* **CatBoost**: Uses leaf index variance as uncertainty proxy when available.

## API Reference

> [tab] POST /predict
>
> Generates a forecast for a given set of conditions. Requires authentication.
>
> ```python Request theme={null}
> import requests
> response = requests.post(
>     "http://localhost:8000/predict",
>     headers={"X-API-Key": "your-api-key"},
>     json={
>         "features": {
>             "hr": 14, "temp": 0.5, "hum": 0.6,
>             "weathersit": 1, "weekday": 2
>         }
>     }
> )
> ```
>
> ```json Response theme={null}
> {
>   "prediction": 145.2,
>   "confidence": 0.75,
>   "model_version": "production",
>   "model_id": "production",
>   "timestamp": "2025-01-15T14:30:00"
> }
> ```

> [tab] GET /health
>
> Verifies service health. Returns `200 OK` with model count and timestamp. **No auth required.**
>
> ```json Response theme={null}
> {
>   "status": "healthy",
>   "models_loaded": 1,
>   "timestamp": "2025-01-15T14:30:00"
> }
> ```

> [tab] GET /model/info
>
> Returns current model metadata including type, features, and version. **No auth required.**
>
> ```json Response theme={null}
> {
>   "model_version": "production",
>   "model_type": "CatBoost",
>   "features": ["temp", "hum", "windspeed", "hr", "weekday"],
>   "last_updated": "2025-01-15T14:30:00",
>   "model_id": "production"
> }
> ```

> [tab] GET /metrics/drift
>
> Returns Evidently-powered drift metrics with 5-minute caching. **No auth required.**
>
> ```json Response theme={null}
> {
>   "status": "ok",
>   "cached": false,
>   "last_check": "2025-01-15T14:30:00",
>   "metrics": {
>     "evidently_dataset_drift": 0.0,
>     "evidently_drift_share": 0.12,
>     "evidently_columns_total": 10,
>     "evidently_columns_drifted": 1
>   }
> }
> ```

> [tab] GET /metrics
>
> Exposes Prometheus (Standard monitoring format for real-time time-series metrics.) metrics:
>
> * `prediction_requests_total` (labels: `model_id`, `status`)
> * `prediction_duration_seconds` (histogram)
> * `model_accuracy` (gauge per model\_id)
> * `evidently_dataset_drift` / `evidently_drift_share` (gauges)

***

Refer to the [API Reference](https://freelance-4888ed08.mintlify.site/api-reference/openapi.md) for complete schema details.

## Related Pages

> [card] [Generate Forecast API](https://freelance-4888ed08.mintlify.site/api-reference/predict.md)
>
> Full API reference with request/response schemas and interactive playground.

> [card] [Infrastructure](https://freelance-4888ed08.mintlify.site/deployment/infrastructure.md)
>
> Docker Compose architecture and service configuration behind the API.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.