> ## Documentation Index
> Fetch the complete documentation index at: https://freelance-4888ed08.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Deployment Strategies

> Safe rollout procedures from Staging to Production

OperationsEngineering

> [!NOTE]
>
> **What this does:** Defines how new model versions move safely from staging to production using a 2-week promotion cycle — ensuring zero-downtime updates and easy rollback if issues arise.

Our deployment strategy prioritizes safety and zero-downtime updates. We utilize a **2-week promotion cycle** with distinct "Staging" and "Production" environments detached via Docker Compose configuration.

```mermaid theme={null}
graph LR
    %% Styling
    classDef dev fill:#DBEAFE,stroke:#2563EB,color:#1E3A5F,stroke-width:2px;
    classDef staging fill:#FEF3C7,stroke:#B45309,color:#78350F,stroke-width:2px;
    classDef prod fill:#DCFCE7,stroke:#15803D,color:#14532D,stroke-width:2px;
    classDef gate fill:#F3E8FF,stroke:#7C3AED,color:#2E1065,stroke-width:2px;
    classDef rollback fill:#FEE2E2,stroke:#DC2626,color:#7F1D1D,stroke-width:2px;

    Dev["Dev<br/><i>Feature Branch</i>"]:::dev
    Gate1{"Smoke Tests<br/><i>pytest -m smoke</i>"}:::gate
    Staging["Staging<br/><i>Week 1 Integration</i>"]:::staging
    Gate2{"Perf Check<br/><i>Latency < 200ms</i>"}:::gate
    Prod["Production<br/><i>Week 2 Go-Live</i>"]:::prod
    Rollback["Rollback<br/><i>Revert to v2.x</i>"]:::rollback

    Dev --> Gate1 -->|Pass| Staging --> Gate2 -->|Pass| Prod
    Gate1 -->|Fail| Dev
    Gate2 -->|Fail| Rollback --> Staging

    %% Link Styling
    linkStyle default stroke:#6366f1,stroke-width:2px;
```

## Environment Architecture

| Env | Docker File | Purpose | Data Source |
| :- | :- | :- | :- |
| **Dev** | `docker-compose.yml` | Feature development | Local / MinIO (Dev) |
| **Staging** | `docker-compose.staging.yml` | Integration testing | MinIO (Staging Replica) |
| **Prod** | `docker-compose.prod.yml` | Live traffic | MinIO (Production) |

## Promotion Lifecycle

### Week 1: Staging (Integration)

1. **Deploy Candidate**: Tag the release `v3.0.0-rc1`.
2. **Smoke Tests**: Run critical path tests (`pytest -m smoke`).
3. **Performance Verification**: Ensure inference latency \< 200ms.

```bash theme={null}
# Example: Deploying to Staging
export AIRFLOW_ENV=staging
docker-compose -f docker-compose.staging.yml up -d --build
./scripts/health_check.sh
```

### Week 2: Production (Go Live)

1. **Backup**: Snapshot the current PostgreSQL metadata.
2. **Traffic Shift**: In a Blue/Green setup, route traffic to the new containers.
3. **Monitoring**: Watch `DriftDetector` logs for immediate anomalies.

## Scripts & Automation

We use tailored scripts to automate these risky operations.

**Backup Script**

**`scripts/backup_production.sh`**

Creates a timestamped dump of the Airflow metadata DB and Optuna studies.

```bash theme={null}
pg_dump -h postgres -U airflow > backup_$(date +%F).sql
```

**Health Check**

**`scripts/verify_ml_integration.py`**

A comprehensive integrity check that validates PostgreSQL, MinIO, and MLflow connections before traffic is shifted.

### Rollback Procedure

If a critical failure is detected (e.g., Error Rate > 1%), execute an immediate rollback:

1. **Revert Image**: Point `docker-compose.prod.yml` back to `v2.x.x`.
2. **Restore DB**: If schema migration failed, restore from the pre-deploy backup.
3. **Restart**: `docker-compose up -d --force-recreate`.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.