Explainability & Trust

Opening the ‘Black Box’ of AI decision making with SHAP

Data ScienceBusiness

What this does: Explains why the model made each prediction — not just the number, but which factors (weather, time, seasonality) drove the forecast, building trust and enabling data-driven decisions.

​

Opening the Black Box

Trust is the currency of any AI system. We don’t just provide a forecast number (e.g., “500 units”); we explain why the model believes that number is correct.

Our transparency engine uses SHAP (SHapley Additive exPlanations), a Nobel-prize winning game theory approach, to “X-Ray” every prediction.

​

1. Global Drivers (The “Big Picture”)

First, we ask: “Overall, what levers move the needle the most?”

This helps business planners understand the strategic rules the model has learned. We visualize this with a Beeswarm Plot.

Show How to read the Beeswarm Plot

• Y-Axis (Rank): Features at the top are the most important.
• Color: Red = High value (e.g., High Price), Blue = Low value (e.g., Low Price).
• X-Axis (Impact): How much it creates (+/-) demand.

Example: If “Price” is at the top, and the Red dots are on the left (negative side), it means Higher Prices strongly reduce Demand.

Business Translation:

• Feature Importance → “Strategic Levers”
• SHAP Value → “Impact Score” (in units sold)

​

2. Local X-Ray (The “Detective”)

Next, we drill down to a specific prediction: “Why did we predict 950 units for Store A next Tuesday?”

We use a Waterfall Plot to tell the story of that specific day. It starts at the baseline (average) and adds/subtracts impact based on active conditions.

​

The Decision Path

1. Baseline: The average demand is 500 units.
2. Pass 1 (Promotion): A promo is active. +200 units.
3. Pass 2 (Seasonality): It is December. +300 units.
4. Pass 3 (Price): Price was raised slightly. -50 units.
5. Final Forecast: 950 units.

Local Interpretation Example

from demand_forecasting.explainability.shap_explainer import ModelExplainer

# Explain a specific prediction (e.g., sample #0)
explanation = explainer.explain_prediction(sample_idx=0)

print(f"Prediction: {explanation['prediction']}")
print(f"Top Booster: {explanation['top_positive_contributors'][0]}") # e.g. ('is_holiday', +50.2)
print(f"Top Drag: {explanation['top_negative_contributors'][0]}")    # e.g. ('price', -10.5)

​

Interactive Transparency

The static reports are just the beginning. Business planners can use the Dash UI to run real-time “What-If” scenarios.

Scenario Planner

Adjust price or promo status and see how the SHAP values change instantly.

Drift Monitoring

Alerts when the model starts relying on features differently than before.