Module 01
The Econometric Mindset
Why MMM? Understanding the shift from user-level tracking (MTA) to top-down statistical inference for measuring incremental impact.
Read ModuleThe Context: StyleStream had already spent $42M over three years on digital media. The CFO wasn’t asking for a better dashboard—he was asking whether the next $1M would help or hurt EBITDA.
The Protagonist: Kamal, Lead Marketing Data Scientist at StyleStream.
The Antagonists:
- The CFO (Marcus): Focused on capital efficiency. "I don't believe in 'brand awareness.' Show me the cash."
- The Head of AI (Dr. Rao): The technical gatekeeper. "Any junior analyst can run a linear regression. Prove to me this model isn't overfitting noise."
Stage 01: The Hypothesis (Capital Allocation)
The Marketing Director wanted Kamal to calculate "ROI" (Return on Investment). Kamal refused.
"ROI is a vanity metric. It tells us what happened in the past on average. It doesn't tell us what to do next."
His Hypothesis: "I believe our primary channels (Facebook/Google) have hit Diminishing Returns. Our Average ROAS looks high (3.0x), but our Marginal ROAS (mROAS) is likely below 1.0x. We are losing money on the margin."
His Goal: Build a model to estimate mROAS and optimize budget allocation to maximize total revenue without increasing spend.
Stage 02: Data Engineering & EDA (The "Messy Reality")
Kamal pulled 156 weeks (3 years) of data. It was not clean.
The Challenges & The Fixes:
- The "Pixel Blackout": For 3 weeks in 2024, the tracking pixel broke, showing "0" spend for Facebook.
Dr. Rao's Challenge: "Did you drop those rows? You'll break the time-series continuity."
Kamal's Fix: "No. I used Imputation (rolling average of the adjacent 4 weeks) to maintain the signal integrity." - The Multicollinearity Trap: Facebook and Google spend were 95% correlated.
The Risk: A standard model wouldn't know which platform caused the sales. It might assign 100% credit to Facebook and 0% to Google arbitrarily.
Kamal's Fix: She documented this correlation and prepared to use Ridge Regression to penalize extreme coefficients. - Exogenous Shocks: Black Friday sales were 10x normal.
Kamal's Fix: She created a "Holiday Flag" (dummy variable) to isolate the event and added "Macro Controls" (Consumer Price Index) to account for market forces.
Stage 03: Methodology Selection (The Technical Defense)
In the technical review, Dr. Rao grilled him on his choice of algorithm.
- Dr. Rao: "Why didn't you just use OLS (Ordinary Least Squares)? It's interpretable."
- Kamal: "OLS extrapolates linear returns beyond observed data, implicitly implying no saturation. It would recommend infinite spending, making it unusable for optimization."
- Dr. Rao: "Okay, so why not a complex LSTM (Neural Network)?"
- Kamal: "Black box problem. The CFO needs to know why we are moving money. I can't explain a Neural Network's weights to the Board. I chose Ridge Regression with Non-Linear Transformations (Hill Function). It balances interpretability with the reality of saturation."
- Dr. Rao: "Proceed. But be clear: The model estimates incremental impact conditional on historical spend variation. It does not claim causal lift from channels that have never been meaningfully varied."
Stage 04: Feature Engineering (Physics of Adstock)
Kamal had to teach the model how human memory works.
- Parameter Selection: Decay rates and Saturation curves were not hand-tuned. They were selected via Grid Search optimizing for out-of-sample MAPE to prevent overfitting.
- Adstock (The Echo):
TV: High Decay (0.8). An ad seen today impacts sales for weeks.
Facebook: Low Decay (0.3). Immediate impact, low memory.
Stage 05: Validation (The "Stress Test")
Before the final presentation, Kamal ran rigorous checks.
- Back-Test: Predicted Q4 revenue with 1.6% MAPE.
- Residual Analysis: Durbin-Watson statistic of 2.05 (no autocorrelation).
- Shock Simulation: A 10% random spend shock simulation showed revenue elasticity behaved monotonically across all channels—no sign reversals.
Dr. Rao's Verdict: "The residuals look like white noise. The model is robust. You may proceed."
Stage 06: The Reveal (The Economics)
Kamal stood before the CFO. She displayed the Waterfall Decomposition Chart.
Insight 1: The Baseline Economics
- CFO: "Your model says the 'Intercept' is $400k. If I cut your budget to zero, do I keep getting that?"
- Kamal: "We decomposed baseline into intercept + seasonality + long-term trend. The $400k reflects non-paid demand under steady-state conditions, not a perpetual guarantee."
Insight 2: The Marginal Efficiency (The Killer Stat)
| Channel | Avg ROAS (Dashboard) | Marginal ROAS (Real Stats) | Status |
|---|---|---|---|
| 3.5x | 0.6x | 🛑 Over-Saturated | |
| Google Brand | 10.0x | 0.1x | 🛑 Maxed Out |
| TikTok | 1.2x | 1.8x | 🟢 High Potential |
Kamal's Narrative: "CFO, look at Facebook. The dashboard says 3.5x, but the Model proves that the last $50k we spent only brought back $30k (0.6x). We are losing EBITDA on every incremental dollar."
Stage 07: The Optimization (Board-Ready Strategy)
Kamal ran the Budget Optimizer, but she applied real-world constraints to ensure operational feasibility.
- Constraints: Channel-level floors (to maintain vendor contracts) and a ±30% change cap (to manage pacing).
- The Shift: Slash Facebook spend by 25% and reallocate to TikTok & YouTube.
- The Impact: "By making these shifts, we keep the total budget flat at $1M, but we project incremental revenue of +$250k next quarter."
The CFO's Response: "I don't understand Ridge Regression, Kamal. But I understand generating $250k for free. Do it."
Stage 08: Model Governance
To ensure this wasn't a one-off project, Kamal established a governance protocol. The MMM will be refreshed quarterly, with coefficient drift monitoring and a re-estimation trigger if MAPE exceeds 5% or mROAS rank order changes materially.
