Monitoring & Maintenance¶
Unlike traditional software bugs — which are deterministic and reproducible — ML model failures are probabilistic, gradual, and often invisible until they cause measurable business harm. Interviewers ask about monitoring to find candidates who understand this distinction and build systems that catch degradation before users do.
Q1: Why do ML models degrade in production, and how is this different from software bugs?¶
Show answer
A software bug is a deterministic error: given the same input, the same wrong output is produced every time. It can be reproduced, diagnosed, and fixed with a code change.
ML model degradation is statistical: the model's code is correct, but the world it was trained to represent has changed. The same input that once produced a good prediction now produces a worse one — not because of a bug, but because the relationship between inputs and outputs has shifted.
The core mechanisms of ML degradation:
Data drift: the distribution of incoming features changes. The model was trained on feature distributions from six months ago; today's inputs look meaningfully different. The model's learned boundaries no longer generalise well to the new distribution.
Concept drift: the relationship between features and the target changes. A fraud model trained on pre-pandemic transaction patterns sees different fraud patterns post-pandemic. The features are the same, but what those features mean for the target label has changed.
Feedback loops: the model's predictions influence user behaviour, which changes the data distribution. A recommendation system that shows users content they click on creates a filter bubble — the distribution of content consumption shifts as a result of the model's own recommendations.
Data pipeline degradation: upstream data quality silently worsens — null rates rise, a feature source goes stale, a schema change breaks a join. The model's inputs degrade without the model itself changing.
Why this is hard to detect: in software, an exception is thrown. In ML, the model keeps returning predictions — they're just increasingly wrong. You need explicit monitoring to catch this.
Q2: What is data drift and how do you detect it?¶
Show answer
Data drift (also called feature drift or covariate shift) occurs when the distribution of input features changes between training time and serving time.
Why it causes problems: the model learned decision boundaries based on the training distribution. When serving inputs come from a different distribution, predictions extrapolate into regions the model was never trained on.
Detection methods:
Population Stability Index (PSI): PSI compares the distribution of a feature in two populations (training vs current serving).
Interpretation: - PSI < 0.1: stable, no action needed - 0.1 ≤ PSI < 0.2: moderate shift, investigate - PSI ≥ 0.2: significant shift, likely requires retraining
Kolmogorov-Smirnov (KS) Test: A statistical test that measures the maximum distance between two empirical cumulative distribution functions. Returns a p-value — reject the null hypothesis (same distribution) if p < 0.05.
from scipy.stats import ks_2samp
def detect_drift(training_values, serving_values, threshold=0.05):
statistic, p_value = ks_2samp(training_values, serving_values)
return {
"drift_detected": p_value < threshold,
"ks_statistic": statistic,
"p_value": p_value
}
Jensen-Shannon Divergence: symmetric version of KL divergence — measures how different two distributions are. Range [0, 1] where 0 = identical distributions.
Practical approach: monitor the top 10–20 features by importance, not all features. High-importance features drifting is a much stronger signal than low-importance feature drift.
Q3: What is the difference between data drift and concept drift?¶
Show answer
These are often confused. The distinction matters because they require different responses.
Data drift (covariate shift): - The distribution of input features X changes: P(X) changes - The relationship between X and Y does not change: P(Y|X) stays the same - Example: your fraud model is trained mostly on desktop transactions. Mobile usage grows and now 60% of traffic is mobile. The feature distributions shift, but fraud still looks the same for any given feature combination. - Response: retraining on data that represents the new distribution often fixes this
Concept drift: - The relationship between X and Y changes: P(Y|X) changes - Input distributions may or may not have changed - Example: a credit model is trained during low-interest rates. When rates rise, borrower behaviour changes — the same credit score now predicts a different default probability. The features are the same; their predictive power has changed. - Response: requires new labelled data from the new regime, and possibly a new model architecture if the underlying relationship has changed fundamentally
Why the distinction matters operationally: - Data drift can sometimes be addressed by re-weighting the training sample (importance weighting) without retraining on new labels - Concept drift requires new labelled data — and collecting that data may take time (especially when labels are delayed, like churn or loan default)
Both can happen simultaneously, which is the hardest case. Monitoring both input distributions and model performance gives you the signal to distinguish them.
Q4: How do you monitor model performance when ground truth is delayed?¶
Show answer
Ground truth lag is the gap between when a prediction is made and when you can observe whether it was correct.
Examples of ground truth delay: - Fraud detection: chargebacks arrive 30–90 days after a transaction - Churn prediction: you predict churn today; you won't know if the user churned for another 30 days - Medical diagnosis: treatment outcomes take months or years to observe - Credit default: loans take years to mature
Strategies for monitoring without immediate ground truth:
Proxy metrics: Use observable signals that correlate with true performance, available without waiting for labels. - For fraud: dispute rate, manual review escalation rate - For recommendation: click-through rate, session length, skip rate - For search: query reformulation rate, dwell time, zero-result rate Risk: proxy metrics may diverge from true performance. Monitor the correlation between proxy metrics and true performance when ground truth does eventually arrive.
Input monitoring: Monitor feature distributions directly. If inputs are drifting, performance is likely drifting — even before you have labels to confirm it.
Slice-based monitoring: Track model performance on fast-feedback cohorts (users who have short outcome observation windows) as an early indicator for the full population.
Calibration monitoring: For models that output probabilities: check that predicted probabilities match observed frequencies on recently-labelled subsets. A model predicting 90% probability for events that are actually occurring 60% of the time is miscalibrated — detectable without waiting for the full ground truth.
Label pipeline: Build a fast-path label pipeline that produces partial, noisy labels quickly (e.g. using heuristics or weak supervision) to enable faster feedback, alongside the slow but accurate true label pipeline.
Q5: What is a good alerting strategy for ML systems?¶
Show answer
Bad alerting is worse than no alerting: too many alerts causes alert fatigue, which means real incidents get ignored. A well-designed alert strategy is selective, actionable, and tiered.
Alert tiers:
Critical (page on-call immediately): - Model server error rate > 1% for > 5 minutes - Prediction latency p99 > SLA threshold - Serving pipeline completely down (no predictions being served) - Data pipeline producing zero output
Warning (ticket for next business day): - PSI > 0.2 on a top-5 important feature - Null rate > 3x historical baseline for any feature - Model performance proxy metric drops > 5% week-over-week - Input volume outside expected range (< 50% or > 200% of expected)
Informational (log, review in weekly review): - Minor distribution shifts (PSI 0.1–0.2) - New categorical values appearing in inputs - Gradual performance trends (downward but within acceptable range)
Anti-patterns to avoid: - Alerting on every metric crossing a fixed threshold: thresholds need context — a 10% drop in CTR on Christmas Day is expected, not an incident - Using the same threshold for all traffic levels: percentage-based thresholds are more stable than absolute thresholds - Not reviewing alerts that fire repeatedly without action — an alert that is never acted on should either have a higher threshold or be removed
What interviewers want to hear: that you think about actionability — every alert should have a clear "if this fires, do X" runbook. An alert with no clear response is noise.
Q6: What are the retraining triggers and which is most reliable?¶
Show answer
Retraining triggers define when the model is retrained. There are three families:
1. Schedule-based (time-triggered): - Retrain every day/week/month regardless of detected drift - Simple to implement and reason about - Reliable as a baseline — even if drift monitoring fails, the model gets periodically refreshed - Wasteful if data is stable (retraining when not needed) and insufficient if data changes faster than the schedule
2. Performance-based (metric-triggered): - Retrain when model performance drops below a threshold (e.g. AUC drops below 0.82) - Requires ground truth to be available promptly — has a blind spot during ground truth lag periods - The most directly motivated trigger: retrain exactly when needed
3. Drift-based (distribution-triggered): - Retrain when input feature drift exceeds a threshold (PSI > 0.2 on a key feature) - Does not require ground truth — can fire before performance degrades - Acts as an early warning system: drift precedes performance degradation - Risk of false positives — drift doesn't always cause meaningful performance degradation
The right answer: use all three, with different priorities. - Drift detection fires first (early warning) - Performance monitoring confirms the signal when ground truth is available - Scheduled retraining serves as a safety net
Drift alert → investigate → confirm with performance metrics if available → trigger retraining
Scheduled retraining → runs weekly regardless → catches anything monitoring missed
What makes an answer strong: acknowledging that each trigger has failure modes, and that a combination of triggers provides defence in depth.
Q7: What is the champion-challenger framework?¶
Show answer
The champion-challenger framework is a structured approach to evaluating and deploying model updates in production.
Champion: the current production model serving the majority of traffic. Known performance, known failure modes.
Challenger: a candidate model — newly trained, using new features, or with different architecture — that might perform better.
The framework: 1. Train and validate the challenger model offline 2. Deploy the challenger alongside the champion in shadow mode — same inputs, predictions logged but not used 3. Compare offline predictions: do the models agree? Where do they disagree — and which one is right? 4. Promote the challenger to a small traffic slice (5–10%) via A/B test 5. Run the A/B test until statistical significance is reached on the primary metric 6. If challenger wins → promote to 100% traffic (new champion), retire old champion 7. If challenger loses or ties → return to 0% traffic, analyse why it didn't improve
Guardrail criteria for promotion: - Primary metric improvement is statistically significant (p < 0.05) - No regression in guardrail metrics (latency, error rate, revenue per user) - Challenger passed shadow mode without producing invalid predictions
Why this matters: it prevents both premature promotion (deploying before you know it's better) and risk aversion (never deploying because the bar is unclear). The champion-challenger model makes the decision criteria explicit and repeatable.
Q8: What is the feedback loop problem in ML systems?¶
Show answer
A feedback loop occurs when a model's predictions influence the data that will be used to train the next version of the model.
The mechanism: 1. Model makes predictions that influence real-world outcomes (recommendations, loan approvals, content ranking) 2. Those outcomes generate new training data 3. The new model learns from data that was shaped by the old model's predictions 4. Each iteration, the model's own biases become more entrenched
Classic examples:
Recommendation systems (filter bubble): a model recommends content a user is likely to click. The user only sees and interacts with recommended content. Future training data reflects only recommended content — content the model didn't show is never clicked and appears to have zero engagement. The model becomes increasingly confident in a narrowing set of items.
Credit scoring (redlining): a model denies loans to residents of a ZIP code due to historical default rates. Those residents never get loans, so they can't build credit history. Future training data confirms the model's prediction — no loans, no repayments, no positive credit signals. The model's bias perpetuates itself.
Fraud detection (survivorship bias): transactions the model flags as fraud are declined — no chargeback data ever arrives for them. The model never gets feedback on the fraud it successfully blocked, only on fraud it missed. Over time, it underestimates fraud rates in the patterns it's most confident about.
Mitigation strategies: - Exploration: randomly serve some non-model-driven decisions (random recommendations, occasional loan approvals outside the model's usual range) to collect unbiased data - Counterfactual logging: record what the model would have shown/decided if it had made a different choice - Offline policy evaluation: simulate alternative policies on logged data before deploying them - Fairness constraints: explicitly constrain the model to expose diverse content or approve across demographic groups to break homogenisation loops
Q9: How do you monitor feature importance in production?¶
Show answer
Feature importance monitoring detects when the predictive signals the model depends on are weakening or changing — a leading indicator of performance degradation.
What to monitor:
SHAP value distributions: Compute SHAP values on a sample of serving predictions (or a held-out validation set) and track the distribution over time. A feature that consistently contributes large SHAP values suddenly contributing near-zero SHAP values means the model is no longer finding that feature useful — either the feature is drifting, or the signal has genuinely weakened.
import shap
explainer = shap.TreeExplainer(model)
shap_values = explainer.shap_values(X_serving_sample)
# Mean absolute SHAP per feature
feature_importance = pd.Series(
abs(shap_values).mean(axis=0),
index=feature_names
).sort_values(ascending=False)
Correlation monitoring: Track the correlation between each feature and the target label on a rolling window of recently-labelled data. A feature whose correlation with the target is declining is losing its predictive power.
Feature ranking stability: Track whether the top-10 features by importance remain stable over time. A sudden change in which features the model relies on most suggests a structural shift in the data.
What a shift in feature importance tells you: - Top feature dropping to near-zero importance → that feature's data source may be broken or stale - A previously low-importance feature suddenly becoming important → a new pattern has emerged that the model is exploiting, possibly spuriously - General reshuffling of importance rankings → the data distribution has shifted enough that the model's learned structure is mismatched to current data
Q10: When should you fall back to human review instead of automated decisions?¶
Show answer
Human-in-the-loop (HITL) is a deliberate architectural choice, not an admission of model failure. The question is: when does human review have positive expected value?
Cases where HITL is mandatory: - Regulatory requirement: credit, medical, and legal decisions often legally require a human to be in the decision chain - High-stakes, irreversible decisions: denying a medical claim, terminating an account, flagging content that results in legal action - Low prediction confidence: the model's predicted probability is close to the decision threshold (e.g. fraud score between 0.4 and 0.6) — these are inherently uncertain cases
Cases where HITL adds value operationally: - Model performance on a specific segment is significantly worse than average — route those cases to reviewers while the model is retrained - New pattern detection: cases that look unlike anything in the training distribution (measured by distance to nearest training neighbours or low softmax confidence) - After a model update: briefly increase human review rate to validate the new model's behaviour before fully trusting it
Designing the HITL queue efficiently: - Only route to humans when the model's confidence is genuinely low — don't route everything - Use a priority queue: highest-stakes or most ambiguous cases reviewed first - Track reviewer decisions and feed them back into training data (with appropriate label quality controls) - Measure reviewer agreement rate — if reviewers disagree with each other, the task itself may be underspecified
The trap to avoid: using human review as a permanent patch for a model that isn't good enough. Human review is expensive and doesn't scale. The goal is to route only the genuinely uncertain cases, not to cover for a poorly performing model indefinitely.
Q11: How do you define and measure SLAs for ML systems?¶
Show answer
A Service Level Agreement (SLA) for an ML system is a formal, measurable contract about the system's behaviour. Vague commitments like "the model should be accurate and fast" are not SLAs.
SLA dimensions for ML systems:
Availability: percentage of time the system serves predictions successfully. - "99.9% availability" = at most ~8.7 hours of downtime per year - Measured as: (successful prediction requests / total prediction requests) × 100
Latency: maximum acceptable response time, measured at a percentile. - "p99 latency < 200ms" — the 99th percentile of response times must be under 200ms - Measured continuously using distributed tracing (Datadog, Jaeger, AWS X-Ray)
Freshness: maximum acceptable staleness of model predictions. - "Model retrained within 24 hours of detecting significant drift" - "Batch predictions refreshed at least daily"
Quality: model performance must stay above a threshold. - "AUC ≥ 0.80 on the weekly evaluation set" - "Precision ≥ 0.90 on critical fraud patterns" - Note: quality SLAs require ground truth — define which evaluation set and frequency
Defining SLOs (Service Level Objectives) vs SLAs: - SLO: internal target (p99 < 150ms) - SLA: external commitment with consequences for breach (p99 < 200ms, or credits issued) - Set your SLO tighter than your SLA to give yourself a buffer
Error budget: If your availability SLA is 99.9%, you have 0.1% of the month (about 43 minutes) as an error budget. When the error budget is exhausted, halt new deployments and focus on reliability until it recovers. This creates the right incentive to not sacrifice reliability for feature velocity.
Q12: What should a post-deployment model card document?¶
Show answer
A model card is structured documentation that accompanies a deployed model and describes what it does, how it was evaluated, and where it is known to fail. Originally proposed by Google; increasingly required by regulation in high-stakes domains.
What a model card should contain:
Model details: - Model type, version, training date - Features used (schema) - Training data description (source, date range, size) - Intended use cases and out-of-scope uses
Performance summary: - Overall metrics on the evaluation set (AUC, F1, RMSE, etc.) - Performance broken down by slice: demographic groups, geographic regions, product categories - Performance vs the baseline (rule-based system, previous model version)
Known failure modes: - Subgroups where performance is significantly worse than average - Input ranges or edge cases where predictions are unreliable - Known biases in training data that may affect predictions
Monitoring information: - Drift thresholds that trigger retraining - Metrics and dashboards to track - Who is the on-call owner
Fairness analysis: - Demographic parity, equalised odds, or other fairness metrics where applicable - Any intentional design choices to constrain predictions for fairness
Why this matters operationally: When a model is causing harm or underperforming, the first question is "what do we know about this model's limitations?" If the answer requires reverse-engineering the training code, the model card was never written. A model card that is filled out at deployment forces teams to confront known weaknesses before they cause incidents — and gives the on-call engineer a starting point when something goes wrong.