These questions landed in my inbox after the last MRM quarterly review. Your team's probably asking the same ones.
The CECL framework replaced incurred loss accounting with a forward-looking expected credit loss model. Great in theory. In practice, you're now building models that depend heavily on macroeconomic forecasts, and those forecasts have a nasty habit of being wrong exactly when you need them most. The CECL framework is significantly more sensitive to macroeconomic forecasting errors and model misspecification than the incurred loss framework it replaced. That sensitivity doesn't just increase your allowance volatility; it fundamentally changes your model risk profile.
These questions come from teams managing consumer loan portfolios (auto, mortgage, credit card) who've now lived through COVID-19 and are staring at the next economic uncertainty. They're tired of theoretical guidance and want to know what actually works.
Q: Our CECL model blew up during COVID. How do we keep that from happening again?
Build for adaptability, not just accuracy.
The problem wasn't that your model was wrong; it's that you couldn't fix it fast enough when conditions shifted. Drawing on more than 20 years of auto loans data and experience from the 2007-9 Great Recession and the 2020-21 COVID-19 pandemic, the pattern is clear: models that survived economic shocks weren't necessarily the most sophisticated ones. They were the ones teams could recalibrate quickly.
Start with your model infrastructure. Can you develop, estimate, and deploy an array of models quickly without a forecasting performance penalty? If your answer is "it takes six months to get a new model through governance," you've got an infrastructure problem, not a modeling problem.
This means:
- Standardize your model development pipeline so you can test alternative specifications rapidly.
- Pre-approve a framework for emergency model recalibration (define triggers, approval paths, documentation requirements).
- Maintain multiple model specifications in parallel, even if you're only using one for reporting.
- Document your macroeconomic scenario selection process so you can defend changes under stress.
Your validators will push back on "too many models." Show them the cost of being stuck with a broken model during the next shock.
Q: Should we be using machine learning for CECL, or is that just asking for trouble?
Use it strategically, not because it's trendy.
Simple machine learning strategies can help you build a nimble and flexible CECL modeling framework, but the key word is "simple." You don't need deep learning for this. You need techniques that let you handle large datasets efficiently and test multiple specifications quickly.
Even in consumer loan portfolios with tens of millions of loans (mortgage, auto, or credit card portfolios), it's possible to develop, estimate, and deploy an array of models quickly and efficiently without a forecasting performance penalty. ML helps here because it automates parts of the feature engineering and model selection process that would otherwise bottleneck your team.
Practical applications:
- Use ML for segmentation and feature selection to identify which loan characteristics actually matter for loss prediction.
- Apply ensemble methods to combine multiple model specifications and reduce reliance on any single forecast.
- Use regularization techniques (LASSO, ridge regression) to build stable models that don't overfit to historical patterns.
What you shouldn't do: deploy a black-box ML model you can't explain to your audit committee. SR 11-7 still applies. Your validators need to understand the model logic, and you need to document why the model is appropriate for your portfolio.
Q: How do we handle macroeconomic forecasts that are obviously wrong?
You need a process for recognizing and correcting biased projections during times of high economic shock.
First, define "obviously wrong." Your documentation should specify the conditions under which you'll override or adjust vendor forecasts. Consider:
- Historical forecast accuracy during prior downturns (track vendor performance through 2008-9 and 2020-21).
- Cross-vendor dispersion (if three forecast providers disagree wildly, that's a signal).
- Internal economic indicators that contradict the vendor scenario.
- Stress test scenarios that should bracket your CECL scenarios.
When you do adjust forecasts, document the econometric principles behind your adjustment. Basic principles work: if unemployment forecasts lag observed data by two quarters, apply a correction factor based on the current employment reports. If GDP forecasts assume a V-shaped recovery but your portfolio shows continued stress, build a scenario that reflects your actual portfolio behavior.
The governance piece matters here. Get your ALCO or model risk committee to pre-approve the conditions and process for forecast adjustments. You don't want to be arguing about methodology in the middle of a crisis.
Q: What does "model resiliency" actually mean in practice?
It means your model infrastructure can withstand novel shocks and uncertain economic conditions without collapsing.
Resilient models have these characteristics:
- They perform acceptably across multiple economic scenarios, not just the base case.
- They can be recalibrated using recent data without complete redevelopment.
- They produce stable estimates when inputs change gradually (no cliff effects).
- They fail gracefully (when they're wrong, they're not catastrophically wrong).
Test this explicitly. Run your CECL model using 2019 data and 2020 forecasts. How far off were your projections? More importantly, if you'd recalibrated using Q2 2020 data, how quickly could you have corrected course?
Build recalibration into your annual model validation cycle. Don't just validate whether the model is performing; validate whether your team can adapt it when performance degrades.
Q: Our validators keep asking for more complexity. How do we push back?
Show them the performance data.
Complexity doesn't equal accuracy, especially in CECL models where macroeconomic forecast error often dominates model specification error. A focus on the resiliency and adaptability of models and model infrastructures to novel shocks beats pure complexity every time.
Run a backtest: compare your current complex model's performance during COVID against a simpler specification. If the simpler model performed comparably (or better), you've got your answer. The simpler model is also easier to recalibrate, easier to explain, and easier to validate.
Your validators' job is to ensure the model is fit for purpose under SR 11-7. "Fit for purpose" includes operational feasibility. A model you can't recalibrate during a crisis isn't fit for purpose, regardless of its R-squared.
Q: What should we be monitoring between validation cycles?
Track forecast accuracy and model sensitivity, not just model output.
Set up ongoing monitoring for:
- Macroeconomic forecast accuracy (compare vendor forecasts to actual outcomes quarterly).
- Model sensitivity to forecast changes (how much does your allowance move when unemployment shifts 1%?).
- Segment-level performance (are specific loan vintages or geographies behaving differently than projected?).
- Recalibration triggers (define quantitative thresholds that would prompt model review).
This monitoring feeds your next validation cycle and gives you early warning when the model needs attention. Don't wait for your annual validation to discover your model stopped working six months ago.
Where to go for more
SR 11-7 remains your foundation. Read Section III on model development and implementation with CECL in mind; the principles around conceptual soundness and ongoing monitoring apply directly.
For your next model validation, ask your validators to assess adaptability alongside accuracy. Can your team recalibrate this model in 30 days if economic conditions shift? If not, you're carrying more model risk than your allowance number suggests.
And document everything. When you do adjust forecasts or recalibrate models, that documentation protects you in the next exam cycle. Examiners understand that forecasts fail. They don't understand why you didn't have a plan for when they did.
