Key takeaways
- Financial institutions remain accountable for their CECL estimates and should be able to demonstrate that their assumptions, data, controls and documentation can withstand scrutiny from auditors, regulators and boards.
- Outcome, assumption and decision backtesting can reveal whether reserves are consistently over- or understated, identify which assumptions are driving results and evaluate whether management actions based on CECL outputs were effective.
- Testing how key assumptions impact the allowance for credit losses (ACL) and comparing reserve levels to peer institutions can help institutions better understand, support and explain their CECL estimates.
- Periodic third-party reviews of models, assumptions, data inputs, governance and documentation help ensure CECL processes remain reliable, defensible and aligned with the institution’s risk profile.
CECL is no longer new. Financial institutions have implemented it, reported under it and been through audits and exams on it. But running the model successfully for a few years doesn’t mean you have it perfected.
The question is no longer “are we compliant,” but “how well do we actually understand our own model?” A CECL estimate that hasn’t drawn examiner or auditor pushback isn’t necessarily well-understood. It may just mean it hasn’t been tested yet.
The practices below aren’t about an initial CECL launch. They’re the next steps: backtesting, sensitivity analysis, benchmarking and validation. These are practices that many financial institutions still haven’t built into their processes, whether their models are built in-house or licensed from a vendor. And if you outsource your model, you’re still accountable for the results.
4 steps to strengthening your CECL process
Here are four steps to test whether your CECL process works as expected.
1. Backtesting
Backtesting compares your institution’s past credit loss forecasts to what actually happened, revealing whether your model is systematically over- or underestimating losses and, most importantly, if you can explain why.
There are three types you should consider doing: outcome, assumption and decision backtesting. Together, they answer three different questions. What happened? Why did it happen? And did we respond to it the right way? You don’t need to run all three at once, and they aren’t equally urgent. If your institution hasn’t done any backtesting yet, outcome should be first on your list. It’s the most accessible entry point, and it doesn’t need to be complex to be useful.
Outcome backtesting
Outcome backtesting is a retrospective comparison: pull your ACL estimates from two to four years ago and analyze how they compare to the actual cumulative net charge-offs that followed. It’s a directional test, not a precision test. The goal is to identify trends and magnitude.
The questions you want to answer are:
- Is the model consistently running high or low?
- By how much?
- Can I isolate why?
If qualitative factor overlays played a major role in driving a higher-than-actual reserve, that’s useful information. If you have no explanation for the divergence at all, that’s a gap worth closing.
Outcome backtesting doesn’t need to be elaborate when you’re getting started. It also doesn’t need a fixed schedule right away, either. Many institutions start by running it once a year, often alongside their annual model review, and build from there as the process matures: breaking results out by loan pool, extending the lookback period, tightening the definition of “actual” losses used in the comparison. The goal is to add sophistication over time, not to wait until you can do it perfectly to get underway.
Assumption backtesting
Where outcome backtesting looks at the final number, assumption backtesting examines the judgments that produced it: the inputs you controlled, not the economic conditions you didn’t. It’s a logical next step after establishing an outcome backtesting routine to help identify why a gap exists.
Pick a few assumptions that actually impact the model, such as prepayment speeds or loss emergence timing, and compare your assumptions to what happened. Did your prepayment assumptions match actual experience? Did your qualitative factor adjustments move in the right direction relative to observable conditions?
Document what you observed, how it compares to your original assumptions, and a simple conclusion. An example format is: “If our prepayment assumptions had matched actual experience, our ACL estimate would likely have been approximately $X lower.” That kind of statement is concrete, defensible and demonstrates an understanding of the model.
Decision backtesting
Decision backtesting is the most strategic of the three, but it’s also the one financial institutions do least often. It asks whether the actions management took in response to CECL outputs were appropriate.
The reserve number your model produces informs decisions. If the model indicated rising losses over the next several quarters and leadership tightened credit standards in response, did that play out as expected? If it projected higher prepayment speeds and you hired more staff to handle refinance volume, was that the right call?
This isn’t where most institutions should start, and few will need to formalize it right away. But for financial institutions with a mature backtesting program already running, it’s a natural extension.
Together, these three types of backtesting do more than satisfy regulators. Outcome backtesting tells you whether your model is running high or low. Assumption backtesting reveals why. Decision backtesting helps you understand whether the actions you took based on that information were correct. Most institutions get real value from stopping at the first two; the third is worth building toward once those are routine.
2. Sensitivity analysis
In sensitivity analysis, financial institutions change one key input at a time and measure how much it moves the ACL. Common inputs to test include prepayment and curtailment assumptions, loss emergence timing, forecast period length and qualitative factor adjustments. If you move your projected unemployment rate from 4% to 5%, how does that impact your reserve change?
This is valuable because it feeds directly into everything else on this list. If you know where your model is sensitive, you know where to focus your resources. The assumptions that move the needle most deserve the most rigorous support and documentation, as well as the closest attention from the board and examiners.
Sensitivity analysis should also impact your assumption backtesting. Use your sensitivity results to decide where to start. Test the assumptions that create the largest swings in your reserve before spending time on the ones that barely matter. That keeps your review focused on the top priorities. It also means you can walk your board and examiners through what’s actually driving your reserve, not just report the number.
3. Benchmarking
Quarterly call reports are public information, which means you can pull data from financial institutions similar in asset size, portfolio mix and geography and compare ACL ratios.
Benchmarking doesn’t provide a target. It also doesn’t replace your model. But if your reserve is significantly higher or lower than your peers’, that warrants deeper examination. If there is a big difference, you need to ask, “Can we explain why we’re different, and does that explanation hold up?”
Useful metrics for comparison include:
- ACL as a percentage of total loans
- ACL as a percentage of nonperforming loans
- Delinquency and nonaccrual ratios
- Charge-off coverage ratios
Be selective when identifying your peer group, and don’t cherry-pick institutions to justify a high or low reserve. If your numbers significantly differ from other institutions, document the reason and make sure it holds up.
4. Validation
Validation involves bringing in an independent third party to assess your model’s assumptions, data inputs, process controls, governance and documentation.
There’s no requirement for how often a CECL model must be validated. But it needs to be done. Model validation is a common area of inquiry for both federal and state examiners, and given how central the ACL is to your financial statements. Most examiners and auditors expect your institution to conduct independent validation on a defined, risk-based cadence, even without a rule specifying the interval.
The real question is not “do we need to validate?” The more important question is, “How often, and how deep?” Not every validation should look the same. The right scope and frequency depend on the nature of the model and the risk it carries, not a one-size-fits-all procedure.
Higher-risk situations, such as black-box third-party models, complex PD/LGD or DCF methodologies or heavy reliance on peer or macroeconomic data, generally warrant validation every 12 to 18 months.
Simpler, more stable models, such as an open-formula WARM calculation, can often run on an 18- to 36-month cycle.
The results of your own backtesting and sensitivity analysis are useful inputs here. A model showing instability in backtesting, or one that’s highly sensitive to a small number of assumptions, may warrant more frequent or more targeted validation.
The scope should also match the model. Validation of a weighted average remaining maturity (WARM) model will likely emphasize data integrity, assumption support and governance documentation. A vendor model will typically require a deeper review of how peer and economic data are applied and may call for loan-level recalculations. Before engaging a validation provider, ensure the proposed scope aligns with your model’s complexity and risk. That will help you avoid paying for procedures that don’t apply to your situation.
Read more
Whether your institution uses a third-party vendor model, an in-house WARM calculation or something in between, Wipfli can help you validate model assumptions, assess documentation and controls, perform back testing and sensitivity analysis and evaluate your ACL against a relevant peer group. We also support institutions navigating unfunded commitments, HTM securities and other evolving areas of CECL application. Start a conversation.