How to validate your loan portfolio stress testing model
Our previous articles on loan portfolio stress testing covered what it is and why smaller financial institutions should do it, as well as why it’s important in the wake of the COVID-19 pandemic. But once you have a loan portfolio stress testing model in place, validating it should become a priority.
The health of the financial institutions industry relies heavily on the health of the economy and the financial markets, which we all know have their ups and downs over time and are cyclical in nature. It can be easy to forget this during prolonged periods of stability and growth, but it means that financial institutions should always be prepared for times of trouble and have procedures in place to identify, measure, monitor and control the various risks to their balance sheet.
Credit risk is particularly important because your loan portfolio probably makes up the largest portion of your institution’s assets. Conducting loan portfolio stress testing model validation on a regular basis can help your institution ensure the model and procedures in use are working as intended.
Model validation vs. process validation
The difference between a model validation and a process validation is often misunderstood, as the two terms are often used interchangeably. However, they are separate procedures that together form a comprehensive validation covering all the areas required or expected by regulatory guidance.
A model validation looks at the black box of the model itself. This is where a qualified, independent third party certifies the model’s mathematical calculations and technical forecast accuracy by running and rerunning the model with a range of inputs and then verifies that the model’s outputs appear reasonable and can be relied on for forecasting purposes.
A process validation looks at the financial institution’s overall process — including review and assessment of the model inputs and assumptions, processing and reporting procedures, as well as the policies and procedures in place to measure, monitor and control any identified risk.
Guidance on model validation and model risk management
In FIL 22-2017, the Federal Deposit Insurance Corporation (FDIC) states that all model components — including input, processing and reporting — are subject to validation. This applies to models developed in house and those purchased from or developed by vendors or consultants.
Model validation activities should continue on an ongoing basis after a model goes into use in order to track known model limitations and to identify any new ones. A great reason why is because during times of benign economic and financial conditions, estimates of risk and potential loss can become overly optimistic, and the data at hand may not fully reflect more stressed conditions. Validation helps identify any areas in which there may be over- or underreporting of the actual level of risk exposure due to inaccuracies in the input data or the use of inappropriate or outdated assumptions. Furthermore, validation procedures help ensure that the institution’s risk monitoring and reporting procedures are sufficient to allow management to identify and respond to potential risks before they materialize.
Your financial institution should conduct a periodic review (at least annually, but more frequently if warranted) of each model used to determine whether it’s working as intended and whether your existing validation activities are sufficient.
The four components of process validation
Process validations typically focus on four key components, one of which is model validation. Your independent third party should cover all four of these components during process validation:
1. Policies and procedures
The third party reviews your policies and procedures to ensure appropriate corporate governance is in place, including appropriate oversight by senior management and the board of directors. They also review whether you have a system of internal controls in place to ensure model inputs are regularly reviewed for accuracy, and model assumptions and inputs are reviewed for reasonableness in comparison to actual results.
2. Model validation
As we mentioned above when defining model validation, this is when a third party, typically engaged directly by the model vendor, performs a test on the mechanics and technical functionality of the model and certifies the mathematical forecast accuracy of the model. Your third party reviewer should verify that this was completed timely by a qualified party and that the testing conducted was sufficiently comprehensive to ensure the model is working as intended.
3. Assumptions
Next, the third party looks at the assumptions supplied in the model and whether the assumptions are derived from institution-specific data or from vendor-provided defaults, which are usually based on industry averages.
The assumptions are assessed for reasonableness as to their applicability to your institution’s historical, current and projected experience. This includes ensuring management has assessed any empirical, quantitative results alongside any qualitative factors used to adjust those results.
Assumption review also includes an assessment of any sensitivity testing and backtesting of model results.
4. Reporting
Lastly, the third party reviews management reports and committee and board meeting minutes to ensure your institution has comprehensive management information systems in place and that model results are regularly reviewed at both the management and board level. They review meeting minutes to ensure any results falling outside board-approved policy limits are promptly reported to committee members and the board, along with any action plans to remedy the exception.
Taking action on your results
Once you have the process validated, it’s time to use those results. Reviewing loan stress testing results creates an opportunity to identify possible areas of weakness in your credit administration and underwriting practices.
Your management team should take the results obtained from loan stress testing models and incorporate them into your interest rate risk, liquidity and capital forecasting models in order to assess the potential impact of identified credit risk on these areas, as well. You can then use the results of these reviews to develop strategies to mitigate potential risks.
Again, your management team should take advantage of any opportunities to mitigate credit risk during times of stability, as those opportunities may no longer be available once a real-life stress event has already begun.
Wipfli can perform your loan portfolio stress testing model validation
If you’re ready to conduct loan stress testing model validation, Wipfli can help. We perform process validation — including all four key components — to verify your model’s outputs appear reasonable and that you can rely on it for forecasting purposes. Contact us to learn about our stress testing model validation service.
Sign up to receive additional financial institutions information in your inbox, or continue reading on:
