Interagency statement clarifies model validation expectations
In April 2021, a group of regulatory agencies, in consultation with the Financial Crimes Enforcement Network (FinCEN), clarified the 2011 Model Risk Management Guidance. This pertained to the validation of all models, not just those used to help financial institutions with their Bank Secrecy Act/Anti-Money Laundering (BSA/AML) and Office of Foreign Assets Control (OFAC) monitoring and regulatory reporting.
While the supervisory guidance does not have the force of a legal regulatory requirement, it is intended to assist financial institutions in determining the complexity of their BSA/AML model.
BSA/AML automated surveillance models include the Verafin, Abrigo, CSI NuMonitor, YellowHammer and BAM+ systems, just to name a few. These models comprise several components:
- Risk-based, which incorporates geographies, the nature of businesses and the volume of activity to determine high-risk customers.
- Behavior-based, which establishes a baseline for customers after analyzing activity for a set period and triggers alerts if baseline activity is exceeded.
- Rules-based, which are money laundering rules defined within the system and identify customer transactions that mimic those rules.
- Intelligence-based, which utilizes risk-based algorithms to identify suspicious activity.
Model validations
The 2014 Federal Financial Institutions Examination Manual (FFIEC) stated that financial institutions should review and test system capabilities and thresholds on a periodic basis and that the review should focus on specific parameters or filters to ensure intended information is accurately captured and the parameter or filter is appropriate for the financial institution’s particular risk profile.
When the OCC and FRB first released their supervisory guidance in April 2011, there was confusion among financial institutions about whether model validations were now a regulatory requirement or whether a system validation would be more appropriate based on the overall use of the system.
The difference between a model validation and a system validation is that the model validation takes a quantitative approach that applies statistical, economic, financial and mathematical theories, techniques and assumptions to process input data into quantitative estimates. A system validation consists of assessing the overall governance of the system and comparing data from the core processing system to ensure it has mapped correctly to the automated surveillance monitoring system.
What is considered a model, and when you need a validation
The 2021 interagency guidance also listed what is not considered a model, such as standalone systems that flag singular transactions over certain dollar thresholds (such as wire transfers or cash activity). Also, cash aggregation reports from the financial institution’s core system used to identify needed CTRs would not be considered a model.
The 2021 interagency guidance provided further clarification by stating that if the financial institution’s use of the model is limited and has limited impact on the financial institution’s financial condition, operations or compliance, a less sophisticated approach may be appropriate.
If the model’s use has a material impact on risk management decisions, operations and compliance, and if failure of the model could create financial harm to the financial institution, a more robust validation would be in order. Basically, financial institutions should conduct a risk-based analysis to determine the use of the system and its role in identifying and reporting suspicious activity.
Maintaining independence
The guidance also notes that as more models rely on artificial intelligence (AI) typology, validating those models may be more difficult. In addition, algorithm data from the model creator is often proprietary and not shared with an independent party. As such, a financial institution may take those restrictions into consideration in determining the scope of the validation.
The guidance did stress that any validation being conducted must be independent of the user, so if the financial institution does the validation in-house, the validation must be conducted by a party independent of the user or administrator of the system. For external validations, while the financial institution may rely on the model creator to assist with tuning the system or transaction code mapping, the model creator would not be independent to perform a validation. It is noted that the model creator could have an independent validation of its model platform that the financial institution should request as part of its vendor due diligence review.
Creating a contingency plan
Finally, the interagency guidance recommends that a contingency plan be in place in case the model becomes obsolete or is no longer effective. Risk considerations include ensuring a seamless process with the suspicious activity monitoring and regulatory reporting and data transfer to comply with BSA recordkeeping requirements.
How Wipfli can help
Wipfli has a team of professionals with expertise in automated surveillance monitoring validations and would be happy to provide a free consultation to discuss your financial institution’s needs. Contact us to learn more.
Sign up to receive additional financial institutions content and information in your inbox, or continue reading on:
