“While each bank is unique and faces its own specific challenges, we have noted several ways banks are transforming the model lifecycle to improve the efficiency and effectiveness of activities across development, validation and governance functions.”
The banking industry is facing increasing demands and challenges across the entirety of the model lifecycle, from a model’s inception and development through validation and eventually monitoring and maintenance. Based on our experience working with clients across the model lifecycle, these include:
- Rising model1 complexity in the ever-changing bank regulatory landscape.
- New technologies and modeling approaches are calling for enhanced capabilities.
- Growing number of models required to address increasing regulatory and stress testing requirements.
- Equally important are increased business requirements driven by greater reliance on Artificial Intelligence and Machine Learning models and algorithms throughout organizations.
- Increase in volume of work disproportionate to the resources available.
- The ever-present requirement for talent with robust, diverse skillsets.
- Growing number of models and model interdependencies.
- Linear relationships between the number of models and the cost of operations.
- Compromised cost efficiency and increased turnaround time to address business needs, meet regulatory compliance, and maintain validation and monitoring quality.
While each bank is unique and faces its own specific challenges, we have noted several ways banks are transforming the model lifecycle to improve the efficiency and effectiveness of activities across development, validation and governance functions.
A foundational building block that facilitates alignment between a bank’s strategy, processes, technology and talent is the identification of appropriate owners of model risk. While many institutions, in our experience, have assigned model owners, a lack of guidance on the ultimate ownership of model risk prohibits banks from right-sizing activities across functions.
Model Risk Profile
Not all risks are created equal, and the components of model risk are no different. By developing an understanding of the overall risk profile of their models, banks could drive appropriate resource allocation and streamline procedures across the model lifecycle. Banks can perform this through model risk assessment frameworks that rank-order models from least to most risky. Our experience shows that such frameworks have been around for some time but faced with increased capacity demands through the proliferation of models, modeling techniques and frameworks, they are being revisited to better discriminate the relative riskiness of models.
Our discussions with clients and industry practitioners highlight that banks are championing initiatives that reduce the number of models in their inventory. This is not merely a case of updating taxonomy, but rather identifies cases where models, or components of a modeling pipeline, only need to be validated once, as opposed to being validated multiple times. A relevant example is a bank performing a single validation of loss forecasting models, but in that single validation exercise, the outcomes testing considers the model’s use for capital stress testing and for Current Expected Credit Losses (CECL) purposes.
Such examples of build-once-use-many are ubiquitous in the treasury and trading/market risk space, where underlying market factors (such as interest rates, equity returns, foreign exchange (FX) rates) are produced by the same models but used for multiple products. Additional opportunities exist to consolidate risk management and front-office or business-as-usual models for purposes such as Value at Risk (VAR) calculations.
Risk-Based Rationalization of Effort
A key lever we’ve observed utilized in improving efficiency, is the frequency of model (re)validations and the depth of development, validation and monitoring activities by model family (i.e. Retail, Wholesale, Trading), model type (Capital Forecasting, Underwriting/Scorecard, Fraud, Valuation) and by risk rating.
While many banking institutions review models at least annually to meet SR 11-72 model risk management guidelines, they are fine-tuning the length of their full (re)validation cycles. From our client observations, the higher risk models generally undergo full revalidation every 12 to 18 months, and the lower risk models undergo full revalidation every 36 months, with instances of even longer review cycles.
Regardless of the level or methods used to attain effort rationalization, robust ongoing monitoring framework is a key consideration to support effort saving initiatives. Automation is one of the ways that this could be realized.
Banks are increasingly relying on technology to support the automation of monitoring and documentation activities. Automation of monitoring execution and documentation can help streamline and increase effectiveness of reviews and model updates.
Documentation is a significant overhead for development and validation teams. Clients have traditionally addressed this through documentation templates that standardize testing activities and documentation content. More recently, we have had discussions with a client around how Artificial Intelligence processes, such as Natural Language Generation (NLG) could be used to help reduce the cost and speed up production of model development and validation documentation.
Governance, Policies and Operating Model Redesign
In conjunction with rationalizing model lifecycle activities and consolidating models, banks can consider revamping their policies, procedures and operating models. By pairing an updated operating model and policies with refined roles and responsibilities, banks can position themselves to unlock efficiencies without sacrificing compliance or quality.
- The definition of what constitutes a model that is subject to validation is determined by each bank individually. However, firms generally rely on regulatory guidance as provided by the Federal Reserve in the U.S. through SR11-7, and the Office of the Superintendent of Financial Institutions in Canada through E-23 . Please see: “SR 11-7 Guidance on Model Risk Management,” Board of Governors of the Federal Reserve System, April 4, 2011, accessed at: https://www.federalreserve.gov/supervisionreg/srletters/sr1107.htm; and “Enterprise-Wide Model Risk Management for Deposit-Taking Institutions,” Office of the Superintendent of Financial Institutions, September 2017, accessed at: http://www.osfi-bsif.gc.ca/Eng/fi-if/rg-ro/gdn-ort/gl-ld/Pages/e23.aspx.
- “SR 11-7 Guidance on Model Risk Management,” Board of Governors of the Federal Reserve System, April 4, 2011, see page 10. Accessed at: https://www.federalreserve.gov/supervisionreg/srletters/sr1107.htm.