There are three key elements of model validation:

1. conceptual soundness,
2. data quality and appropriateness, and
3. model performance

Data quality and appropriateness: involve assessing the quality, completeness, and accuracy of the data used in the model. This involves verifying that the data used in the model is appropriate for the intended use, that it is free from errors or biases, and that it is representative of the population being modelled.

Model performance: involves evaluating the accuracy, reliability, and consistency of the model’s output. This includes testing the model against historical data and comparing its predictions with actual outcomes. Model performance testing should also include sensitivity analysis to assess the model’s performance under different scenarios and assumptions. This can help identify potential weaknesses or limitations in the model and improve its accuracy and reliability.

In addition to these key elements, model validation also involves ongoing monitoring and maintenance of the model to ensure that it remains accurate and relevant. This includes regular updates to the model to reflect changes in the underlying data or the business environment, as well as ongoing testing and review of the model’s performance.

Model validation is an essential component of risk management in banking. It helps ensure that banks are making informed decisions based on accurate and reliable data, and that they are complying with regulatory requirements and industry standards. Effective model validation can help improve the accuracy of risk assessments, enhance the quality of financial products, and ultimately, promote financial stability and consumer protection.

History has shown that failure to properly validate models can have serious consequences for financial institutions. In the lead-up to the 2008 financial crisis, for example, some banks used complex models to value mortgage-backed securities and other complex financial instruments. These models failed to account for the potential risks associated with the housing market, leading to significant losses when the housing bubble burst [1].

In another example, in 2012, JPMorgan Chase suffered a $6 billion trading loss resulting from the inadequate validation of its Value-at-Risk (VaR) model. The model, which was used to estimate the potential losses that could be incurred by JPMorgan’s Chief Investment Officer, was found to have several weaknesses, including a failure to take into account certain risk factors [2].

In 2018, Wells Fargo was fined $1 billion for, among other things, failing to properly validate its models for calculating risk-weighted assets under the Basel III regulatory framework. The models, which were used to determine the amount of capital the bank needed to hold to cover potential losses, were found to be deficient and resulted in an incorrect calculation of risk-weighted assets [3].

In the early 2010s during South Africa’s “unsecured lending” crisis, several South African banks experienced significant losses due to their exposure to unsecured lending, which refers to loans that are not backed by collateral. It was later revealed that the banks’ models for assessing credit risk in this area were inadequate, leading to an underestimation of the default risk. This led to a wave of defaults and a subsequent crackdown by regulators on unsecured lending [4].

Over the past few years, the US banking industry has been alarmed by a troubling trend of regional bank failures, with institutions like Silicon Valley Bank (SVB), Signature Bank, and First Republic Bank facing intense scrutiny. Among the various factors contributing to these failures, one significant concern has been the inadequate validation of risk management models.

Silicon Valley Bank (SVB), renowned for catering to American venture capital firms and tech start-ups, encountered financial distress due to its exposure to these high-risk industries [5]. A critical aspect that may have led to SVB’s downfall was its models’ failure to adequately account for potential risks associated with its substantial exposure to the technology sector.

In the aftermath of the 2008 financial crisis, American technology start-ups enjoyed a prolonged period of low interest rates and significant quantitative easing from the US Federal Reserve, granting them access to affordable investment capital [6]. This era of “easy money” reached its peak during the 2020 Covid-19 pandemic. At that time, approximately 50% of American tech start-ups banked with SVB [7], leading to a staggering increase in deposits from $61 billion to $189 billion[8]. The bank chose to invest these deposits in long-term bonds, which proved profitable in the low-interest rate environment, with expectations of bond appreciation.

However, in late 2021, rising inflation in the US prompted the Federal Reserve to raise interest rates, shifting the investment landscape unfavourably for SVB, as the value of their long-term bond investments declined. By the end of 2022, SVB faced $15 billion in unrealized losses from the fall in the value of their investments [9].

Furthermore, the high-interest environment posed challenges for tech start-ups, which constituted the majority of SVB’s deposits, causing them to struggle with financing their operations. Consequently, they had to rely on their bank deposits to cover liquidity shortfalls. To accommodate these large withdrawals, SVB was forced to sell its long-term bonds at a loss, triggering fears of the bank’s insolvency. As only deposits up to $250,000 are insured by the Federal Deposit Insurance Corporation (FDIC), and 97% of SVB’s deposits exceeded this amount [10], concerns about the bank’s solvency resulted in a bank run, culminating in its eventual closure.

The inability of SVB’s risk models to adapt to the changing dynamics within the US economy and accurately assess the bank’s vulnerability due to its investments and asset makeup profoundly eventually impacted its stability and solvency.

In conclusion, the importance of model validation in the banking industry cannot be overstated. It is a critical process for ensuring that models used for decision-making are accurate, reliable, and consistent with their intended purpose. With the increasing reliance on data and analytics in the banking industry, model validation will continue to be a key area of focus for financial institutions, regulators, and consumers alike. By following best practices for model validation, including thorough testing, documentation, and validation by independent parties, banks can mitigate the risk of model errors and ensure that their models are accurate, reliable, and consistent with their intended purpose.

References

1. Financial Crisis Inquiry Commission. (2011). The Financial Crisis Inquiry Report.
2. Financial Stability Oversight Council. (2013). 2013 Annual Report.3.
3. Board of Governors of the Federal Reserve System. (2018). Consent Order for a Civil Money Penalty In the Matter of Wells Fargo Bank, N.A.
4. “Standard Bank must pay R528m penalty for weak anti-money laundering controls” by Lameez Omarjee (Fin24, 2019)
5. “Mortgages, Wine and Renovations: Silicon Valley Bank’s Deep Tech Ties” – by Erin Griffith, Mike Isaac and Sheera Frenkel (New York Times, 2023)
6. “The Lords of Easy Money — where the Fed went wrong” by Rana Foroohar ( Financial Times,2023)
7. “Silicon Valley Bank’s Collapse Will Leave A Big Hole In The Startup World” by Chris Metinko ( Crunchbase, 2023)
8. “SILICON VALLEY BANK (SVB) – WHAT HAPPENED?” by Austin Morris Associates (2023)
9. “Analysis: Silicon Valley Bank’s Bankruptcy” by Julien-Pierre Nouen, CFA and François Lavier, CFA ( Lazard Frères Gestion, 2023)
10. “Beyond saving Silicon Valley Bank’s depositors, here’s what needs to happen next” by Simon Johnson

Draft Article

https://www.linkedin.com/pulse/draft/AgFsgVoQ9jOAcwAAAYkHOMaQj7XjWR0_bf3D17K9IFvidBRFuEwtaMJEll1IwAvRJg8LWR8