Getting model validation right

Why model validation matters more than ever

In today’s insurance landscape, models are more than technical tools. They have become strategic assets that underpin key decisions in pricing, reserving, capital planning, and climate risk. Actuaries now work with increasingly complex models that play a central role in shaping business outcomes. With that complexity comes greater responsibility.

Over the past decade, technological advances have turbocharged model development. We’ve seen an explosion in both the volume and intricacy of models used across the industry. Yet, this evolution brings with it a heightened exposure to model risk, that is the risk that a model may mislead rather than inform due to poor design, flawed assumptions, or misinterpretation of outputs.

To safeguard against this, a robust model risk management framework is essential, with model validation at its core. Model validation is not just about regulatory compliance. It’s a professional obligation that ensures models remain fit for purpose, reliable, and aligned with evolving business and regulatory environments.

The urgency around validation is growing. In Ireland, scrutiny of pricing models has intensified following the Central Bank of Ireland’s (CBI) work on differential pricing. Internationally, the rise of climate-focused modelling, spurred by developments like the Corporate Sustainability Reporting Directive (CSRD), is drawing actuaries into new modelling territory. These developments underscore the need for rigorous, ongoing validation to maintain trust in the models we build and the decisions they inform.

The strategic role of model validation

At its core, model validation is an independent, expert assessment of a model’s design, assumptions, calculations, and outputs. When carried out properly, it helps build confidence in the model’s reliability and ensures it continues to serve its intended purpose in a dynamic business environment.

For actuaries, validation plays a key role in guarding against model drift. This is the gradual erosion of accuracy as assumptions age, data evolves, or business practices shift. A model that was robust five years ago may now be misaligned with reality, especially in fast-moving areas like pricing, capital modelling, or climate risk.

Validation should not be viewed as a one-off task. It needs to be part of an ongoing process. Significant model updates, changes in the underlying input data, or shifts in regulatory expectations should all trigger a fresh review. Under Solvency II, regular validation is mandated for all internal models, with the frequency depending on the model’s complexity, materiality, and risk exposure.

Ultimately, validation is about ensuring that models remain robust, transparent, and fit for decision-making. In actuarial work, the numbers must stand up to scrutiny and that means the models behind them must too..

Validation in practice: a deeper dive for actuaries

Model validation is not a box ticking exercise. It is a critical safeguard that demands a deep understanding of both the technical detail and the business context. In the Irish market, where regulatory expectations continue to evolve, validation must be both thorough and forward-looking.

A robust validation framework should challenge every stage of the model lifecycle: inputs, calculations, assumptions, controls, and outputs. Each of these components carries its own risks, and overlooking any one of them can undermine the model’s integrity. 

Model inputs

The starting point for any thorough validation is the model’s inputs. These must be accurate, complete, and appropriate for the model’s purpose. If the underlying data is flawed or poorly understood, even the most sophisticated assumptions and calculations will produce unreliable results. When reviewing inputs, it is important to ensure that:

• The input data for the model is accurate and complete. This involves verifying that the model has working checks on numeric inputs. These checks should ensure that the data used was correct and without gaps. This is done by comparing the inputs with independent source data and data used in previous reporting cycles.
• The input data for the model is appropriate and reasonable. This means confirming that the data source aligns with the model’s intended use and the underlying assumptions. It is also useful to compare input values to market data or regulatory benchmarks to assess whether they fall within a reasonable range.
• The assumptions which underpin the model are reasonable. The evolving economic and regulatory climate calls for continuous and meticulous validation of model assumptions. The accuracy of the model depends on its assumptions and therefore it is vital to check during each validation that they remain reasonable.

Model calculation

Validating a model’s calculations is a crucial part of the overall validation process. This typically involves comparing the model’s outputs to those produced by an independently constructed model. The goal is to confirm that the calculations are functioning as intended and delivering reliable results.

The selection of an appropriate independent model depends on the complexity and materiality of the model being reviewed. In some cases, a model may be built from first principles specifically for the validation exercise. While this approach offers the highest degree of assurance, it's important to note it demands considerable time and expertise.

Alternatively, existing challenger models, which have been developed for separate purposes and often residing in different software environments, may be useful. Provided the challenger's process is sufficiently comparable to the model under review, they can be effectively contrasted to verify calculations. 

Finally, a simplified model can also be built to reflect the key features of the model under validation. While it may not match the sophistication of more complex challenger models and may produce wider deviations, these differences should be carefully considered and explained as part of the comparison process.

Model results

As with inputs and calculations, it is important to validate the outputs of a model. There are several well-established techniques that can be used to support this stage of the validation process:

• Stress testing of inputs: This involves verifying that even minor alterations to input variables do not lead to disproportionate, or indeed unexpected, changes in the model’s output. 
• Extreme value testing: It's crucial to assess how a model performs under extreme scenarios. Experience shows that models, given their mathematical underpinnings, can sometimes produce unreasonable or nonsensical results when input values fall outside their expected range. 
• Sensitivity testing of assumptions: By adjusting one assumption at a time and observing the impact on results, actuaries can identify which assumptions are most influential. If small changes lead to large shifts in output, that assumption may need closer scrutiny.
• Scenario testing of assumptions: Building on sensitivity analysis, this involves simultaneously varying multiple assumptions to replicate plausible future scenarios. 
• Back testing: Here, the model is run using historical input data for which the actual real-world outcomes are already known. The model's output is then rigorously compared against these observed results. 

Governance and documentation

A thorough model validation exercise should also include a review of the model’s supporting documentation and the overall governance framework in place. 

• Documentation: The model documentation should be clear and detailed enough to allow an independent and suitably experienced person to fully understand the model’s purpose, structure, and functionality. It should also enable them to replicate key processes and calculations with ease. 
• Governance: Validation should confirm that there is an appropriate governance structure in place, with clear ownership of the model and evidence of regular and proportionate review. The frequency and depth of validation activities should be aligned with the model’s complexity and the level of risk it introduces to the business.

Solvency II and the regulatory landscape

Solvency II mandates regular model validation for internal models. It also details how model validation should be performed and what it should entail. The sections below summarise some of the key regulatory requirements under Solvency II.

Frequency and scope of validation

Solvency II requires internal models to undergo regular validation. A comprehensive validation should assess the model’s methodology, the calibration of parameters, the appropriateness of the model for the relevant lines of business, the model’s operation, and the associated qualitative considerations.

While these elements do not need to be reviewed all at once, they must each be validated on a regular basis. A sound validation cycle will test the above with regular and risk-based frequency alongside ad-hoc validations based on certain triggers.

Independence

Model validation must be independent of both model development and day-to-day operation. Best practice suggests considering the following areas:

• Organisational structure: Model validation typically sits within the risk function, which may also oversee model development. In such cases, safeguards are needed to manage potential conflicts of interest and preserve independence. 
• Reporting lines: Effective communication is critical to maintain independence. Validation reports should be shared with all relevant stakeholders and the model validation team should have the opportunity to present and discuss their findings at the appropriate governance forums. Senior management should be informed of validation outcomes on a predefined and regular basis.
• Independence of mind: Those performing validation should maintain professional scepticism throughout the process.
• Testing plan: Solvency II outlines specific testing requirements. In practice, some of these tests may be carried out by the development team, with validators relying on the results. To protect the independence of the process, a clear test plan should be agreed in advance. This ensures that the validation is not shaped by development choices and allows for an objective review.
• Testing execution: Validators should ensure that tests are conducted in line with the agreed plan, that the tests are reproducible, and that any deviations are appropriately explained. The validation team should form its own conclusions based on the evidence, separate from the views of the developers. 
• External personnel: Independence requirements also apply to external experts involved in the validation process. Firms should ensure that external parties are free from conflicts of interest and that their engagement terms allow them to carry out the validation without constraint or undue influence.

The emergence of artificial intelligence

As technology continues to evolve, so too does the world of modelling and model validation. One of the most significant developments in recent years is the growing use of artificial intelligence (AI). As AI tools become more sophisticated, firms are placing increased emphasis on building internal AI literacy and technical competence.

At a European level, the EU has made clear its ambition to be a global leader in the safe and ethical use of AI, indicating that the role of AI in financial services will only expand in the years ahead. This evolution presents new challenges for insurers, not least in terms of regulation and governance.

As AI begins to play a greater role in internal modelling, actuaries and risk professionals must remain mindful of how these tools are embedded into their modelling frameworks. Without proper transparency and oversight, AI-enabled models can become opaque, or so-called “black boxes,” where decisions are generated without clear visibility of the underlying processes.

In other industries situations like this have led to models unintentionally unfairly discriminating or employing “social scoring” systems. These risks underscore the continued importance of strong model validation practices. 

Safeguarding the future with strong model validation

Model validation plays a central role in safeguarding the reliability and accuracy of outputs used to support decision-making across the (re)insurance sector. As technological innovation continues to shape how models are developed and deployed, the importance of a robust validation framework is more significant than ever. 

This article has outlined the key components of a comprehensive validation process, including the review of model inputs, assumptions, calculations, and outputs. Under Solvency II, insurers are required to carry out validation on a regular and risk-based basis, ensuring that models remain appropriate, well understood, and aligned with evolving regulatory standards.

Looking ahead, the integration of Artificial Intelligence into actuarial and risk models will bring both opportunities and new challenges. Model validators will need to keep pace with these developments to ensure that models remain transparent, explainable, and fit for purpose in a more complex modelling environment.

By maintaining a strong focus on validation, actuaries and insurers can continue to support sound decision-making and uphold confidence in the models that underpin the financial strength and resilience of their businesses.

Insurance expertise

Navigate risk, compliance and growth with confidence

Explore our insurance services