Category: Validation & Testing

Sensitivity Analysis

Also known as: what-if analysis

Simply put

Sensitivity analysis is a method for studying how changes in a model's inputs or assumptions affect its outputs. By varying one or more input variables and observing the resulting change in the outcome, practitioners can see which assumptions matter most and how uncertain the result may be. It is commonly used to support decision-making and to understand where a model's conclusions are most fragile.

Formal definition

Sensitivity analysis is the study of how uncertainty or variation in the output of a mathematical model or system can be attributed to variation in its inputs, assumptions, or parameters. In practice it involves systematically perturbing one or more independent variables—holding others fixed or varying them jointly—and measuring the effect on a dependent variable or model output, typically to identify influential drivers, assess robustness to assumptions, and inform risk and decision analysis. Note that the term is used across distinct domains (for example financial modeling, statistics, and observational research), so the specific technique and interpretation vary by context; the evidence provided does not establish a single standardized methodology or a specific application within AI governance or model risk management guidance.

Why it matters

Sensitivity analysis matters because models are only as reliable as the assumptions and inputs they depend on. By systematically varying inputs and observing how outputs respond, practitioners can identify which assumptions most strongly drive a model's conclusions and where those conclusions become fragile. This is central to understanding model uncertainty rather than treating a single output as a definitive answer, and it supports more defensible decision-making by making the influence of key drivers explicit.

In a model risk management context, sensitivity analysis is commonly used as one technique among several to assess robustness—that is, how stable a model's outputs are when its inputs or assumptions change. It can help surface concentrations of risk tied to particular variables, reveal where small input errors could produce disproportionately large output changes, and inform judgments about how much confidence to place in a result. It should be understood as a tool that helps characterize and manage uncertainty, not one that eliminates it.

The term is used across distinct domains, including financial modeling, statistics, and observational research, and the specific technique and interpretation vary by context. The evidence available here does not establish a single standardized methodology or a specific prescribed application within AI governance or model risk management guidance, so practitioners should be cautious about assuming that any one approach is authoritative across all settings.

Who it's relevant to

Model risk managers and validators

Sensitivity analysis is one technique that can be used to assess how robust a model's outputs are to changes in its inputs and assumptions, helping surface which drivers most influence results and where conclusions may be fragile. Practitioners should treat it as a means of characterizing uncertainty rather than a guarantee of model reliability, and should recognize that the evidence here does not tie the method to a specific standardized model risk management requirement.

Financial modelers and analysts

In financial modeling, sensitivity analysis is commonly applied to examine how different values of independent variables affect a specific dependent variable, supporting decision-making and the identification of opportunities and risks. It helps analysts understand which assumptions their conclusions rest on most heavily.

Researchers using statistical and observational methods

In statistics and observational research, sensitivity analysis is used to test how results depend on study design choices and analytic assumptions. This helps researchers gauge whether conclusions hold under alternative assumptions, though the specific techniques differ from those used in financial modeling.

Decision-makers relying on model outputs

Those who act on model results benefit from sensitivity analysis because it makes explicit how outcomes change under different assumptions, informing risk-aware decisions. It can help mitigate risk and identify opportunities, but it describes uncertainty rather than removing it.

Inside Sensitivity Analysis

Input Perturbation

The core mechanism of sensitivity analysis: systematically varying one or more model inputs, parameters, or assumptions and observing the resulting change in outputs to understand how responsive the model is to those changes.

Local vs. Global Analysis

Local sensitivity analysis examines model response to small changes around a specific point or baseline, while global sensitivity analysis explores behavior across the full plausible range of inputs. The two answer different questions and are typically not interchangeable.

Stress and Scenario Testing Overlap

In many model risk management contexts, sensitivity analysis is applied to test model behavior under extreme or adverse input values. This overlaps with, but is commonly distinguished from, formal stress testing, which is often oriented toward specific hypothetical adverse conditions.

Assumption and Parameter Testing

Evaluation of how sensitive outputs are to modeling assumptions, calibrated parameters, and estimated coefficients, helping identify which drivers most influence results and where estimation uncertainty carries the greatest consequence.

Role Within Validation

Sensitivity analysis is frequently used as one technique within model validation activities to probe robustness and stability, contributing evidence about model behavior rather than serving as a standalone conclusion about model soundness.

Common questions

Answers to the questions practitioners most commonly ask about Sensitivity Analysis.

Is sensitivity analysis the same thing as stress testing?

No, though they are related and often used together. Sensitivity analysis typically examines how a model's outputs respond to changes in one or a limited set of inputs, parameters, or assumptions, often varied individually to isolate their influence. Stress testing, as commonly practiced, evaluates model or portfolio behavior under severe but plausible scenarios that may involve many variables shifting simultaneously. Treating the two as interchangeable is a frequent error; sensitivity analysis is often a component that informs the design of stress scenarios rather than a substitute for them.

Does a sensitivity analysis showing stable outputs mean the model is validated or low risk?

Not on its own. Sensitivity analysis is one technique that can support model validation, but it does not by itself constitute validation, which in many frameworks encompasses a broader assessment of conceptual soundness, data quality, outcomes analysis, and ongoing monitoring. Stable outputs across tested input ranges also do not eliminate model risk; they address only the dimensions and ranges actually examined. Untested inputs, unexamined interactions, or conditions outside the tested range may still produce material risk. Sensitivity analysis reduces uncertainty about specific behaviors rather than certifying overall model reliability.

Which inputs or assumptions should be prioritized when scoping a sensitivity analysis?

Practitioners commonly prioritize inputs and assumptions that are both influential on outputs and subject to meaningful uncertainty, such as key estimated parameters, judgmental overlays, or data elements known to be volatile or of lower quality. The scope typically reflects the model's use and materiality; a model driving high-stakes decisions may warrant broader and more granular sensitivity testing. Documenting the rationale for what was included and, importantly, what was excluded helps reviewers understand the coverage and limitations of the analysis.

How should the results of a sensitivity analysis be documented and reported?

Documentation generally records the inputs or assumptions varied, the ranges and increments used, the resulting output changes, and the interpretation of those results, including any thresholds that would trigger further review. Clearly stating what was not tested is as important as reporting what was, so that residual uncertainty is transparent. In organizations structured around lines of defense, results are often prepared or reviewed by model developers or owners and then examined independently, for example by a validation or second-line function, though specific arrangements vary by organization.

How often should sensitivity analysis be repeated over a model's lifecycle?

There is no single required cadence across all contexts. Many organizations tie the frequency to model materiality, the rate of change in underlying data or conditions, and internal policy or applicable guidance. Sensitivity analysis is often revisited at scheduled periodic reviews and also on a triggered basis, for example after significant changes to the model, its inputs, or the operating environment, or when performance monitoring suggests the model may be behaving unexpectedly. The appropriate interval is a governance and risk-based decision rather than a fixed universal rule.

What are the practical limitations to keep in mind when relying on sensitivity analysis?

Sensitivity analysis is bounded by the inputs, ranges, and scenarios actually tested, so results should not be generalized beyond that coverage. When inputs are varied one at a time, the approach may miss interactions among variables that occur when several change together. Conclusions also depend on the model itself being appropriate; a sensitivity analysis reflects how a given model responds, not whether the model's underlying assumptions are correct. Because of these limitations, it is typically used alongside other techniques rather than as a standalone basis for conclusions about model risk.

Common misconceptions

Sensitivity analysis and stress testing are the same activity.

While they overlap and both involve varying conditions, they are commonly treated as distinct. Sensitivity analysis typically focuses on how outputs respond to changes in individual inputs, parameters, or assumptions, whereas stress testing is often oriented toward the effect of specific adverse or extreme scenarios. Practitioners in many frameworks keep the two conceptually separate.

A low-sensitivity result means the model is validated or low risk.

Sensitivity analysis is generally one input among several within validation; it provides evidence about robustness and stability but does not by itself confirm that a model is conceptually sound, fit for purpose, or low risk. It does not eliminate model risk, and interpreting it as a validation verdict conflates a single technique with the broader validation process.

Testing each input separately captures a model's full sensitivity.

Varying one input at a time (a local approach) can miss interactions and joint effects among inputs that global approaches are designed to explore. Relying only on one-at-a-time testing may understate how the model behaves across the plausible range of combined inputs.

Best practices

Clearly state whether the analysis is local or global, and match the approach to the question being asked rather than defaulting to one-at-a-time input variation.

Document the baseline point, the range over which inputs are varied, and the rationale for those ranges so results are reproducible and interpretable.

Where inputs may interact, consider joint variation rather than relying solely on perturbing one input at a time.

Treat sensitivity analysis as one component of validation evidence, and avoid presenting favorable results as confirmation that a model is sound or low risk.

Distinguish sensitivity analysis from stress or scenario testing in documentation, describing how each is used and where they overlap without collapsing the distinction.

Focus attention on the assumptions and parameters carrying the greatest estimation uncertainty, and record how their sensitivity informs monitoring and control decisions.