Skip to main content
Bureau of AI
Category: Monitoring & Drift

Model Monitoring Plan

Also known as: ML Model Monitoring Plan, Model Performance Monitoring Plan, Ongoing Monitoring Plan
Simply put

A model monitoring plan sets out how an organization will keep track of a deployed model's behavior once it is running in the real world, rather than assuming it will continue to work as it did at launch. It typically describes what will be measured, how often, and what to do if the model's outputs or performance start to slip. The goal is to catch problems early so the model can be reviewed, corrected, or retired before it causes harm.

Formal definition

A model monitoring plan is a documented specification of the ongoing surveillance activities applied to a model after deployment into production. As commonly described in the evidence, model monitoring is the lifecycle step that tracks model performance in production and evaluates model outcomes against expected behavior; a monitoring plan formalizes the metrics, thresholds, monitoring frequency, and issues to be detected (such as concept drift or degradation in functional performance). It is important to distinguish the ongoing monitoring described here (detecting changes in performance over time in live use) from initial model validation (assessing whether a model is conceptually sound and fit for purpose before or independently of deployment); monitoring detects deterioration but does not by itself re-establish model soundness. The evidence provided is drawn from general machine-learning operational sources and does not address how a monitoring plan should be structured under any specific regulatory or supervisory framework; sector-specific expectations (for example, in banking model risk management) may impose additional documentation, governance, and independent-review requirements not reflected here. A monitoring plan is a control that helps identify and manage risk from model performance changes; it reduces rather than eliminates that risk.

Why it matters

Models can appear to work well at launch and then quietly deteriorate as the world they operate in shifts. A model monitoring plan matters because it converts the assumption that a deployed model will keep performing into an explicit, documented commitment to check whether it actually does. Without such a plan, performance changes—including concept drift and degradation in functional performance—can go unnoticed until they surface as incorrect decisions, poor outcomes, or downstream harm. Monitoring is commonly described as the final step in the machine learning lifecycle: the point at which a model's real-world outcomes are compared against expected behavior.

A monitoring plan also supports accountability. By specifying in advance what will be measured, how often, and what response is triggered when metrics slip, an organization creates a basis for timely review, correction, or retirement of a model. This makes model risk management more disciplined and less reactive, and it gives second-line and third-line reviewers something concrete to assess.

It is important to be realistic about what a monitoring plan does and does not achieve. Monitoring detects deterioration in live performance, but it does not by itself re-establish whether a model is conceptually sound or fit for purpose—that is the role of validation, which is a distinct activity. A monitoring plan is a control that helps identify and manage risk from performance changes; it reduces rather than eliminates that risk. The evidence here is drawn from general machine-learning operational sources and does not describe how a monitoring plan should be structured under any specific regulatory or supervisory regime.

Who it's relevant to

Model risk managers
Model risk managers rely on monitoring plans to define, in advance, how deployed models will be tracked and what triggers a review. The plan gives them an evidence base for identifying and managing risk arising from performance changes over time, while keeping that activity distinct from initial validation of model soundness.
Data scientists and ML engineers
Those who build and operate models use the monitoring plan to specify which metrics to track, at what frequency, and what thresholds indicate problems such as concept drift or degradation in functional performance. It translates operational monitoring practice into a documented, repeatable specification for production models.
Model validators and independent reviewers
Validators and second- or third-line reviewers use the monitoring plan to assess whether ongoing surveillance is adequate and whether detected deterioration would be caught and escalated. They also confirm that monitoring is not being treated as a substitute for validation, which addresses conceptual soundness rather than live performance drift.
Auditors and compliance professionals
Auditors and compliance staff examine whether a documented monitoring plan exists, whether it defines measurable thresholds and response actions, and whether it is being followed in practice. Because sector-specific requirements may add documentation and governance expectations beyond general operational sources, these professionals should confirm what applies in their own regulatory context.

Inside Model Monitoring Plan

Monitoring Scope and Objectives
A statement of which model or models the plan covers, the risks it is intended to detect, and the objectives of ongoing monitoring. This typically distinguishes monitoring aimed at model performance degradation from monitoring aimed at broader model risk, and clarifies what is out of scope.
Performance Metrics and Thresholds
The quantitative and qualitative measures used to assess whether a model continues to perform as intended, together with predefined thresholds or tolerance bands that, when breached, trigger review or action. Metric selection commonly depends on model type and use context.
Input and Data Drift Monitoring
Procedures for tracking changes in input data distributions, population characteristics, and data quality over time. This is typically distinguished from output or performance monitoring, since input changes can precede measurable performance degradation.
Monitoring Frequency and Cadence
The schedule on which monitoring activities are performed, which in many frameworks is calibrated to the model's assessed risk, materiality, and rate of change in its operating environment.
Roles and Responsibilities
Assignment of accountability for monitoring activities, commonly mapped across the first, second, and third lines of defense so that ownership, independent oversight, and assurance functions are distinguished rather than blurred.
Escalation and Response Procedures
Defined actions to be taken when thresholds are breached or anomalies are detected, including escalation paths, remediation options, and criteria that may trigger model recalibration, revalidation, or decommissioning.
Documentation and Reporting
Requirements for recording monitoring results, decisions, and exceptions, and for reporting outcomes to relevant governance bodies or oversight functions to support accountability and auditability.

Common questions

Answers to the questions practitioners most commonly ask about Model Monitoring Plan.

Is a model monitoring plan the same thing as model validation?
No. Validation and ongoing monitoring are distinct activities, though they are related and often described together in frameworks such as SR 11-7. Validation is typically a point-in-time (or periodic) independent assessment of whether a model is conceptually sound and fit for its intended use. Monitoring is the continuing process of observing a model's behavior in production over time. A monitoring plan does not replace validation, and finding no issues in monitoring does not substitute for the independent challenge that validation is intended to provide. The two are commonly treated as complementary components of a broader model risk management program.
Does having a monitoring plan mean the model's risk has been eliminated?
No. Monitoring is a control that helps detect and manage risk, not one that removes it. A monitoring plan increases the likelihood that problems such as performance degradation or shifts in input data are identified promptly, but it does not prevent those problems from occurring, and undetected or novel failure modes remain possible. In risk terms, monitoring contributes to reducing residual risk relative to inherent risk; it does not drive risk to zero. The value of a plan depends on whether the chosen metrics, thresholds, and escalation paths are appropriate and whether findings are acted upon.
What should a model monitoring plan typically specify?
A monitoring plan commonly specifies what will be monitored (for example, performance metrics, input data characteristics, and output distributions), how frequently, against what thresholds or expected ranges, and who is responsible for review. Many plans also define escalation procedures for when thresholds are breached, roles across the lines of defense, and the conditions that would trigger revalidation, recalibration, or retirement. The specific contents vary by organization, model risk rating, and applicable framework, so this list should be treated as illustrative rather than a fixed checklist.
How is monitoring frequency usually determined?
Monitoring frequency is often calibrated to the model's assessed risk and to how quickly its performance or inputs are expected to change. Higher-risk or more volatile models are typically monitored more frequently than lower-risk or stable ones. Some organizations distinguish between routine metric tracking that may run continuously or at short intervals and deeper periodic reviews conducted less often. There is no single universally required cadence; the appropriate frequency depends on the model's use, materiality, and the surrounding governance expectations.
What is the difference between monitoring for model performance degradation and monitoring for data drift?
These are related but distinct signals. Performance degradation refers to a decline in how well the model achieves its intended outcomes, which generally requires outcome or ground-truth data to measure. Data drift refers to changes in the distribution of inputs (or the relationship between inputs and outputs) relative to what the model was built on, which can sometimes be detected before outcomes are available. Drift may or may not lead to degraded performance, and degradation can occur for reasons other than drift, so many monitoring plans track both rather than treating one as a proxy for the other.
How do the lines of defense typically relate to executing a monitoring plan?
Responsibilities for monitoring are commonly distributed across the lines of defense, though the exact allocation varies by organization. In many arrangements, the first line (model owners or developers) performs day-to-day monitoring and initial response, the second line (an independent risk or model risk function) provides oversight, challenge, and review of monitoring results, and the third line (internal audit) provides independent assurance that the monitoring process is designed and operating as intended. A monitoring plan often documents these roles so that ownership, review, and escalation are clear rather than assumed.

Common misconceptions

A model monitoring plan is the same as model validation.
Validation and ongoing monitoring are distinct activities. Validation typically assesses whether a model is conceptually sound and fit for purpose at a point in time, while a monitoring plan governs the continuous, post-deployment tracking of a model's behavior. Monitoring may inform when revalidation is needed but does not replace it.
Monitoring only needs to track model performance metrics.
Performance degradation is one concern, but monitoring commonly also covers input and data drift, changes in the operating environment, and broader model risk considerations. Focusing solely on output performance can miss upstream changes that signal emerging risk.
A monitoring plan eliminates model risk.
Monitoring is a control that helps reduce and manage risk by enabling earlier detection and response; it does not eliminate risk. Residual risk typically remains even with an effective monitoring plan in place.

Best practices

Calibrate monitoring frequency and depth to the assessed risk and materiality of each model, rather than applying a uniform cadence across all models.
Define thresholds and tolerance bands in advance, along with the specific escalation and response actions triggered when they are breached.
Monitor input and data drift alongside output performance, so that upstream changes can be detected before they materialize as performance degradation.
Assign clear ownership across the first, second, and third lines of defense, keeping operational monitoring, independent oversight, and assurance distinct.
Document monitoring results, exceptions, and decisions consistently, and report outcomes to the relevant governance bodies to support accountability and auditability.
Establish criteria that connect monitoring findings to downstream actions such as recalibration, revalidation, or decommissioning, so that detection leads to defined responses.