Category: Model Lifecycle & MLOps

Model Development

Also known as: Model Building, Model Development Process

Simply put

Model development is the iterative process of creating, training, and refining machine learning or AI models so they can extract insights from data and address a defined problem. It typically involves building and testing multiple candidate models until one meets the desired criteria. In many frameworks it is described as one stage within a broader model lifecycle that also includes deployment and ongoing maintenance.

Formal definition

Model development refers to the iterative process of deriving, testing, and building successive candidate models until a model meeting specified criteria is produced. It is commonly organized into structured phases; one widely cited breakdown identifies six phases: business understanding, data understanding, data preparation, modelling, evaluation, and deployment. As commonly defined, model development is the construction-focused portion of the model lifecycle and should be distinguished from downstream activities such as independent validation, deployment, and monitoring, which in many governance and model risk management frameworks are treated as separate, often independently owned, functions.

Why it matters

Model development is the stage where a model's core capabilities and limitations are first established, which means many of the risks that later governance and oversight functions must manage originate here. The choices made during business understanding, data preparation, and modelling shape a model's performance characteristics, its data dependencies, and its potential weaknesses. Because it is an iterative process of building and testing successive candidate models, the rigor and documentation applied during development strongly influences how effectively downstream activities such as validation and monitoring can be carried out.

A key reason model development matters to governance and model risk management professionals is the principle of separation of duties. In many governance and model risk management frameworks, development is treated as the construction-focused portion of the lifecycle and is deliberately distinguished from independent validation, deployment, and monitoring, which are often owned by different functions. Conflating development with validation is a common error: the team that builds a model is generally not positioned to provide the independent challenge that validation is intended to supply. Clear boundaries around what belongs to development help preserve that independence.

It is worth noting that the significance and required rigor of model development can vary by sector and context. Practices and expectations differ between, for example, banking model risk environments and general enterprise AI settings, and the specific controls applied depend on the applicable framework and the model's intended use. Development is a measure that shapes and can reduce risk, but it does not by itself eliminate the need for the independent oversight that follows.

Who it's relevant to

Data scientists and model developers

As the practitioners who build, train, and refine candidate models across the development phases, this group is directly responsible for the iterative construction work. Their documentation of data understanding, data preparation, and evaluation choices provides the basis on which later independent review depends.

Model risk managers

In many model risk management frameworks, model development is treated as a distinct stage that must be separated from independent validation. Model risk managers are concerned with ensuring that development activities are appropriately scoped, documented, and kept distinct from the downstream oversight functions that provide independent challenge.

Model validators and reviewers

Because development and validation are commonly distinguished as separate, often independently owned functions, validators rely on the quality and completeness of the development record to perform their assessments. The rigor applied during development directly affects how effectively validation can be conducted.

AI governance and compliance professionals

Those responsible for organizational oversight of AI systems use the boundaries of the model lifecycle — including where development ends and validation, deployment, and monitoring begin — to assign accountability and structure controls. Understanding how development fits within the broader lifecycle helps them design policies that preserve separation of duties.

Inside Model Development

Conceptual Design and Model Purpose

The articulation of the model's intended use, business or regulatory objective, target population, and the theoretical or empirical rationale supporting the chosen methodology. In many model risk frameworks, documenting intended use is treated as foundational because model risk is assessed relative to how and where a model is applied.

Data Sourcing and Preparation

The selection, collection, cleaning, transformation, and representativeness assessment of input data. This typically includes documenting data lineage, handling of missing or anomalous values, and assessing whether training data is appropriate for the model's intended population and use.

Methodology and Model Specification

The choice of algorithm or statistical technique, feature selection, hyperparameters, and any assumptions embedded in the approach. Development should record why a given method was selected and what alternatives were considered, so that limitations are understood downstream.

Model Estimation and Training

The process of fitting or training the model on prepared data, including the treatment of training, validation, and test partitions. Note that internal developer testing during this stage is distinct from independent model validation, which is typically performed separately as part of second-line review.

Developer Testing and Performance Assessment

Testing conducted by the development team to evaluate whether the model meets its objectives, such as accuracy, discrimination, or calibration measures. This is a check on model performance and should not be confused with independent validation or with ongoing monitoring for performance degradation after deployment.

Documentation and Assumptions Log

A record of design decisions, data limitations, known weaknesses, and boundary conditions of appropriate use. In many frameworks, thorough development documentation is expected so that validators, reviewers, and auditors can reconstruct and challenge the development choices.

Limitations and Known Weaknesses

An explicit statement of what the model is not designed to do, populations or conditions where it may perform poorly, and the residual uncertainty that remains after development. Identifying limitations at development supports later risk measurement without implying the model is risk-free.

Common questions

Answers to the questions practitioners most commonly ask about Model Development.

Is model development the same as model validation?

No. Model development is typically the process of designing, building, and training a model, usually carried out by the first line of defense or a dedicated development team. Model validation, in many frameworks (such as the model risk management approach described in SR 11-7), is a distinct and often independent activity intended to provide effective challenge and assess whether a model is sound and fit for its intended purpose. Conflating the two undermines the independence that validation is generally expected to provide. The line between them can vary by organization and sector, so roles should be documented rather than assumed.

Does thorough model development remove the need for ongoing model risk management?

No. Model development addresses risks at the point of building the model, but it does not eliminate model risk. Even a well-developed model can degrade over time, be applied outside its intended use, or interact with changing data and conditions. Model risk management, as commonly framed, spans the full lifecycle including ongoing monitoring, periodic review, and controls after deployment. Development is one input to managing risk, not a substitute for it, and no development practice should be described as eliminating risk.

What documentation is typically expected during model development?

In many model risk management frameworks, development documentation is expected to be sufficient for a knowledgeable third party to understand and, in principle, reconstruct the model. This commonly includes the model's intended purpose and scope, data sources and treatment, methodology and assumptions, alternatives considered, testing performed, and known limitations. The specific expectations vary by organization, sector, and applicable guidance or standards, so teams should confirm requirements against their own governance policies rather than assume a universal checklist.

How should assumptions and limitations be handled during development?

Assumptions and limitations are generally expected to be identified, documented, and communicated to those who validate, approve, and use the model. Recording them during development supports later effective challenge and helps users understand where the model may be unreliable or applied outside its tested range. Practices differ across organizations, but leaving assumptions implicit is a frequent source of downstream error, particularly when a model is reused for a purpose it was not developed for.

How does model development interact with the lines of defense?

Development is typically performed within the first line of defense, which owns and builds the model. Second-line functions, such as an independent validation or model risk management group, commonly provide challenge and oversight, while third-line functions such as internal audit may assess whether the overall process operates as intended. The exact allocation of responsibilities varies by organization and sector, so the division should be established in governance policy rather than inferred from a generic model.

When should development-stage testing occur relative to deployment?

Testing is generally an integral part of development rather than a step deferred until after a model is built, and it is commonly distinguished from later independent validation. Development-stage testing typically examines whether the model behaves as designed under expected and stressed conditions and within its intended scope. The scope, depth, and timing of such testing depend on the model's risk profile and applicable internal or regulatory expectations, so teams should align testing to their own risk-tiering and governance requirements.

Common misconceptions

Model development includes model validation, so a well-tested model does not need separate independent review.

Developer testing and independent validation are commonly treated as distinct activities. In frameworks influenced by guidance such as SR 11-7, validation is typically performed by parties independent of development to provide effective challenge. Testing performed by developers assesses performance but does not substitute for independent validation.

A model that performs well at development will continue to perform well, so development effectively controls ongoing model risk.

Strong development-stage performance does not eliminate model risk. Model risk is broader than model performance and includes the possibility of incorrect or misused output. Separately, models can experience performance degradation over time as data or conditions shift, which is addressed through ongoing monitoring rather than development alone.

Model development and AI governance are the same activity.

Development is a technical lifecycle stage focused on building and testing a model. AI governance refers to the organizational structures, policies, accountability, and oversight surrounding AI systems. They overlap where governance sets requirements and controls for how development is conducted and documented, but they are not interchangeable.

Best practices

Document the model's intended use, target population, assumptions, and known limitations at the outset, so downstream validators and auditors can reconstruct and challenge development choices.

Assess and record data representativeness and lineage, including how missing, anomalous, or unrepresentative data was handled relative to the intended population.

Maintain a clear separation between developer testing and independent validation, and avoid presenting internal testing results as a substitute for independent effective challenge.

Explicitly state boundary conditions and residual weaknesses rather than implying the model is risk-free, so that risk measurement and monitoring can be scoped appropriately.

Coordinate development with applicable governance requirements and controls, treating governance policies as inputs that shape documentation and approval expectations without collapsing the two functions.

Design development artifacts to support later monitoring by capturing baseline performance metrics and assumptions that can be revisited to detect performance degradation after deployment.