Category: Model Lifecycle & MLOps

Feature Engineering

Simply put

Feature engineering is the process of turning raw data into useful inputs that a machine learning model can work with. This can involve selecting, creating, or modifying the pieces of information (features) that the model uses to make predictions. The goal is to present data in a form that helps the model learn more effectively.

Formal definition

Feature engineering is a preprocessing step in supervised machine learning and statistical modeling that transforms raw data into a more effective set of input variables (features) through selection, creation, and modification. It typically involves manipulating raw data so that the resulting features better support model learning and predictive performance. As commonly described, it is most associated with supervised learning and statistical modeling contexts; the specific techniques applied vary by data type, model, and problem domain, and the evidence provided does not enumerate a fixed set of methods.

Why it matters

Feature engineering directly shapes the inputs a model learns from, which means it can materially influence a model's predictive performance. Because the resulting features better support model learning, decisions made during this step—which raw data to select, how to modify it, and which new features to create—are among the more consequential choices in building a supervised model. For model risk management purposes, these choices are part of the model's development record and are typically relevant to how a model is understood, documented, and later validated.

From a governance and validation standpoint, feature engineering is a point where assumptions and judgments enter a model. Because the transformation of raw data into input variables involves human decisions, it is an area where errors, unstated assumptions, or data-handling choices can propagate into downstream predictions. This makes clear documentation of how features were selected, created, and modified important for anyone responsible for reviewing or challenging a model. Note that this entry describes feature engineering as a technical practice; the evidence provided does not establish specific regulatory requirements governing it, and any such requirements would depend on jurisdiction and sector.

It is worth distinguishing feature engineering as a data-preparation practice from the broader questions of model performance and validation that surround it. Effective feature engineering may support better predictive performance, but the evidence here does not quantify that effect, and improved features do not by themselves guarantee a fit-for-purpose or well-controlled model. The techniques applied vary by data type, model, and problem domain, so the practice cannot be reduced to a single fixed procedure.

Who it's relevant to

Data scientists and model developers

Practitioners who build supervised models perform feature engineering directly, selecting, creating, and modifying input variables to support model learning. The choices they make here are part of the development record and shape downstream predictive performance.

Model validators and reviewers

Those responsible for independently reviewing or challenging a model typically need to understand how features were derived from raw data, since these transformations embed assumptions and judgments that can affect model behavior. Clear documentation of feature engineering supports effective review.

Model risk managers

Feature engineering is one point where human decisions and potential errors enter a model. Those managing model risk have an interest in ensuring these steps are documented and understood, though the evidence here does not specify particular control requirements, which would depend on jurisdiction and sector.

Auditors and compliance reviewers

Professionals examining how a model was built may look to feature engineering documentation to trace how raw data became model inputs. This supports transparency into the model's construction without, in itself, establishing that a model is fit for purpose.

Inside Feature Engineering

Feature Construction

The creation of new input variables from raw data, for example by combining, transforming, or decomposing existing fields to better represent the underlying patterns a model is intended to capture.

Feature Transformation

The application of mathematical or statistical operations such as scaling, normalization, log transforms, or encoding of categorical variables to prepare inputs for a given modeling technique.

Feature Selection

The process of identifying and retaining the subset of features that contribute meaningfully to model behavior, while removing redundant, irrelevant, or low-value inputs. This is distinct from construction, though the two are often performed iteratively.

Feature Extraction

The derivation of a smaller set of informative variables from a larger or higher-dimensional set, for example through dimensionality-reduction techniques. This differs from selection in that it typically produces new derived variables rather than choosing among existing ones.

Domain Knowledge Integration

The incorporation of subject-matter expertise to design features that reflect meaningful real-world relationships, which can affect both model performance and the interpretability of resulting features.

Documentation and Lineage

The recording of how features are derived, from which data sources, and under what assumptions. In model risk management contexts, such documentation is commonly expected to support validation and independent review, though the depth of expectation varies by framework and sector.

Common questions

Answers to the questions practitioners most commonly ask about Feature Engineering.

Is feature engineering the same as data cleaning or preprocessing?

No. While the activities overlap and are often performed in the same pipeline, they are conceptually distinct. Data cleaning and preprocessing typically address data quality issues such as missing values, formatting, and error correction. Feature engineering, as commonly defined, refers to the creation, transformation, or selection of input variables (features) intended to improve a model's ability to represent the underlying problem. Conflating the two can obscure where modeling assumptions are introduced, which matters for validation and documentation.

Does feature engineering only matter for traditional or simpler models, since modern approaches can learn features automatically?

This is a common oversimplification. Certain modeling approaches are capable of learning representations from raw inputs, which can reduce reliance on manually constructed features in some settings. However, this does not mean feature engineering is irrelevant. The choice, transformation, and provenance of inputs still affect model behavior, interpretability, and risk. Treating automatically learned representations as if they eliminate the need to understand and document inputs can create gaps in governance and validation.

How should feature engineering steps be documented for model risk purposes?

In many model risk frameworks, the transformations applied to inputs are expected to be documented in sufficient detail that an independent reviewer can understand and, where appropriate, reproduce them. This typically includes the rationale for each feature, the source data, the transformation logic, and any assumptions embedded in the process. Documentation practices vary by organization and by the sensitivity of the model, so specific expectations should be confirmed against the applicable internal policy or supervisory guidance.

What risks should be considered when engineered features are derived from sensitive or protected attributes?

Features that directly encode, or that act as proxies for, sensitive attributes can raise concerns related to bias and fairness, which are distinct concepts. The construction of such features may warrant additional scrutiny during validation and review. Because the treatment of protected attributes can carry legal and regulatory implications that differ by jurisdiction and sector, decisions about their use in features should typically involve relevant compliance and legal input rather than being handled solely as a technical modeling choice.

How does feature engineering relate to ongoing model monitoring?

Because engineered features depend on input data and assumptions that can change over time, shifts in the distribution or meaning of underlying inputs may affect features and, in turn, model behavior. Monitoring practices in many frameworks therefore extend to the stability of inputs and features, not only to output performance. Distinguishing changes attributable to feature drift from other forms of model performance degradation can help direct appropriate remediation.

Who is typically responsible for reviewing feature engineering choices within a lines-of-defense structure?

Responsibilities vary by organization, but in structures that separate first, second, and third lines of defense, the development and initial documentation of features commonly sit with the model developers or owners in the first line, while independent challenge and validation of those choices often fall to a second-line function. Audit or a third-line function may assess whether the associated controls and processes are operating as intended. The precise allocation depends on an institution's own governance framework.

Common misconceptions

Feature engineering is a purely technical, one-time step with no governance relevance.

Feature engineering choices shape a model's inputs and can materially affect its behavior, and in many model risk management frameworks the derivation and justification of features fall within the scope of validation and documentation. It is typically treated as an ongoing concern rather than a single fixed step, though specific expectations vary by framework and sector.

Removing a sensitive attribute from the feature set eliminates bias in the model.

Dropping a protected characteristic does not necessarily remove bias, because other engineered or retained features can act as proxies for that characteristic. Bias (a systematic deviation in data or outputs) and fairness (a normative property assessed against defined criteria) are distinct concepts, and feature engineering alone does not resolve either; this remains an evolving and context-dependent area.

More engineered features always improve the model.

Adding features can increase complexity, raise the risk of overfitting, and reduce interpretability without necessarily improving generalization. Improved fit on available data is not equivalent to genuine model performance, and feature selection or extraction is often used to manage this trade-off.

Best practices

Document how each feature is constructed, its data sources, and the assumptions behind it, so that the derivation can support independent validation and review.

Treat feature selection, extraction, and transformation as distinct activities and record which technique was applied and why, rather than blurring them into a single undocumented step.

Evaluate whether retained or derived features may act as proxies for sensitive attributes before relying on attribute removal as a bias-management measure.

Involve subject-matter experts to ensure engineered features reflect meaningful real-world relationships and to support downstream interpretability.

Guard against overfitting by assessing whether added features improve generalization rather than only improving fit on available data, and by pruning low-value inputs.

Maintain feature lineage over time so that changes to inputs can be traced and re-validated, recognizing that documentation expectations vary by framework and sector.