Category: Fairness & Bias

Proxy Discrimination

Simply put

Proxy discrimination happens when a decision system uses a seemingly neutral characteristic as a stand-in for a protected trait (such as race, sex, or disability), producing discriminatory outcomes even without directly using the protected trait itself. This can occur unintentionally, and commentators have argued that AI and big data may increase this risk because such systems can identify subtle correlations that act as proxies. The exact meaning of the term is contested, and precise definitions vary across the literature.

Formal definition

Proxy discrimination refers to the use of a facially neutral variable (or combination of variables) that functions as a stand-in for a legally prohibited or protected characteristic, such that reliance on the neutral variable reproduces discrimination associated with the protected trait. In the AI and big data context, some scholarship characterizes this as potentially 'rational' but unintentional, arising when models learn correlations that serve as proxies for protected attributes even when those attributes are excluded from the inputs. As noted in the literature, there is no single agreed-upon formalization: surveys of the concept find substantial disagreement over what constitutes a proxy and what makes proxy use discriminatory, so practitioners should treat the term as one with multiple contested definitions rather than a settled technical standard. Proposed mitigation strategies discussed in this literature include restricting the use of certain non-protected variables that act as proxies; the applicability and legal status of any such measure depends on jurisdiction and legal context, which is out of scope for this definition.

Why it matters

Proxy discrimination matters because excluding protected characteristics from a model's inputs does not, by itself, guarantee non-discriminatory outcomes. A system can reproduce discrimination associated with a protected trait by relying on facially neutral variables that correlate with that trait. Some scholarship argues that AI and big data are game changers for this risk, because such systems can surface subtle correlations that function as proxies even when the protected attribute has been deliberately removed from the data. For compliance officers, model risk managers, and legal specialists, this means that a claim of fairness based solely on 'we didn't use race or sex' can be incomplete or misleading.

Who it's relevant to

Model risk managers and validators

Those responsible for identifying, measuring, and monitoring model risk should treat the absence of protected attributes in the input data as insufficient evidence of non-discriminatory behavior. Because seemingly neutral variables can act as proxies, validation and testing may need to examine correlations between model inputs and protected traits rather than relying on input exclusion alone. Note that the specific techniques and thresholds are not settled and vary by context.

Compliance officers and fairness specialists

Professionals assessing AI fairness and non-discrimination should be aware that proxy discrimination can occur unintentionally and that the term itself is contested, with multiple definitions across the literature. This affects how discrimination claims are evaluated and documented, and it means fairness assessments should account for indirect pathways through which protected characteristics can influence outcomes.

Legal and policy professionals

Legal specialists should recognize that whether a given proxy use is prohibited, and what mitigation is permitted, depends on jurisdiction and legal context. The literature offers proposed strategies such as restricting non-protected variables that act as proxies, but the legal status of these measures is context-dependent and out of scope for this definition. The lack of a single agreed formalization of 'proxy' is itself relevant to how disputes may be framed.

Data scientists and model developers

Developers who remove protected attributes to promote fairness should understand that models may still learn proxy relationships from remaining features, particularly given the ability of AI and big data methods to identify subtle correlations. This shifts responsibility from simply excluding sensitive fields toward examining how retained variables relate to protected traits, while recognizing that no single formal definition of proxy discrimination governs this analysis.

Inside Proxy Discrimination

Proxy Variable

A feature that is not itself a protected or prohibited attribute (such as race, sex, or age) but that correlates with one strongly enough to function as a stand-in for it. Common examples discussed in the literature include ZIP code, certain purchasing patterns, or educational institution, though whether any specific variable acts as a proxy is context- and data-dependent.

Correlation with a Protected Attribute

The statistical relationship that links the proxy to the protected class. Proxy discrimination typically arises when a model relies on this correlation to produce outcomes that track a protected characteristic, even when that characteristic is not an explicit input.

Disparate Impact Pathway

The mechanism by which facially neutral inputs generate outcomes that disadvantage a protected group. Proxy discrimination is frequently analyzed under a disparate impact lens in jurisdictions where that concept applies, though the applicable legal standard varies by jurisdiction and sector.

Intent-Independence

The concept that proxy discrimination can occur without any deliberate design choice to discriminate; it may emerge from the data or from optimization for predictive accuracy. This distinguishes it from overt or intentional discrimination in many analytical frameworks.

Feature Interaction Effects

Cases where no single variable acts as a proxy, but combinations of otherwise innocuous features jointly reconstruct a protected attribute. This makes proxy discrimination harder to detect through single-variable inspection alone.

Common questions

Answers to the questions practitioners most commonly ask about Proxy Discrimination.

Does removing protected attributes like race or gender from a model prevent proxy discrimination?

No. This is a common misconception. Proxy discrimination arises when a model relies on features that are correlated with protected attributes, so removing the protected attributes themselves does not eliminate the risk. Variables such as postal code, purchasing patterns, or other seemingly neutral features can act as proxies that reproduce disparate outcomes even when the protected characteristic is absent from the model inputs. Detecting proxy effects typically requires analyzing outcomes and feature relationships rather than relying on omission of the protected variable alone.

Is proxy discrimination the same thing as intentional discrimination?

Not necessarily, and treating them as identical is a frequent error. Proxy discrimination can occur without any intent to discriminate, arising from statistical correlations learned by a model. In many legal and regulatory contexts, unintentional disparate impact and intentional disparate treatment are analyzed differently, though the applicable framework depends on jurisdiction and sector. The presence of a proxy effect does not by itself establish intent, and the relevant legal treatment is context-dependent and sometimes contested.

How can a team identify potential proxy variables during model development?

Teams commonly examine the statistical relationship between candidate features and protected attributes, for example by testing how strongly a feature predicts a protected characteristic. Features that are highly predictive of a protected attribute warrant closer scrutiny. This analysis is typically combined with review of feature provenance and domain reasoning about why a variable is included. The appropriate thresholds and methods vary by context, and no single test is universally accepted as definitive.

What monitoring practices help detect proxy discrimination after deployment?

Ongoing outcome monitoring across relevant subgroups is commonly used to detect disparities that may indicate proxy effects, since correlations between features and protected attributes can shift over time as populations and data change. This typically fits within broader model monitoring rather than being a one-time check at validation. The specific metrics and subgroup definitions depend on the use case and applicable requirements.

How should proxy discrimination analysis be documented for governance and audit purposes?

Documentation typically records which features were assessed for proxy effects, the methods used, the results, and the rationale for retaining or excluding variables. This supports review by second-line functions such as model validation or compliance and by third-line internal audit. Clear documentation of decisions and residual concerns helps demonstrate that proxy risk was considered, though it reduces rather than eliminates the underlying risk. Documentation expectations vary by organization and regulatory context.

Who within an organization is typically responsible for addressing proxy discrimination?

Responsibility is often distributed across lines of defense: model developers and business owners in the first line typically conduct initial feature analysis, while second-line functions such as model validation, compliance, or fairness review provide independent challenge, and third-line internal audit may assess the overall process. Legal and compliance input is frequently involved because proxy discrimination can implicate anti-discrimination obligations that differ by jurisdiction and sector. The precise allocation of roles varies by organization.

Common misconceptions

Removing protected attributes from the model inputs eliminates proxy discrimination.

Excluding variables such as race or sex does not, on its own, prevent a model from reconstructing those attributes through correlated proxies or feature interactions. So-called 'fairness through unawareness' is widely regarded as insufficient, and in some cases removing the attribute can also hinder the measurement needed to detect proxy effects.

Proxy discrimination requires discriminatory intent.

As commonly defined, proxy discrimination can arise from data patterns and model optimization without any intent to discriminate. Because of this, it is often assessed by examining outcomes and correlations rather than solely the designer's motivations, though the relevance of intent depends on the applicable legal framework.

Proxy discrimination is purely a fairness concern, separate from model risk management.

It sits at the overlap of the two areas. It raises fairness and legal-compliance questions, but it is also a source of model risk that can fall within model risk management activities such as validation and ongoing monitoring. The two disciplines address it from different angles and should not be collapsed into one.

Best practices

Test for correlation between candidate input features (individually and in combination) and protected attributes, rather than relying on the exclusion of protected attributes alone to prevent proxy effects.

Analyze model outcomes for disparities across protected groups, keeping in mind that the applicable standard for what constitutes an impermissible disparity varies by jurisdiction and sector.

Examine feature interactions, not just single variables, since combinations of neutral features can jointly reconstruct a protected characteristic.

Document the rationale for retaining features that show correlation with protected attributes, including any legitimate business justification, so decisions can be reviewed by second-line and third-line functions.

Incorporate proxy discrimination checks into ongoing monitoring, because correlations and disparate impacts can emerge or shift as data and populations change over time.

Engage compliance and legal specialists early, given that the treatment of proxy discrimination differs across legal regimes and describe controls as measures that reduce rather than eliminate the risk.