Category: Fairness & Bias

Group Fairness

Also known as: Statistical Parity

Simply put

Group fairness is a concept in machine learning concerned with ensuring that an algorithm treats different groups of people equitably rather than favoring one group over another. It typically compares outcomes across groups defined by sensitive attributes such as race or gender, requiring that protected groups be treated similarly to more advantaged groups. It is one of several competing notions of fairness and does not, on its own, guarantee that any individual is treated fairly.

Formal definition

Group fairness is a family of fairness criteria that evaluate whether a model produces comparable outcomes across groups defined by sensitive or protected attributes (for example, race or gender). In its most commonly cited form it is defined by statistical parity, under which the requirement is that protected groups be treated similarly to the advantaged group with respect to a chosen outcome measure. It is distinct from individual fairness, and the two notions can be in tension; some work has examined the conditions under which group fairness and individual fairness are compatible or incompatible. Group fairness is one approach among several for addressing algorithmic bias, and specific formalizations (and the underlying choice of protected attribute and outcome metric) vary by context and are not standardized across all frameworks.

Why it matters

Group fairness matters because automated decision systems increasingly influence high-stakes outcomes in areas such as lending, hiring, and access to services, where systematically favoring one demographic group over another can produce discriminatory effects at scale. By comparing outcomes across groups defined by sensitive attributes such as race or gender, group fairness gives practitioners a measurable way to detect and address unfavorable treatment that might otherwise remain hidden inside a model's aggregate performance metrics. For compliance officers and model risk managers, this makes group fairness a practical entry point for scrutinizing whether a model's behavior is defensible with respect to protected groups.

At the same time, group fairness is only one of several competing notions of fairness, and satisfying it does not, on its own, guarantee that any given individual is treated fairly. This limitation is important for professionals who must avoid overstating what a fairness check demonstrates. A model that achieves statistical parity across groups may still treat similar individuals inconsistently, and group fairness and individual fairness can be in tension; some work has examined the conditions under which the two notions are compatible or incompatible. Treating a passing group-fairness test as proof of overall fairness is a common and consequential error.

Because specific formalizations, the choice of protected attribute, and the outcome metric all vary by context and are not standardized across all frameworks, group fairness should be understood as a family of criteria rather than a single settled standard. Professionals should document which formalization they are applying and why, since a different but equally defensible metric could yield a different conclusion about the same model.

Who it's relevant to

Data scientists and model developers

Those building or tuning models use group fairness criteria, such as statistical parity, to test whether outcomes differ across groups defined by sensitive attributes. They are responsible for selecting the protected attribute and outcome metric, documenting those choices, and recognizing that satisfying a group fairness criterion does not guarantee individual fairness or resolve tensions between competing fairness notions.

Model risk managers and validators

Professionals responsible for challenging and reviewing models can use group fairness comparisons as one input when assessing whether a model's treatment of protected groups is defensible. They should treat it as one of several fairness approaches rather than a complete measure, and probe which formalization was applied and why, since a different metric may yield a different conclusion.

Compliance and legal professionals

Those assessing whether automated decisions may produce discriminatory effects across groups defined by attributes such as race or gender can use group fairness as a structured way to frame that scrutiny. They should be cautious not to overstate what a passing group-fairness test demonstrates, given that specific formalizations vary by context and are not standardized across all frameworks.

Auditors and oversight functions

Auditors reviewing AI systems can examine how group fairness was defined, measured, and documented as part of evaluating an organization's handling of algorithmic bias. Because the choice of protected attribute and outcome metric affects the result, auditors should confirm that these choices are recorded and that group fairness is not presented as evidence of individual fairness.

Inside Group Fairness

Statistical parity (demographic parity)

A group fairness criterion under which the rate of a favorable model outcome is intended to be approximately equal across defined groups, independent of the true label. It focuses on outcome distribution rather than accuracy within groups.

Equalized odds

A criterion under which error rates, typically true positive and false positive rates, are intended to be approximately equal across groups. It conditions on the true outcome and therefore differs from statistical parity.

Equal opportunity

A relaxation of equalized odds that focuses on equalizing true positive rates (or, depending on framing, false negative rates) across groups, applied to those who share a given true label.

Predictive parity (calibration within groups)

A criterion under which the meaning of a predicted score is intended to be consistent across groups, for example that a given predicted probability corresponds to a similar observed rate regardless of group membership.

Protected or sensitive attribute

The group-defining variable (such as a demographic characteristic) across which fairness is assessed. The choice, availability, and legal permissibility of using such attributes vary by jurisdiction and use case.

Group-level aggregation

The defining feature of group fairness: metrics are computed and compared at the level of predefined groups rather than for individuals or matched pairs, distinguishing it from individual fairness notions.

Common questions

Answers to the questions practitioners most commonly ask about Group Fairness.

Does satisfying group fairness mean the model treats every individual fairly?

No. Group fairness evaluates statistical parity of outcomes or error rates across defined groups, not the treatment of any specific individual. A model can satisfy a group-level metric while still producing outcomes that a given individual would experience as unfair. This is why group fairness is commonly distinguished from individual fairness, which focuses on similar individuals receiving similar treatment. The two can conflict, and satisfying one does not guarantee the other.

Is there a single group fairness metric that, if met, confirms the model is fair?

No. Group fairness is not one metric but a family of them, including measures oriented around demographic parity, equalized odds, and equal opportunity, among others. These metrics encode different notions of fairness and cannot generally all be satisfied at once except under specific conditions. Selecting a metric involves normative and context-specific judgments rather than the application of one authoritative standard, so meeting a single metric does not establish fairness in a general sense.

How do we decide which group fairness metric applies to a given use case?

Metric selection typically depends on the decision context, what harms are of concern, and applicable policy or legal considerations, and it often involves stakeholders beyond the modeling team. Because different metrics reflect different fairness definitions and can be mutually incompatible, teams commonly document the rationale for the chosen metric, the groups being compared, and the tradeoffs accepted. This choice is generally treated as a governance decision rather than a purely technical one.

How should the groups themselves be defined for a group fairness assessment?

Group definitions depend on the protected or relevant attributes at issue and on data availability, and they can materially affect results. Practitioners commonly consider whether attributes are directly observed, inferred, or unavailable, and whether intersectional groups should be examined in addition to single-attribute groups. Because definitions carry assumptions and legal sensitivities, they are typically documented and reviewed rather than assumed.

Where does group fairness assessment fit within model validation and monitoring?

Group fairness metrics are often computed during development and validation and then tracked over time, since disparities can emerge or shift as data and populations change. Incorporating these checks into ongoing monitoring aligns with the broader practice of tracking model behavior after deployment. The specific placement and frequency depend on the organization's governance structure and applicable expectations rather than a universal rule.

What are the practical limits of relying on group fairness metrics?

Group fairness metrics can be sensitive to how groups are defined, to sample size within groups, and to the metric chosen, and they do not by themselves address individual-level concerns or the reasons behind observed disparities. They measure statistical patterns rather than causes, and improving one metric may worsen another or affect overall performance. For these reasons they are commonly used as part of a broader assessment rather than as a standalone determination of fairness.

Common misconceptions

Group fairness is a single, well-defined property a model either satisfies or fails.

Group fairness is an umbrella for several distinct and often mutually incompatible criteria (for example statistical parity, equalized odds, and predictive parity). In many settings these cannot all hold simultaneously except in special cases, so satisfying one does not imply satisfying others, and practitioners must choose which criterion is appropriate for their context.

Fairness and bias are interchangeable terms.

As commonly used, bias typically refers to a systematic error or skew in data, measurement, or outcomes, whereas fairness is a normative judgment about whether outcomes are acceptable across groups. A model can exhibit measurable statistical differences without those differences necessarily being deemed unfair, and vice versa, so the two should not be collapsed.

Achieving group fairness on chosen metrics eliminates discrimination or legal risk.

Group fairness measures manage and reduce certain measurable disparities; they do not by themselves guarantee compliance with applicable anti-discrimination law, which varies by jurisdiction and may turn on factors beyond the metrics computed. Group-level parity can also coexist with unfairness at the individual or subgroup level.

Best practices

Explicitly state which group fairness criterion or criteria you are using (for example statistical parity, equalized odds, equal opportunity, or predictive parity) and document why that choice fits the specific use case and its potential harms.

Acknowledge and record the trade-offs and known incompatibilities among criteria, so stakeholders understand that improving one measure may worsen another.

Define the groups and the sensitive attributes used for assessment carefully, and confirm that collecting or using those attributes is permissible under the applicable jurisdiction before relying on them.

Report group fairness metrics alongside overall and per-group performance measures, keeping fairness distinct from accuracy and from generic notions of bias.

Complement group-level analysis with checks at finer granularity (such as subgroups) where feasible, since group parity does not guarantee equitable treatment of individuals or intersectional subgroups.

Treat fairness assessment as ongoing rather than one-time, monitoring metrics over time and documenting limitations, contested definitions, and any legal or domain-specific constraints that qualify the results.