Home / Curated / Evaluation Criteria / Counterfactual fairness

Counterfactual fairness

Evaluation Criteria AIMech:CounterfactualFairness DRAFT

Aliases: CF (counterfactual fairness)

A fairness criterion in a specified causal model requiring a predictor's conditional output distribution, given an individual's observed attributes and protected-attribute value, to remain unchanged under counterfactual interventions that assign other attainable protected-attribute values.

Identity Notes & Concept Scope This is Kusner and colleagues' Definition 5, an equality of conditional counterfactual distributions. It is not merely changing an input feature while freezing its causal descendants, a group-level observational equality, the FAIRLEARNING algorithm, or a generic guarantee of social fairness. All 442 pinned AIO labels/scoped synonyms, 94 preceding native identities and ignored-inclusive text in both checkouts were searched. Later parity connections do not establish identity under an arbitrary fixed causal model.

Claims & Grounded Evidence

3 Claims

Scope #criterion-scope

Condition on the factual context X=x,A=a to infer background-variable uncertainty; replace the protected-attribute structural assignment by a counterfactual value and propagate that intervention through the causal model. Compare the resulting prediction distributions for all attainable protected values while retaining the same factual conditioning context. The criterion is distributional, and the paper gives using non-descendants of the protected attributes as a sufficient construction, not a necessary restriction on all possible predictors.

supports Source: Counterfactual Fairness · Locator: Section 2.2 and Section 3, Definition 5/equation (1), PDF p. 3; Section 3.2, Lemma 1 and its qualification, PDF p. 5; Sections 4.2–4.3, causal assumptions and interpretive limits, PDF p. 7

Limitation #causal-model-misspecification

The sensitivity study distinguishes a guessed model used to construct a fair predictor from a model allowing additional unobserved confounding through correlated errors. Fairness in the guessed model does not guarantee fairness under the alternative data-generating assumptions; the paper frames this as sensitivity analysis rather than identification of a uniquely correct hidden model.

supports Source: The Sensitivity of Counterfactual Fairness to Unmeasured Confounding · Locator: Section 2.1, PDF pp. 2–3 (volume pp. 617–618); Section 3.1 and Figure 1, PDF p. 4 (volume p. 619)

Limitation #parity-relation-scope

Rosenblatt and Witter's parity connections use their explicit independence criterion for latent and protected variables, and the demographic-parity-to-counterfactual-fairness direction constructs a causal model and latent representation. The paper distinguishes that setting from assessment with a fixed causal model; the relationship is not an unrestricted synonym assertion.

supports Source: Counterfactual Fairness Is Basically Demographic Parity · Locator: Related Work distinction between a fixed and freely constructed causal model, PDF p. 3; Criteria 3 and Theorems 4–5, PDF p. 4

⚠️ Curated Limitations & Negative Results

The comparison depends on causal and counterfactual assumptions that observational data alone generally cannot establish. Specifying protected attributes and meaningful interventions requires domain interpretation.
Ignoring protected attributes as explicit inputs is insufficient when other inputs carry their causal effects. Conversely, using a descendant is not categorically excluded if the relevant total dependence cancels, as the source qualifies its sufficient construction.
A predictor satisfying the condition in an assumed model need not satisfy it under a model with omitted confounding; the inspected sensitivity paper studies this vulnerability in additive-noise models.
The criterion constrains conditional output distributions, not necessarily pointwise equality for every unobserved state or usefulness of a predictor. Connections to demographic parity retain their model/independence assumptions.

Artificial Intelligence Ontology (AIO) Grounding

Mapped AIO Classes

AIO Target Class	Mapping Relation	Mapping Rationale
`AIO:CausalGraphicalModel` Causal Graphical Model	related_match	The criterion is defined and assessed using a structural causal model that encodes the data-generating assumptions. AIO CausalGraphicalModel describes that supporting model type; the criterion itself is neither a graphical model nor an equivalent causal graph.

Evidence Sources & Primary Literature

3 Sources

Source ID	Publication / Document	Type / Access	Reference & Checksum
`original-paper`	Counterfactual Fairness Matt J. Kusner, Joshua R. Loftus, Chris Russell, Ricardo Silva	primary paper full_text	ID: arXiv:1703.06856 sha256:2d7371209e44…
`confounding-sensitivity`	The Sensitivity of Counterfactual Fairness to Unmeasured Confounding Niki Kilbertus, Philip J. Ball, Matt J. Kusner, Adrian Weller, Ricardo Silva	primary paper full_text	ID: https://proceedings.mlr.press/v115/kilbertus20a.html sha256:f7083ab492a2…
`causal-parity-analysis`	Counterfactual Fairness Is Basically Demographic Parity Lucas Rosenblatt, R. Teal Witter	primary paper full_text	ID: arXiv:2208.03843 sha256:ebc09da1ee78…

Curation Provenance

Curator: Codex

Method: ai_assisted

Model: GPT-6

Created: 2026-10-11T04:48:50Z

Updated: 2026-10-11T04:48:50Z

Source File: data/curated/CounterfactualFairness.yaml