Non-discrimination — PACT Standard assessment axis

What this axis protects.

Non-discrimination protects equal dignity, treatment and access. It asks whether an AI-enabled process reproduces direct or indirect discrimination, creates new barriers, or distributes errors and opportunities unequally across protected and marginalised groups.

Bias can enter through the question chosen, historical decisions, labels, missing data, measurement, proxies, optimisation targets, interface design and deployment conditions. It can also emerge after launch through feedback loops: people receive fewer opportunities, generate less favourable data and are then treated as evidence that the earlier decision was correct.

Fairness is not a property of a model in isolation. It is a judgement about people, outcomes and power in a particular context.

Analyse the whole decision chain.

Problem definition. Ask whether the target is legitimate, measurable and connected to the real decision. A precise proxy for an unjust objective remains unjust.
Representation and quality. Examine who is present, absent or poorly measured; which languages, disabilities and intersectional groups have adequate evidence; and whose history defines “normal”.
Labels and outcomes. Determine whether labels reflect reality or past institutional choices, unequal access, selective enforcement or biased human judgement.
Error distribution. Compare error types that matter in context. False positives and false negatives can impose very different burdens on different groups.
Access and interaction. Include disability, language, connectivity, literacy, documentation requirements and availability of a non-digital route.
Feedback and remedy. Test whether outcomes reinforce future disadvantage and whether people can correct data, challenge decisions and obtain meaningful redress.

Where non-discrimination commonly fails.

Average performance presented as fairness. Overall accuracy hides smaller groups, rare cases and unequal costs of error.
A single metric without a normative choice. Teams select a parity measure without explaining why it fits the right, decision and harm at issue.
Protected attributes removed, proxies retained. Location, education, language, work history or behaviour can reproduce protected or socioeconomic patterns.
Small samples treated as no problem. Insufficient evidence is reported as equality rather than uncertainty that limits deployment.
Benchmark mismatch. A model validated on another country, language, institution or population is assumed to transfer without local evidence.
Accessibility reduced to compliance. Technical conformance is checked without observing whether disabled people can complete the actual task independently.
Mitigation ends at launch. No one monitors real outcomes, complaints, overrides or cumulative feedback loops after deployment.

Assessment questions.

Which decisions, benefits, burdens and rights are affected, and which groups face a plausible risk of direct, indirect or intersectional disadvantage?
What historical or institutional inequality may be encoded in the target, labels, data collection or baseline process?
Are data sufficiently representative and reliable for each relevant population, language, disability and deployment context?
Which error types cause which harms, and how do performance, uncertainty and outcomes differ across relevant groups?
Why were the selected fairness measures chosen, what trade-offs do they create and who approved the normative decision?
Could apparently neutral features, thresholds, procedures or access requirements disadvantage a protected group?
Can people use an accessible alternative, correct data, request accommodation and obtain human review without extra penalty or delay?
How will the organisation detect feedback loops, emerging disparities and harms that quantitative testing did not predict?

Evidence to collect.

Impact map: Decision, affected groups, protected rights, potential direct and indirect discrimination, intersectional risk, error consequences and existing inequality.
Data record: Provenance, collection context, population coverage, missingness, label quality, proxy analysis, language and accessibility limits, with uncertainty made explicit.
Outcome analysis: Context-relevant performance and error measures by group and intersection where lawful, statistically supportable and necessary, including confidence intervals and sample limitations.
Qualitative evidence: Structured input from affected people, representative organisations, frontline staff, accessibility specialists and independent equality or domain experts.
Mitigation record: Changes to objective, data, features, thresholds, interface, human review or access route, plus revalidation showing both benefit and new trade-offs.
Remedy and monitoring: Accessible appeal, correction and accommodation procedures; complaint and outcome monitoring; escalation thresholds and authority to restrict or stop use.

Decision thresholds.

Proceed: The objective is legitimate; evidence covers relevant groups and contexts; material disparities are understood and controlled; access, accommodation, review and monitoring are operational.
Proceed with conditions: Evidence gaps are bounded through limited scope, additional human review, accessible alternatives or a monitored pilot that does not expose people to irreversible harm.
Stop or redesign: The objective or proxy embeds unlawful or unjustified inequality; material disparities cannot be measured or mitigated; relevant groups are absent from evidence; or affected people lack a practical alternative and remedy.

Handle protected data lawfully. Do not collect or infer protected characteristics merely because bias testing would be useful. Establish a valid legal basis, necessity, safeguards and governance first. If lawful measurement is not possible, record the uncertainty and restrict the use; do not assume the system is fair.

Lifecycle checks.

Before first use

Define the equality question, map affected groups, assess data and proxy risk, choose context-specific measures, test accessibility and involve people likely to carry the greatest burden.

During operation

Monitor real outcomes, not only model metrics. Review disparities in approvals, errors, appeals, wait times, drop-off, overrides, accommodations and complaints, including changes caused by feedback loops.

When conditions change

Repeat assessment after shifts in population, language, policy, target, threshold, data, interface, service channel or economic conditions. A system can become discriminatory when its surrounding institution changes.

Sources and further reading.

Primary and institutional sources reviewed for this working document. Legal applicability must be assessed for the specific system and context.