Privacy — PACT Standard assessment axis

What this axis protects.

Privacy protects dignity, autonomy and control over information about a person. In AI systems, risk does not end when direct identifiers are removed. Models can retain, reveal or allow inferences about people; prompts and retrieval systems can expose confidential records; outputs can create sensitive or inaccurate profiles; and vendor logs can become a second, poorly governed dataset.

The GDPR principles remain the starting point: lawfulness, fairness and transparency; purpose limitation; data minimisation; accuracy; storage limitation; integrity and confidentiality; and accountability. Data protection by design means these principles shape architecture and procurement before data enters the system.

The first privacy control is not encryption. It is a justified decision that the organisation needs this data and this processing at all.

Map the full data lifecycle.

Inputs. Identify training, fine-tuning, evaluation, prompt, retrieval, feedback and monitoring data, including inferred data and metadata.
Roles and flows. Establish controllers, joint controllers and processors; every recipient, subprocessor, location and international transfer; and what suppliers may reuse.
Purpose and legal basis. Separate development, evaluation, deployment, security, improvement and secondary uses. A lawful basis for one purpose does not automatically cover another.
Outputs and inferences. Assess personal data generated about people, not only data supplied to the system. Include inaccurate associations, scoring, categorisation and sensitive inferences.
Retention and deletion. Define how long raw data, prompts, embeddings, logs, outputs, backups and trained artefacts remain, and how rights or deletion requests propagate.

Where privacy commonly fails.

Unknown provenance. Teams cannot show where training or evaluation data came from, what permissions apply or whether collection was lawful.
Purpose expansion. Data collected to deliver a service is later used to train or improve a model without a separate necessity and legal-basis analysis.
Minimisation by assumption. Every available field is retained because it might improve performance, without measuring whether the gain justifies the intrusion.
Pseudonymisation treated as anonymity. Re-identification remains reasonably possible, yet the organisation stops applying data protection safeguards.
Supplier reuse hidden in terms. Prompts, files or feedback may be retained, inspected or used for service improvement beyond the deployer’s intended purpose.
Rights without implementation. Notices promise access, correction or deletion, but the organisation cannot locate data across logs, embeddings, training sets or outputs.
Model leakage and inference. Tests ignore extraction, membership inference, memorisation, prompt injection and exposure through connected tools or retrieval stores.

Assessment questions.

Which personal data and inferred data enter or emerge from every lifecycle stage, and which categories require additional protection?
Is each purpose specific and necessary, with a documented lawful basis and a clear allocation of controller and processor responsibilities?
Could the objective be achieved with less data, lower granularity, shorter retention, local processing or a non-AI alternative?
How were data sources obtained, documented, filtered and checked for incompatible secondary use or unlawful content?
Can people exercise access, correction, objection, restriction, deletion and rights related to automated decision-making in practice?
What prevents personal or confidential data from leaking through outputs, logs, model extraction, retrieval, tools or supplier access?
Has anonymity been demonstrated against realistic identification and extraction methods rather than asserted from de-identification alone?
Does the likely risk require a data protection impact assessment, prior consultation or additional restrictions before processing begins?

Evidence to collect.

Data inventory: Categories, sources, subjects, purposes, legal bases, recipients, locations, retention, deletion routes and ownership for data at every lifecycle stage.
Necessity test: Evidence that each field, granularity and retention period contributes materially to a legitimate purpose, including results from data-reduction experiments.
Supplier controls: Data-processing terms, subprocessor list, transfer safeguards, security commitments, audit rights, incident duties, reuse restrictions and verified deletion or return.
DPIA record: Screening decision and, where required, a full data protection impact assessment covering risks to people, mitigations, residual risk and consultation.
Rights procedure: Tested retrieval, correction, objection and deletion workflows across operational data, logs, derived data and model-related artefacts.
Privacy testing: Re-identification, extraction, memorisation, prompt injection, access control and data leakage tests matched to plausible attackers and user errors.

Decision thresholds.

Proceed: Purposes, legal bases and roles are clear; processing is demonstrably necessary and minimised; rights and deletion work; supplier terms and technical tests control foreseeable leakage.
Proceed with conditions: Residual issues are limited, time-bound and reversible, with restricted data, access, retention or functionality until specified evidence is produced.
Stop or redesign: Data provenance or lawful basis is unresolved; unnecessary sensitive data is central to the design; supplier reuse is uncontrolled; rights cannot be fulfilled; or plausible leakage creates an unmitigated high risk.

Privacy risk belongs to the model and its context. Whether a model can be considered anonymous requires a case-by-case assessment of both identification and extraction risk. Removing names from a dataset or calling data “synthetic” is not sufficient evidence.

Lifecycle checks.

Before first use

Complete data mapping, necessity and legal-basis analysis, supplier due diligence and DPIA screening before uploading real data or enabling integrations. Configure restrictive defaults and test rights workflows.

During operation

Monitor access, unusual queries, leakage, retention, supplier changes and new inferred uses. Audit whether teams bypass approved channels or enter personal data into general-purpose tools.

When conditions change

Reassess when the purpose, data source, model, supplier, hosting region, integration, population or output use changes. A new capability can create a new processing purpose even when the interface looks unchanged.

Sources and further reading.

Primary and institutional sources reviewed for this working document. Legal applicability must be assessed for the specific system and context.