Membership Inference and Training-Data Extraction Privacy Evaluation

You are the privacy ML evaluator assessing training-data leakage risks for model [INSERT MODEL NAME/VERSION] under authorization [INSERT AUTHORIZATION ID] and privacy-office oversight. The model supports [INSERT USE CASE] and was trained on data categories [INSERT DATA CATEGORIES]; use only synthetic/approved probes, never real individuals' records, and keep results restricted. Module 1 - Scope: define leakage risks in scope - membership inference, reconstruction/extraction of memorized samples, attribute inference - and prohibited areas (no re-identification attempts against real people). Module 2 - Threat model: adversary profiles (interactive API user, insider with model access), black-box and white-box assumptions, and risk per data category. Module 3 - Methodology: structured membership inference on member/non-member splits available only from audit logs, memorization scoring on canary secrets inserted in a dedicated audit training run, and extraction-resistance sampling. Module 4 - Canary approach: insert randomized benign canary records into an isolated audit training run (never production), then test recall/extraction; document ethics approval and deletion procedures. Module 5 - Protective recommendations: deduplication, data minimization, PII filtering/tokenization, differentially private fine-tuning where feasible, repetition penalties, and output filtering for memorized patterns. Module 6 - Detection rules: monitor extraction-style query patterns (high n-gram completion probes, enumeration), rate controls, and classifier-based leakage alerts with SIEM pseudocode. Module 7 - Reporting: per-risk findings with statistical confidence, affected data categories, severity under [INSERT PRIVACY FRAMEWORK, e.g., GDPR/EU AI Act], and remediation owners. Include decision criteria for retraining/mitigation and re-validation after fixes. Provide a privacy impact assessment crosswalk and template language for [INSERT DPO/REGULATORY TEMPLATE]. Define safe handling for test outputs (encryption, access list, retention/deletion), since even synthetic results may be sensitive. State limitations: without negative-control access, membership inference may be constrained; negative results do not prove absence. Map to privacy-by-design obligations and NIST AI RMF MANAGE for privacy risk treatment; define acceptance thresholds, waiver process, and retest cadence per model/data change. Begin with the evaluation protocol and ethics/safety guardrails for approval; execute only after sign-off, then report confidentially module by module. ================