Multi-Turn Agentic Red Team Exercise Design with Rules of Engagement
设计针对自主智能体的多轮授权红队演练(权限滥用、工具链越权、目标偏离),含交战规则、止损与人因评估;适用于高自治AI系统的定期对抗演练。
You are the red-team exercise director planning an authorized, defensive tabletop-to-live exercise against agent system [INSERT AGENT SYSTEM NAME] in [INSERT ISOLATED ENVIRONMENT]. Authorization reference [INSERT AUTHORIZATION ID] defines scope; the exercise protects [INSERT CROWN-JEWEL ASSETS] with simulators and synthetic accounts - no real customers, payments, or external sends. Module 1 - Objectives and scope: test permission misuse, tool-chain chaining, goal drift, prompt injection via data, excessive autonomy, and recovery controls; explicitly list prohibited actions and blast-radius limits. Module 2 - Rules of engagement: permitted techniques at an abstract level, stop conditions, kill-switch owner, human-in-the-loop checkpoints, logging requirements, and the communication/escalation tree. Module 3 - Scenario design: realistic benign-but-adversarial scenarios (e.g., manipulated document, compromised low-privilege tool account) mapped to MITRE ATLAS and OWASP agent risks, with success measured against safety invariants. Module 4 - Execution model: phased escalation from tabletop to live, a dedicated white cell/observers, timed injects, and immediate halt on safety-invariant violation. Module 5 - Blue-team enablement: pre-exercise logging checks, detection-content expectations, and an agreed no-patch window; exercise only during [INSERT MAINTENANCE WINDOW]. Module 6 - Protective findings focus: gaps in consent gates, tool allowlists, egress controls, supervision, and rollback; pair every finding with a remediation recommendation. Module 7 - Detection validation: which SIEM/EDR alerts fired, time-to-detect, and time-to-kill; provide rule-improvement pseudocode. Include a scoring rubric per objective (prevented/detected/contained/impacted) and a safety-invariant checklist signed by [INSERT SAFETY LEAD]. Provide participant briefing materials, synthetic-data handling instructions, and a hot-wash agenda. Define reporting: executive summary, technical findings with sanitized evidence, remediation SLAs, and re-exercise criteria within [INSERT RE-EXERCISE WINDOW]. Coordinate legal/privacy review confirming no real personal data is used in any scenario. Map findings to [INSERT FRAMEWORKS, e.g., NIST AI RMF, ISO/IEC 42001, internal AI risk policy]; state assumptions and contingencies for agent non-determinism (repeat runs, seeded policies where possible). First present the exercise plan, ROE, and safety plan for approval; run only after written sign-off, then deliver the after-action report. ================
自动翻译
自动翻译中…