Automated Playtesting and GenAI Evaluation Harness

You are the QA engineering director for [INSERT GAME TITLE]; design an automated playtesting harness that evaluates AI-generated levels, rules, NPCs, and items before players see them. The harness combines LLM player agents, behavior-clone and RL agents, and static analysis to find breaks, imbalance, confusion, and safety issues in [INSERT CONTENT TYPES]. Module 1 - Gameplay design: define the quality model—completion, flow, legibility, fairness, novelty, fun proxies, and safety thresholds every artifact must meet. Module 2 - Generation pipeline: specify scenario sampling, diverse agent profiles, headless engine farms, trajectory collection, failure clustering, and LLM-written bug reports. Include adversarial personas—rusher, explorer, griefer, child-like user—and pairwise human-preference calibration for learned fun metrics. Define the gating policy: auto-approve, human review, and auto-reject bands, plus canary exposure against real-player outcomes. Explain how failing artifacts route back to the generator with structured repair hints. Define longitudinal checks comparing canary retention and fun with pre-release agent predictions. Deploy safety-specific agents probing for toxicity, self-harm, and grooming pathways in generated content. Module 3 - Data structures: provide schemas for test jobs, agent profiles, trajectories, defect clusters, scorecards, and human-calibration labels. Module 4 - Testing: define evaluator-reliability tests, flake budgets, coverage across the generation space, and persona bias audits. Module 5 - Operations: cover CI integration for every model and template release, capacity planning, dashboards, trend tracking, and regression baselines. Provide a text architecture diagram, a sample scorecard, risk register, and KPIs for defect escape rate and false approvals. List assumptions about headless engine support, CI budget, and existing telemetry. Begin with a one-page quality-model summary, then deliver each module one at a time for approval.