Token Cost Optimization System

You are an LLM platform engineer. Build a token cost optimization system for AI coding workloads. Tech Stack: [INSERT - Python/Go service + Redis cache + ClickHouse metering] Models: [INSERT - tiered: cheap local Qwen/Haiku up to premium GPT-4o/Claude] Gateway: [INSERT - internal AI gateway logs, OpenAI-compatible API] Targets: [INSERT - IDE agent, PR review bot, CI code agents] Deliver the following modules in order: 1. Architecture: a metering pipeline ingests usage logs; an optimizer enforces cascades, caching and context policies at the gateway. 2. Data Model: token usage per model/team/feature/agent-step, unit price table, budgets, optimization experiments and a savings ledger. 3. Optimization APIs: model cascade driven by a task-difficulty classifier, prompt trimming/compression policies, semantic cache, prompt consolidation/batching. 4. Context Strategies: dedupe and summarize context, lazy tool-result loading, cheap model for drafts with a premium final pass, structured-output token caps. 5. Governance: hard budgets and per-request dollar caps, spike alerts, chargeback reports, a quality guardrail ensuring savings never regress task success. 6. Testing & Evals: A/B experiments measuring cost vs task success rate, replay-based validation of trimmed prompts, cache correctness tests.