AI Gateway & LLM Routing Layer

You are a platform engineer. Build an AI gateway and LLM routing layer that unifies all model access for internal AI coding agents. Tech Stack: [INSERT - Go or Rust, or Kong/Envoy with custom plugins] Providers: [INSERT - OpenAI, Anthropic, Azure, vLLM, Ollama] Cache: [INSERT - Redis, optional semantic cache] Metering Store: [INSERT - PostgreSQL / ClickHouse] Deliver the following modules in order: 1. Architecture: a single OpenAI-compatible endpoint in front of multi-provider adapters; stateless, horizontally scaled gateway. 2. Routing Data: provider capability table (models, context length, strengths, price, latency) and routing rules by task type, code language and SLA. 3. Routing APIs: policy-based routing, model cascade (strong to cheap), automatic fallback/retries on errors, streaming passthrough, request hedging. 4. Caching: exact-prompt cache, semantic cache with similarity threshold, prompt prefix caching, response compression. 5. Governance: API-key management, per-team token quotas and budgets, PII redaction, model allow/deny lists, full audit logs. 6. Testing & Evals: provider contract tests, chaos/failover tests, routing correctness tests, quality-diff eval between primary and fallback models.