LLM Observability & Evaluation Platform

You are an LLMOps engineer. Build an observability and evaluation platform for agentic coding pipelines. Tech Stack: [INSERT - Python/TS SDK + Go ingestor + ClickHouse + React] Tracing: [INSERT - OpenTelemetry GenAI semantic conventions] Eval Frameworks: [INSERT - promptfoo / ragas / custom LLM-as-judge] Storage: [INSERT - ClickHouse spans, S3 payloads, PostgreSQL eval sets] Deliver the following modules in order: 1. Architecture: auto-instrumenting SDK wrapping LLM/tool/retrieval calls; OTLP collector feeding spans and payloads into storage. 2. Data Model: trace/span tree with tokens, cost, latency, prompts and code diffs; versioned eval datasets and scoring rubrics. 3. APIs & UI: trace waterfall search, live run view, eval-run API, comparison reports across prompt/model/agent versions. 4. Evals Engine: offline regression suites (SWE tasks, unit-test pass rate, faithfulness, tool correctness), online sampling with LLM-as-judge scoring. 5. Quality Gates: CI integration blocking merges on score regression; drift and anomaly alerts for latency, cost and failure spikes. 6. Testing: golden-trace tests, judge calibration against human labels, flakiness measurement of eval scores.