RAG Poisoning Detection and Knowledge-Base Integrity Protection

You are securing the retrieval-augmented generation pipeline of [INSERT_APP_NAME]. This defensive project detects and prevents poisoning of the knowledge base, embeddings, and retrieved content that could mislead the model or smuggle instructions. 1. Authorization: Work under the data/platform owner's approval over the staging and production indices you are assigned to protect; external sources require approved content agreements. 2. Architecture Mapping: Document ingestion connectors, source repositories, embedding jobs, the vector database, ranking logic, and the trust level of each source. 3. Threat Model: Cover source compromise, insider-contributed malicious documents, web-source injection, embedding collisions/adversarial content, outdated conflicting docs, and indirect prompt injection in retrieved chunks. 4. Ingestion Controls: Enforce source allow-listing, contributor authentication, content approval workflows, schema validation, and malware scanning before indexing. 5. Content Analysis: Scan documents for embedded instructions, hidden text, suspicious links/iframes, impersonation of official authors, and anomalous formatting; quarantine before serving. 6. Embedding Integrity: Baseline index statistics, detect anomalous clusters/outliers and sudden content shifts, version snapshots, and cryptographically sign approved index builds. 7. Retrieval-Time Defenses: Apply trust-tier weighting, enforce source attribution/citations, filter chunk-level content, and treat retrieved text as untrusted data for the model. 8. Authorized Red-Team Validation: With written approval, plant benign canary documents and simulate poisoning in an isolated test index to measure detection and response. 9. Tooling: Use pipeline-native checks, approved LLM-security scanners ([INSERT_LLM_SCANNER]), vector-DB audit logs, and SIEM ingestion of index changes. 10. Evidence & Compliance: Maintain source lineage, approval records, index change logs, and canary results; map controls to [INSERT_REGULATION] and data-governance policy. 11. Monitoring: Alert on bulk-ingestion spikes, new connectors, anonymous edits, retrieval of quarantined content, and citations of untrusted sources. 12. Remediation: Roll back to signed index snapshots, purge malicious chunks, revoke compromised source credentials, notify data owners, and re-verify downstream outputs.