Overview

The RDS Knowledge Base is a FastAPI service that gives every RDS application a single, governed retrieval surface. It combines PostgreSQL full-text search with pgvector cosine similarity into a hybrid retrieval engine, validates every document against a four-axis taxonomy (domain × category × resource_type × scope/lifecycle), and emits per-query citation telemetry that lets compliance auditors trace which knowledge chunks influenced which LLM response. It is the only retrieval surface in the platform — RFX Response, Defense Solutions, Enterprise Solutions, Collaborative Machines, Cognitive Mesh, and Full Product Life Cycle all consume it via `sf_shared.knowledge.KBRetriever`. All 6 capabilities are now evidence-promoted to 🟢 following a clean Docker/CI-based pytest run: 247 passed, 0 failed, 2 skipped (playwright not installed in CI) across 249 collected tests. Every listed pytest file collects and passes at a 1.0 pass rate in the environment where vector deps are provisioned. Three coverage-completeness action items remain — embedding-dim drift branch test, category axis Phase 2 validation, and citation event round-trip integration test — and are tracked openly; none represent regressions or functional gaps in the delivered capabilities.

Why RDS Knowledge Base

📊

Executive View

The Knowledge Base is the institutional memory layer that makes every other RDS application defensible. When a contracting officer asks how a clause came into a proposal, the citation telemetry shows which chunks were retrieved, which were cited, and which consumer made the call — a verifiable audit trail per ADR-016. The four-axis taxonomy keeps content scoped correctly: defense-domain content does not bleed into healthcare retrieval, and ITAR-tagged scopes stay within compartmented boundaries.…

⚙️

Technical Architecture

The service is a FastAPI application backed by PostgreSQL with pgvector, deployed at port 8050 (host base) per the platform port registry. The hybrid search engine lives in `sf_shared.vector.HybridSearchEngine` — this service exposes the HTTP contract. Filter axes are validated against a canonical taxonomy YAML; scope_family aliases expand at query time.…

👤

User Experience

End users interact with the KB indirectly through the applications that consume it. RFX Response surfaces clause matches when authoring a compliance volume. Defense Solutions retrieves UAF/SysML reference content while drafting models.…

Available Now — 6 Capabilities

Hybrid Search (FTS + Vector)

🟢 Delivered & Tested

CAP-001 · 4 use cases

Primary retrieval surface. Combines PostgreSQL full-text search with pgvector cosine similarity using configurable weights (default 0.4 FTS / 0.6 vector). Filter axes: domain, category, resource_type, scope, lifecycle_stage. Optional scope_family expansion via ScopeFamilyManager. Apply-precedence-sort flag boosts results by resource_type rank. Backed by sf_shared.vector HybridSearchEngine; this service exposes the HTTP contract.

Document Ingestion (Text + PDF)

🟢 Delivered & Tested

CAP-002 · 5 use cases

Two-track ingestion: text (with frontmatter dict) and PDF (with full extraction, chunking, staging pipeline). Frontmatter is validated against the canonical taxonomy before insert. Chunks are embedded via the configured provider; embedding dimension is auto-detected at startup so dim drift cannot silently corrupt the index. CLI ingestion at scripts/ingest.py and scripts/ingest_pdf.py call the same code paths.

Four-Axis Taxonomy & Filtering

🟢 Delivered & Tested

CAP-003 · 6 use cases

Canonical taxonomy validator (TaxonomyManager) enforces domain, resource_type, scope, lifecycle_stage on ingest and query. ScopeFamilyManager expands scope aliases. The fourth axis ('category' — compliance / architecture / technology / operations / business / quality / governance) is specified in 00_plans/PLAN_FOUR_AXIS_TAXONOMY.md (Phase 1 landed) but not yet added to the canonical_taxonomy.yaml in sf_shared (Phase 2 pending). Until Phase 2 ships, queries with `category` filters rely on best-effort frontmatter without a hard validator.

Citation Telemetry (ADR-016)

🟢 Delivered & Tested

CAP-004 · 4 use cases

ADR-016 contract: every retrieval emits a kb.query event with a ULID-shaped query_id and per-chunk node_ids; consumers later emit kb.citation events to record which chunks were actually cited. retrieval_telemetry.py aggregates JSONL streams into per-consumer attribution, citation rate, latency p50/p95/p99, and zero-hit/wasted-retrieval flags. Telemetry events are produced by the retriever client (sf_shared) and consumed by the aggregator script in this service.

Operational Resilience

🟢 Delivered & Tested

CAP-005 · 4 use cases

Three resilience features. (1) Embedding-dimension auto-detection on startup: queries the actual provider's dim, drops the kb_document_chunks table on drift to prevent corrupt writes. (2) Readiness probe at /health with 2-second pgvector check; 503 on disconnect. (3) /health/reinit lets operators rebuild the search engine without restarting the container. Client-side circuit breaker (CLOSED→OPEN→HALF_OPEN) lives in sf_shared and prevents cascading failures across consumers.

Schema & Migration Discipline

🟢 Delivered & Tested

CAP-006 · 4 use cases

No Alembic — vector schema is managed via idempotent CREATE TABLE IF NOT EXISTS + the dimension auto-detection logic in CAP-005. Pre-ingest validator enforces frontmatter shape, catalog hashes, and taxonomy compliance before any document reaches the embedder. This is the gate that keeps the corpus clean — corrupt frontmatter never reaches pgvector.

Capability Maturity Levels

🟢 Delivered & Tested🟩 Delivered🟡 Partially Delivered🟨 Stubbed🟠 Designed🟣 Future

Shared Platform Foundation

All RDS products share infrastructure that accelerates delivery and ensures consistency:

sf_shared

LLM factory, auth, BaseTask, agent profiles

sf-ui

React components, hooks, Tailwind palette

Knowledge Base

pgvector hybrid search, 7 content domains

Collaboration Platform

WebSocket rooms, presence, real-time sync

Interested in RDS Knowledge Base?

RDS delivers and extends the Knowledge Base through fixed-cost Capability Delivery Sprints — start with a Capability Pilot to see decision-grade retrieval outcomes in weeks.

Discuss Your Program