vllm-project/semantic-router
feat: add HaluGate-driven per-model per-domain reputation routing
Closed
#2,542 opened on Jul 15, 2026
area/model-selectionarea/observabilityenhancementevaluationhelp wantedpriority/P1roadmapsignal-decision-engine
Repository metrics
- Stars
- (4,293 stars)
- PR merge metrics
- (PR metrics pending)
Description
Motivation
HaluGate already produces request-level signals that are captured in Router Replay. The missing capability is a trustworthy materialization pipeline that converts those observations into bounded model/domain reputation evidence and then exposes it to Router Learning.
Audited upstream baseline (2026-07-15)
- HaluGate is registered in the model/config surfaces and replay records include HaluGate-related fields.
- Router Replay records the selected logical model, but a durable model-version/config-version identity is not guaranteed across every record.
- There is no production materializer that aggregates HaluGate outcomes into per-model/per-domain reputation.
- Router Learning—not the generic model selection package—is the intended consumer for bounded adaptation.
Scope
- Define a versioned HaluGate outcome event with logical model, deployable model/version identity, domain/taxonomy version, config/policy version, score/verdict, and provenance.
- Add an asynchronous materializer with minimum sample thresholds, time decay, uncertainty, and bounded storage/cardinality.
- Separate observed evidence from the derived reputation snapshot.
- Make Router Learning consume reputation only as a bounded feature/action prior under
protection. - Provide shadow mode, replay diagnostics, rollback, and operator inspection.
- Specify behavior for missing versions, model aliases, delayed outcomes, and evaluator drift.
Safety and statistical boundaries
- Sparse or stale evidence must not produce confident routing changes.
- HaluGate evidence cannot override authorization, hard safety rules, or the validated candidate set.
- Aggregate by privacy-safe keys; do not require raw prompt retention.
- Correlation is not a causal quality estimate; evaluation must compare against fixed baselines.
Acceptance criteria
- Event and snapshot schemas are versioned and migration-tested.
- Model/config/taxonomy identity is sufficient to prevent cross-version contamination.
- Decay, sample thresholds, uncertainty, and cold-start behavior are deterministic.
- Shadow/replay evaluation precedes live Router Learning consumption.
- Every applied effect is reconstructable from Router Replay and emits a bounded reason.
- Restart, multi-replica, and delayed-outcome semantics are documented.
- E2E tests prove hard safety constraints take precedence.
Likely change surfaces
src/semantic-router/pkg/routerreplay/, src/semantic-router/pkg/routerruntime/, src/semantic-router/pkg/extproc/, src/semantic-router/pkg/config/, observability, CLI diagnostics, docs, and tests.
Related: #2238, #2243.