Governance Overview
Loading metrics…
LLM Governance
checking…
Runtime safety of governed LLM traffic — what was blocked, what fell back, how grounded the answers were, and what it cost.
Governed runs
—
Blocked rate
—
Fallback rate
—
Answer relevancy
—
Groundedness
—
1.0 = fully grounded
Spend (USD)
—
Active rules —
Blocked —
Fallbacks —
Avg latency — s
Data Governance
checking…
Quality of retrieval-backed (RAG) answers — did we fetch the right documents, and were answers grounded in them? Precision@K and Hit Rate@K are score-based proxies (no ground-truth labels).
RAG runs
—
Retrieval Precision@K (score-proxy)
—
Hit Rate@K (score-proxy)
—
Context Relevance
—
Answer Groundedness
—
Retrieval Latency (ms)
—
Total LLM Calls
—
in selected period
Total Tokens
—
prompt + completion
Total Cost (USD)
—
based on model pricing
Avg Latency
—
seconds per call
Daily Calls & Cost Trend
Top Models by Usage
successful calls
Calls by Type
Guardrail Activity — requests per day
one bar unit = one request, not one rule hit
Guardrail Breakdown — checks fired, by action
WPIntelliChat LLM Governance — Live Use Case
6-step governance pipeline running inside WPIntelliChat
Prompt Lifecycle Status
Eval Scores by Prompt
Prompt Catalog — Current State
| Prompt | Category | Status | Relevancy | Coherence | Completeness | Overall | Action |
|---|---|---|---|---|---|---|---|
| Loading… | |||||||
LLM Audit Log
Complete record of every LLM call — who, what model, tokens, cost
Last 24h:
loading…
| Sr. No. | Timestamp ▼ | User | Model | Type | Request ID | User Prompt | Prompt Tok | Completion Tok | Latency | Cost USD | |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Loading… | |||||||||||
—
Models & Experiments
The model registry as a planetarium — distance = cost, size = context, ring = guardrail profile, green = the active API key's model — side by side with the prompt-experiment runs that A/B test those models. Click a planet for its model card; click a run for its variants.
Registry Atlas
Active API key: checking…
LLM
RAG
Tool
active key
distance = cost · size = context · ring = guardrail profile · click a planet to view
Experiment Runs
| Run ID | Experiment Name | Variants | Best Time (s) | Min Cost ($) | Date | |
|---|---|---|---|---|---|---|
| Loading… | ||||||
Prompt Catalog
Engineer, evaluate, and promote prompts through a governed lifecycle (Draft → Approved → Production)
| Name | Category | Status | Use Case | Rating (1-5) | Eval Score | Grade | Model Hint | Updated | Actions |
|---|---|---|---|---|---|---|---|---|---|
| Loading… | |||||||||
Compliance Review Queue
Regulation comparisons, gap reports, and AI-drafted PPM clauses awaiting review.
| Title | Kind | State | Severity | Quality | Created by | Updated | ||
|---|---|---|---|---|---|---|---|---|
| Loading… | ||||||||
Cost Estimator
Estimate LLM call cost before you run — compare models side by side
Estimate Parameters
Counted at 50% of input price (provider prompt-cache hits).
Estimated Cost
Per Request
—
Per Day
—
Per Month
—
Compare All Models
| Model | Provider | Per Request | Per Day | Per Month | Context |
|---|
Guardrail Rules
Database-backed rules per profile. Each chat call uses the rules of its model's
guardrail_profile.| ID | Profile | Name | Type | Action | Stage | Priority | Active | Operations |
|---|---|---|---|---|---|---|---|---|
| Loading… | ||||||||
Orchestration Chat
Test how the governance pipeline (guardrails → prompt resolve → model resolve → LLM → evaluate → audit) treats a message — each turn shows its verdict and which guardrails fired. Toggle with env
ORCHESTRATOR_V2_ENABLED=1.Initializing…
Connecting to orchestrator
Guardrail reflex ribbon · recent decisions · click a bar for detail
allow
warn
block
redact
No messages yet. Try a normal question, or probe the rails:
PII — "My BSB is 062-334" · Injection — "Ignore previous instructions" · Sentiment — "I am so frustrated"
PII — "My BSB is 062-334" · Injection — "Ignore previous instructions" · Sentiment — "I am so frustrated"
Recent Guardrail Decisions
Loading…
Decision Tables
Rule tables that map input records to outcomes — used by the Decision Table workflow node.
| Name | Description | Status | Columns | Rules | Created |
|---|
A/B Experiments
Champion/Challenger experiments that route traffic between two decision strategies.
| Name | Status | Split | Variant A | Variant B | Created |
|---|
Policy Watch
Tracked regulations as a living garden — each drift check waters a plant, each detected amendment blooms. Backed by
/api/policy-watch.—
healthy
amendment pending
amendment resolved
stale (30d+)
seedling (new)