The observability layer

Every decision Lupid makes lands somewhere queryable. There’s no concept of an unrecorded call.

The audit log

The primary audit store is an append-only audit_events table in Postgres. Schema columns (the ones you’ll care about as an operator):

Column	Purpose
event_id	UUID for the event
event_type	Discriminator (e.g. tool_call, auth_failed, policy_denied)
ts	When the event occurred (UTC)
agent_id	The agent that triggered it
session_id	The session within the agent
actor	The decision-maker (policy id, operator email, etc.)
tool	Tool name for tool-call events
resource	Resource the agent was touching
outcome	allow / deny / redact / require_hitl / etc.
policy_hash	SHA-256 of the policy that ran (proves reproducibility)
risk_score	Numeric severity
data_classes	Detected DLP classes for the prompt + response
detail	Free-form JSON for event-type-specific fields
trace_id	Joins with OpenTelemetry traces
sequence_no	Monotonic counter for gap detection

The dashboard’s audit page renders this as a three-pane drill-down: timeline → event detail → raw JSON. You can also run agentum audit query --agent-id <id> --limit 50 from the CLI for the same data piped through your usual JSON tools.

Optional ClickHouse warehouse

Postgres works for most deployments. If you’re at high enough event volume that the audit_events table is hot, enable the ClickHouse profile:

CLICKHOUSE_PASSWORD=$(openssl rand -hex 16) docker compose --profile audit up -d

The audit pipeline auto-detects AGENTUM__CLICKHOUSE_URL and writes new events to ClickHouse instead. The schema is mirrored; queries from the dashboard and CLI work unchanged.

Note

The audit retention engine and integrity-signing pipeline are live but their configuration surface is minimal in the opensource build. Retention defaults to 365 days; integrity batches sign every N events with an Ed25519 key.

The dashboard

Three panes the operator will spend most of their time in:

• Audit (/audit) — live SSE-streamed tool calls with filters by agent, severity, outcome, data class. The filters compile to the same query the CLI uses.
• Alerts (/alerts) — every alert with deduplication: equal dedup_key values collapse to a single row with occurrence_count. A dead-letter queue holds alert deliveries that failed (e.g. your webhook returned 500). Bulk-ack, bulk-resolve.
• HITL (/hitl) — the human-approval queue. Each row shows the agent, tool, resource, the reason from the policy’s @advice annotation, and Approve / Deny buttons. Multi-approver flows show how many approvals are still needed.

Live updates over SSE

The dashboard subscribes to three Server-Sent Events streams: audit events, alerts, and HITL requests. Each event arrives in the browser within a few hundred milliseconds of being written. There’s no polling.

If a subscriber disconnects (laptop sleeps, network drops), the dashboard reconnects automatically and replays from the last sequence_no it saw.

OpenTelemetry

Set OTLP_ENDPOINT and agentum start exports traces over OTLP gRPC. Span tags include agent.id, event.type, policy.hash, and decision.outcome, so you can pivot from a Jaeger / Tempo / Datadog trace straight into the Lupid audit row by trace_id.

Prometheus metrics

The API exposes /metrics (Prometheus format) on :7071. The names registered today are:

Metric	Type	Purpose
agentum_agent_registrations_total	counter	Agents registered
agentum_sessions_started_total	counter	Agent sessions started
agentum_audit_write_failures_total	counter	Failed audit-event writes (Postgres or ClickHouse)
agentum_audit_integrity_anchor_success_total	counter	Successful audit-batch integrity anchor
agentum_audit_integrity_anchor_failure_total	counter	Anchor failures
agentum_audit_integrity_anchor_skipped_total	counter	Anchors skipped (config or quota)
agentum_mcp_tool_calls_observed_total	counter	MCP tool-calls seen at the gateway
agentum_mcp_dedup_hits_total	counter	MCP dedup cache hits
agentum_dlp_mcp_request_findings_total{kind}	counter	DLP findings on MCP request bodies, by detector kind
agentum_dlp_mcp_response_findings_total{kind}	counter	DLP findings on MCP response bodies
agentum_incident_snapshot_retention_deleted_total	counter	Incident snapshots purged by retention
agentum_incident_snapshot_retention_failures_total	counter	Retention sweep failures
agentum_cold_scan_files_total	counter	Cold-storage scanner files processed
agentum_gateway_classify_duration_ms	histogram	Gateway request classify latency
agentum_mcp_classify_duration_ms	histogram	MCP request classify latency
agentum_revocation_lag_seconds	gauge	Time since the last revocation poll

Metrics is unauthenticated on the loopback interface by default. Set METRICS_BEARER_TOKEN to require a bearer token if you’re reverse-proxying it.

Note

This table is the current source-of-truth snapshot from crates/agentum-*/src/**/*.rs. New metrics are added in regular releases — to see the live list, hit /metrics and grep ^agentum_.

Alert webhooks

Alerts can fan out to any HTTP URL you register. Payloads are HMAC-SHA256 signed in an X-Agentum-Signature header so the receiver can verify authenticity. See Webhooks for the payload shape and signing algorithm.

The webhook dispatcher has a dead-letter queue capped at 256 entries. If your receiver was down, undelivered alerts live in the DLQ and you can re-queue them from the alerts page.