Monitoring

The Inference Stats dashboard shows how your runtime is performing in real time: latency, throughput, error rate, and where time is spent in the pipeline. It updates continuously while a datasource is streaming.

Inference Stats — latency percentiles, throughput, error rate, and execution provider.

Key indicators

• Latency percentiles — p50 / p95 / p99 of end-to-end inference time.
• Throughput — predictions per second.
• Error rate — share of inferences that failed.
• Execution provider — whether inference is running on TensorRT, CUDA, or CPU. A GPU box that falls back to CPU shows up here.

Latency breakdown

Each inference is decomposed so you can see where time goes:

• Queue wait — time spent waiting in the request queue
• Pre-process — input normalization
• Model exec — pure inference time
• Post-process — output decoding

If latency rises, the breakdown tells you whether the model itself slowed down or the box is saturated upstream.

Real-time Monitor — live inference values streaming as a datasource feeds the model.

Window and bucket controls

Two controls shape the time view:

• Window — how far back you look (for example 5m, 1h, 24h).
• Bucket — how wide each point on the chart is (for example 10s, 1m).

The number of points is window ÷ bucket. The dashboard auto-snaps the bucket when you change the window so charts stay readable — wider windows use wider buckets.

How the chart stays steady

The latest point is aligned to a bucket boundary, so the current bucket never shows an artificially low count. Empty buckets are filled with zeros, so the line stays continuous even when traffic is sparse.

Data retention

Live metrics are retained for 3 days. Windows longer than that return no data. A brand-new install needs a short warm-up before the charts show traffic.

Reading a fresh install

If the chart says “no data”, the most common reasons are:

• No datasource is streaming yet — enable and Start a source.
• The install is still warming up — wait for the first window of samples.

Involuntary stops and restart recovery

• Inference stopped and no one pressed Stop? Check Notifications — a stop you didn’t request (another datasource taking over the single active slot, or the active datasource being deleted, retagged, or edited while running) raises a Warning-level alert there instead of failing silently.
• After a backend restart, a previously-enabled input regains its Ready pill and Start control on its own. (v1.28.2) A startup reconciler waits for the inference engine to report healthy, then restores the lifecycle to Ready for the enabled datasource using its real paired model — the in-memory lifecycle otherwise resets to Idle on every process restart while the datasource’s Enabled flag stays persisted. The restore is non-destructive: a transient boot-time connection failure leaves Enabled intact and the lifecycle Idle so you can just re-toggle Enable to retry — it never silently disables a correctly configured datasource.

If something goes wrong

• Empty charts, unexpected CPU fallback, or rising error rate — see the Troubleshooting runbook.

Next steps

Check versions Confirm frontend and backend versions match.

Connect a datasource Start streaming to populate the dashboard.

Preview the pipeline See the Input → Model → Output flow as a live graph.