Monitoring¶

An instance reports its health, its build, the state of each hub and events that need a person. It uses two small JSON endpoints, Prometheus metrics and its log.

Health and version¶

GET /healthz

200 when the process is live and the database is ready, 503 when the database is not. Use it for container health checks and uptime monitoring.

GET /version

The running build, for example {"version":"v0.1.0+ac65974"}. Check it instead of trusting a floating image tag.

Metrics¶

GET /metrics serves Prometheus text format:

status is one of the five hub statuses. hub_id is the hub's number in this instance's database, given in the order the instance first followed the hubs. It is not necessarily the hub_id in the Hub-List.

Alerts to set:

• Any increase of iscc_monitor_violations_total, or iscc_monitor_hub_status{status="frozen"} == 1. A hub rewrote its history, and a person must look at the evidence.
• iscc_monitor_hub_status{status=~"unresolvable|unverified"} == 1 for longer than a few polls.
• time() - iscc_monitor_last_observed_at well above the poll interval.
• A rising rate of iscc_monitor_poll_failures_total.

Public or private¶

Metrics carry no secrets, but they are operator telemetry, not evidence. The monitor has one listener for all routes, so you decide at the reverse proxy whether /metrics is public. The Caddy example refuses it.

Log¶

The monitor writes one line per event to standard error, in logfmt (key=value). The lines to know: