Observability

Observability#

SkyPortal can expose OpenTelemetry request metrics in Prometheus format. It is opt-in and off by default.

Enabling metrics#

In your config.yaml:

observability:
  enabled: True
  service_name: skyportal

This instruments the Tornado app and serves metrics at /api/internal/metrics, which requires a System admin token (scrape with Authorization: token <id>). Each app worker serves its own metrics on its internal port (ports.app_internal + N, default 65000..65003); scrape those directly rather than the nginx/public endpoint, which round-robins workers and returns partial counters.

Prometheus + Grafana#

The observability/ directory ships a ready-to-run stack: a scrape config, Grafana provisioning, and a SkyPortal Overview dashboard. Copy observability/prometheus/skyportal_token.example to skyportal_token, add a System admin token, then start it alongside the app:

docker compose -f docker-compose.yaml \
  -f observability/docker-compose.observability.yaml up

Grafana is then at http://localhost:3000 (admin/admin by default).

Contents