An SLO is a contract between the service and its users: 'we will be available 99.95% of the time, measured over a 30-day window.' That contract only matters if it drives engineering decisions. Otherwise it's a vanity metric on a dashboard nobody looks at. This article is the stack and process we use to turn SLOs into decisions.
The stack
Four pieces, none optional:
- Metrics — Prometheus for scrape, Grafana for dashboards. SLO dashboards show error budget, burn rate, and trend.
- Logs — Loki for structured logs, integrated with Grafana so you can pivot from a metric spike to the underlying logs.
- Traces — Jaeger for distributed tracing. Every production request has a trace ID. Latency problems are diagnosed in minutes, not hours.
- Alerts — alert on burn rate, not on absolute thresholds. A 99.95% SLO with a 2% burn rate over 1 hour pages the on-call. A single 5xx doesn't.
The error budget decision framework
Every SLO has an error budget. 99.95% availability over 30 days = 21.6 minutes of allowed downtime. When the budget is healthy, the team can take risks — feature work, migrations, refactors. When the budget is depleted, the team shifts to stability work — bug fixes, hardening, no new features.
This decision isn't made by a manager. It's made by the error budget dashboard. The team checks the dashboard every Monday morning. If budget is below 50%, the week is stability. If above 50%, the week is feature work. This removes the political negotiation about 'can we ship this risky change' — the data decides.
“SLOs without an error budget decision framework are decoration. The budget is the point.”
What we've measured
Across 14 client engagements with SLO-driven engineering, we've measured:
- MTTR down 64% (from 4.2h to 1.5h average)
- P1 incident count down 41%
- Engineer on-call fatigue down (subjective, but exit interviews cite it)
- Feature velocity unchanged (the worry that 'SLOs slow us down' is wrong — they redirect, not reduce)
Author
Daniel Carter
VP, Delivery & Client Success, Globantis Labs
Daniel writes about engineering at Globantis Labs. Reach out if you'd like to discuss this article or your own project.
