On-call is the tax everyone pays for running software, and most tools make it heavier than it needs to be: they wake the wrong person, page fifty times for one outage, and hand over a one-line alert with no context at the worst possible hour. 24Observe runs on-call, escalation, and the whole incident lifecycle inside the platform that detected the problem — so the page reaches exactly who is responsible, one outage pages once, and the incident arrives already investigated.
Nobody quits over the occasional 3 a.m. page for a real emergency. People quit over being paged fifty times for one outage, woken for something that was not theirs, and handed a cryptic alert with no context and no help.
The standard on-call setup is a patchwork that fails in predictable ways. Alerts come from several products — the uptime tool, the metrics tool, the security tool — and get funnelled into a separate on-call router whose only job is to decide who to wake. Because that router lives apart from the systems that detected the problem, the page it sends is thin: a title, a severity, a link back to a dashboard you will have to interpret yourself, half-asleep. The hardest part of the night — understanding what is actually wrong — is exactly the part the tooling washes its hands of.
Then there is the volume problem. When one failure cascades, every downstream symptom fires its own alert, and a dumb router faithfully forwards all of them. You wake to a phone that will not stop buzzing, fifty notifications deep, and your first task is not to fix anything but to work out, by hand, that these fifty alarms are one incident. The storm is not just annoying; it actively delays the response, because the signal is buried in its own echoes.
And the routing itself is often crude — page the whole team and hope someone owns it, or page one person whether or not they are even on shift. Diffusion of responsibility means everyone assumes someone else has it; rigid routing means the page goes to someone on holiday. Either way the incident sits unacknowledged while the clock runs.
24Observe treats on-call as part of the platform, not a bolt-on router. Every incident — from a failed check, a metric breach, or a security detection — flows through one lifecycle and one routing system that knows who is genuinely on call right now, climbs an escalation chain if they do not answer, collapses a cascade into a single case so one outage pages once, and arrives carrying the analyst's verdict so the page is a head start, not a cold start. The goal is not just to wake the right person — it is to wake them as kindly as the situation allows.
A page that wakes the wrong person, fifty times, with no context, is how you lose your best engineers. Routing, grouping, and an attached verdict are how you keep them.
Good on-call is mostly about getting the routing right so humans can stop thinking about it. These are the pieces, and they compose into a policy you set once and then trust.
Define who is on call and when; the schedule resolves to exactly one responsible person at any instant, with overrides for when life intervenes.
An ordered sequence that runs until someone acknowledges — primary, secondary, lead, wider channel. A missed page climbs instead of vanishing.
One failure that opens many incidents collapses into a single case with one notification, so a cascade pages you about its cause, once.
Email, webhook, Slack, Discord, Teams, Telegram, PagerDuty, Opsgenie, SMS, and voice — routed by severity so the channel matches the stakes.
Declare planned work and its expected noise does not page anyone, so a deploy or migration does not light up the rotation for changes you already knew about.
Acknowledge, resolve, post updates, and manage schedules through the API; incident.opened/acknowledged/resolved events drive your own automation.
A failed uptime check, a saturated disk, and a brute-force detection are wildly different events, but the work of responding to them is the same shape: someone is told, someone takes ownership, someone understands it, and someone closes it. So they share one lifecycle and one queue. Your responders learn a single way to work an incident and use it for everything, instead of switching mental models between a monitoring "alert" and a security "case." The muscle memory compounds.
When a responder acknowledges, two things happen: the escalation chain stops, so nobody else gets woken, and the incident is now clearly owned, so there is no ambiguity about who has it. That single action is what turns a noisy alarm into a managed incident — and because it is one action in one place, it works the same whether the responder is at a laptop or tapping a notification on their phone at night.
Because incidents live in the platform that detected them, a page is never just a title and a link. It arrives wired to the topology map so the blast radius is visible, and the analyst attaches a root-cause verdict with cited evidence. The most common on-call experience here is acknowledging a page and finding the investigation already done — you are deciding what to do, not discovering what happened. That is the difference between an on-call shift that exhausts people and one they can sustain.
Resolving an incident records what happened and what was done, feeds the look-back, and — where the analyst was involved — lets your correction teach the system to read your environment better next time. The incident does not just disappear; it becomes part of the history that makes the next one easier, and the public status page updates itself to match.
The same cascade that ruins an on-call shift in a patchwork setup — and how it plays out when routing, grouping, and investigation are one system.
It is 3 a.m. A core switch loses power. Within seconds the hosts behind it stop reporting, their services fail their checks, the metrics on those services breach, and the downstream calls start timing out. In a patchwork setup this is the nightmare shift: the on-call engineer's phone erupts with dozens of alerts from three different tools, none ranked, none connected, and the first job of the night is to silence the buzzing long enough to think.
Here, the phone buzzes once. The platform saw that all those incidents trace back to one impacted entity in the context graph and collapsed them into a single case rooted at the switch, so a single page goes out — to A. Rivera, who the on-call schedule knows is actually holding the pager tonight, not to the whole team and not to whoever happens to be listed in a stale config. Rivera acknowledges in under a minute, and the escalation chain stops there; the secondary and the team lead sleep through it, because the system that would have escalated saw the acknowledgement.
Rivera opens the case to find it is not a cold start. The analyst has already investigated: the verdict says the switch suffered a power fault, the fourteen downstream hosts are unreachable as a consequence, and the customer-facing checkout and payments services are at risk — with the supporting evidence cited. The topology map shows the same story as a picture. Rivera did not spend the first twenty minutes establishing that it is the switch and figuring out who is affected; that work was waiting for them when they woke up.
So the night becomes a decision instead of an investigation. Rivera dispatches someone to the switch, posts the affected components to the status page — which the case already identified — and the customers who are watching get an honest update without Rivera having to compose one from scratch. When the switch is restored, the hosts report, the services clear, the grouped case resolves in one action, and the status page follows.
In the morning the whole episode is one coherent record: one case, one page, one acknowledgement, one resolution, and a verdict that explains the root cause — rather than a scattered trail across three tools and fifty alarms that someone has to reconstruct for the post-mortem. The switch failure was unavoidable. The miserable night was not, and the platform is the reason it did not happen.
The deeper point is what this does to a team over months, not minutes. On-call is sustainable or it is not, and what makes it unsustainable is rarely the number of real emergencies — those are few — but the accumulation of bad nights: the floods, the wrong-person pages, the cold-start investigations at 3 a.m. Each one is a small withdrawal from your engineers' goodwill, and enough of them is how you lose the people you most want holding the pager. A setup that pages the right person once, with the investigation attached, makes the withdrawals far smaller and far rarer.
That is the real return on treating incidents as a first-class part of the platform rather than bolting a router onto the side. The routing, the grouping, the escalation, and the attached verdict are not separate features you assemble; they are one coherent answer to the question "how do we tell the right human about the right problem, kindly, with help already in hand." Get that answer right and on-call stops being the thing your best engineers dread and quietly plan their exit around — which, over a year, is worth more than any single outage it helps you resolve.
One incident lifecycle for checks, metrics, and detections — with rotations, escalation, root-cause grouping, ten channels, and an analyst that investigates before you wake.