Most data teams we meet have an on-call rotation that grew by accident. Someone set up Slack alerts, someone else added PagerDuty for the finance dashboards, and now one person a week sleeps badly while most alerts turn out to need nothing.
Decide what is urgent
Start by listing the tables that someone outside the data team reads before 10 a.m. For most companies that is ten to twenty tables, not hundreds. Those, and only those, should be able to page anyone at night.
- Page for tables with a promise, such as “ready by 7:00”.
- Post to Slack for everything else, grouped by cause.
- Send a daily digest for warnings that can wait.
Give every alert an owner
An alert in a shared channel is an alert for nobody. Put owners in your dbt meta tags or CODEOWNERS file, and route incidents to them. The on-call person becomes a backstop rather than the first responder for everything.
Fewer, better alerts beat faster responses to bad ones.
Make the morning handover short
We suggest a five-minute handover at the start of each day: what fired overnight, what was fixed, what is still open. With incidents grouped and fixes proposed as pull requests, this is usually a list of two or three items to review.
Measure it
Track pages per week and the share that needed action. Teams we work with aim for fewer than three pages a week with more than 80% actionable. If you are far from that, the problem is the alerts, not the people.
