We had a client whose checkout flow was down for four hours and their status page stayed green the entire time. Not because the monitoring lied on purpose. Because we'd built it to answer the wrong question.
The health check that checked nothing
Their /health endpoint did exactly what most do: it returned 200 OK if the Node process was running and could respond to a request. That's it. No database ping, no check on the payment provider, no verification that the thing users actually cared about — completing a purchase — was possible.
The process was fine. It was up, it was fast, it answered every ping in under 20ms. Meanwhile Stripe webhooks were silently failing because a queue consumer had crashed, orders were stacking up unconfirmed, and every customer hitting "Pay now" saw a spinner that never resolved. The load balancer kept routing traffic to a server that was, by every metric it tracked, perfectly healthy.
This is the trap: a health check that only proves the process didn't crash is really just a liveness check wearing a health check's name. Those are different questions, and conflating them is how you end up debugging a "everything's green but customers are furious" incident at 2am.
Liveness and readiness are not the same thing
Kubernetes actually forces you to separate these — liveness probes restart a dead process, readiness probes decide whether it should get traffic — but most teams that don't run k8s skip the distinction entirely and wire one shallow endpoint to everything: the load balancer, the uptime monitor, the on-call alert. One endpoint, three jobs, none of them done well.
What we changed for that client, and what we now build by default:
A liveness check that's cheap and dumb — did the process respond, yes or no. This is what restarts things.
A readiness check that actually exercises the dependency graph — can we reach the database with a real query, is the queue consumer's last heartbeat recent, did the last three webhook deliveries succeed. This is what should gate traffic and page a human.
The second one is more expensive to run, so you run it less often and you're honest about what "unhealthy" means when it fires — not "the box is on fire," but "we can't currently do the thing this service exists to do."
Why this is worth the extra 45 minutes
Writing a real readiness check takes maybe an hour longer than res.send(200). Teams skip it because the shallow version passes code review and looks the same in a demo. The cost shows up later, and it shows up as the exact scenario above — an outage your own monitoring insists isn't happening, which is worse than no monitoring at all because it actively tells you to stop looking.
This is also the thing we built SiteBleed around: it's not enough to know a URL returned something. We push clients to define what "up" actually means for their business — checkout completes, login succeeds, the API returns real data — and translate downtime against that definition into a revenue number, because "green for four hours during an outage" is a number nobody trusts twice.
If you only take one thing from this: go look at your health check right now. If it would still return 200 with your database unplugged, you don't have a health check. You have a pulse.