← Writing

Check the state, not the clock

“Wait five minutes, then assume it arrived” is one of the most common bugs I see. It can't tell broken from idle.

2 min read

Almost every system that waits for something has a line like this somewhere:

if now() - started_at > timedelta(minutes=5):
    assume_done()

A file should have landed by now. A job should have finished. A transaction should be confirmed. So after five minutes, carry on as if it has.

It works most of the time, which is why it survives. But elapsed time is a proxy for the thing you care about, and it fails both ways.

  • On a slow day, it lies. The upstream is late, the five minutes pass, and the system goes ahead with data that hasn’t arrived. Nothing errors. The numbers are just wrong.
  • On a broken day, it can’t tell. If the upstream has stopped completely, the check looks exactly like a quiet period. “Nothing new for five minutes” means healthy idleness and total failure equally.

The fix that makes it worse

When a timer like this fires falsely, the usual response is to widen it. Five minutes becomes fifteen. The false alarm goes away.

But you haven’t fixed anything. You’ve moved the error. The check now waits longer before going ahead with missing data, and stays quiet longer when something has really broken. A false positive has become a false negative, and false negatives don’t page anyone.

Ask the question you meant

Behind every timer there’s a real question, and it can almost always be asked directly.

  • Not “has it been five minutes since the batch started?” but “does the downstream count match the upstream count?”
  • Not “has the transaction had time to confirm?” but “is there a receipt for this hash?”
  • Not “is the feed older than ten minutes?” but “has the feed’s own watermark moved since the last check?”

These are state checks. They compare two things that should agree, or look for the thing you’re waiting for. They’re right on slow days, they’re loud on broken days, and they don’t need tuning when traffic changes.

In MonkePay, a payment is recorded when its transfer exists on-chain, not after a delay. If the receipt isn’t there yet, the backend asks again. Nothing about it depends on how fast Base happens to be that day.

When you do need time

Time still has a place: as a deadline for escalation, not as evidence. “This should have matched by now, so tell a person” is a fine use of a clock. “It has been long enough, so it must be fine” isn’t.

If you find a threshold that keeps being adjusted, it’s usually a sign that the real condition was never written down. Find that condition and check it instead.