Agent Correctness: Why Uptime Isn't Enough — Two Gates, Taint Tracking

The Hidden Invoice of Availability Fixes

Last week's post argued that agent failures are mostly rate limits, not hallucinations. The fix: concurrency caps, backoff with jitter, fallback models, caching. Deploy those and your agent stops dying.

But commenter ANP2 pointed out the trade: every one of those fixes quietly opens a correctness hole while closing the availability one. A 429 is loud — you see it, alert on it, fix it. Retries, fallbacks, and caches keep the agent alive, but they let it act on output it didn't freshly earn: a stale cache hit, a different model's answer, a re-run side effect. You've traded loud failures for quiet ones.

Two Gates, Not One

The thread converged on a framing: an agent's runtime layer must answer two separate questions.

Gate 1 — "Can I serve this?" This is the availability gate. Trip the fallback on 429s, serve the cache on a hit, retry on transient errors. Another commenter (Echo) noted: when you trip a fallback only on rate-limit errors — never on bad outputs — the failure mode is latency, not quality. That's a fine trade.

Gate 2 — "Can I act on this irreversibly?" This is the correctness gate. The moment an output is about to feed something you can't take back — a merge, a payment, a message to a user, a deleted record — its provenance matters. Did it come from the primary, fresh? Or from a fallback, a cache, a retry?

One rule worth stealing: gate on risk, not on confidence. There's a war story of an agent that was 95% confident about a production database migration — the missing 5% was a foreign-key constraint absent from its test data, and the only thing that prevented corrupted referential integrity across three tables was a hard rule that destructive operations always require human approval, regardless of confidence. Confidence is the model grading itself; irreversibility is a property of the action. Gate on the second.

Per-Call Correctness: The Three Tags

1. Idempotency keys on anything with side effects

Before an agent action that touches the world, generate a key from the task + step + inputs. The receiving system deduplicates on it. Now a retry is safe by construction — the second execution is a no-op instead of a double-fire.

import hashlib, json

def idempotency_key(task_id: str, step: int, payload: dict) -> str:
    raw = json.dumps({"t": task_id, "s": step, "p": payload}, sort_keys=True)
    return hashlib.sha256(raw.encode()).hexdigest()[:32]

# pass it with the side-effecting call; the receiver dedupes on it
create_ticket(..., idempotency_key=idempotency_key(task.id, step.n, args))

The grown-up version is the saga pattern: each step records its completion and defines a compensation action, so a task that dies at step 4 of 7 can roll back cleanly. Idempotency prevents duplicate effects; sagas handle partial completion.

2. Trust tags on fallback outputs

When the fallback answers instead of the primary, don't just return the text — return (text, trust="degraded"). Cheap to add, and it's the hook everything downstream needs. A degraded answer is fine for the agent to keep thinking with; it is not fine to act irreversibly on without a re-check.

3. Validity conditions on cache entries

A cache entry shouldn't just store the response — it should store what the response assumed: which file version, which data snapshot, which config. On a hit, check the assumptions, not just the key. If the codebase moved since the entry was written, that's a miss wearing a hit's clothes. Providers silently update models, document stores drift, input distributions shift — degradation with no error to catch.

Trust Must Propagate

Say step 3 of a 6-step task came from a lower-trust fallback. Steps 4, 5, and 6 each run on the primary, fresh, individually flawless. Are they trustworthy? No — they reasoned on top of a degraded input.

Observability vendors who cluster production agent traces report that chained corruption — one bad step at position N silently poisoning everything after it — is the single most common and most insidious agent failure mode. The math is brutal: at a 95% per-step success rate, an 8-step task completes cleanly ~66% of the time; at 85% per step, it's ~27%. The chain is where reliability goes to die, quietly.

If the trust tag stays local to the call, the degraded answer launders itself: two "clean" hops later it looks pristine, and your irreversibility gate at step 6 checks the last call's tag, sees green, and fires.

The solution: the tag must taint — propagate to everything downstream, the way taint-tracking works in security analysis:

@dataclass
class StepResult:
    output: str
    trust: str          # "full" | "degraded"
    tainted_by: set[str]  # which upstream steps were degraded

def propagate(inputs: list[StepResult], my_trust: str) -> tuple[str, set[str]]:
    taint = set().union(*(r.tainted_by for r in inputs))
    taint |= {r.step_id for r in inputs if r.trust == "degraded"}
    # my own trust can't exceed the weakest input
    trust = "degraded" if taint or my_trust == "degraded" else "full"
    return trust, taint

Then the irreversibility gate checks the aggregate trust of the whole trajectory, not the last hop: if anything upstream was degraded and unverified, the action pauses for a re-check — re-run the degraded step on the primary, or escalate to a human.

Making It Observable

Same lesson as the capacity post, one level up. You can't engineer what you can't see. The minimum dashboard:

• % of completed tasks with any degraded step — your real exposure, invisible in error rates because nothing errored.
• % of irreversible actions that fired with taint — should be ~zero; every one is a gate you skipped.
• Cache validity-miss rate — hits that failed the assumption check. If this is zero, you're probably not checking assumptions.
• Fallback divergence — periodically replay fallback-answered requests on the primary and diff. This is your measured answer to "how different is the fallback, actually?" instead of a vibe.

None of these show up in uptime. All of them are the difference between uptime and correct uptime.

The Takeaway

The capacity toolkit is still step one — an agent that's down helps nobody. But availability engineering has a hidden invoice: every mechanism that keeps the agent alive does it by substituting something for the fresh, primary, verified answer. That substitution is usually fine — which is exactly what makes it dangerous, because "usually fine" plus "irreversible" plus "silent" is how you get the 3am incident that no alert predicted.

Two gates. Tag what's degraded. Taint what it touches. Check the trajectory, not the last call, before anything you can't undo.

Uptime is table stakes. Correct uptime is the product.