Skip to content

Errors and retries

Audience: partners & internalStatus: reference

The cross-cutting rules behind the status codes on every endpoint. The per-endpoint status tables live in Endpoints and contracts; this page is the general contract they all follow, so the rule lives in one place.

One principle: separate transient from permanent

Every failure at the boundary is one of two kinds, and they are handled oppositely:

  • Transient — the other side is temporarily unavailable or overloaded: 5xx, connection timeout, 429. Retry with backoff. It will probably succeed later.
  • Permanent — the request is wrong and will fail identically on replay: 400 (bad signature or unparseable body), 4xx on a write (a stale cross-reference id, a validation failure). Do not retry. Surface it to a human instead.

Retrying a permanent failure wastes both sides' capacity and delays the alert that a human actually needs to see. Retrying a transient failure is the whole point of the retry machinery.

How we retry you (outbound and webhooks-out)

  • Transient continue — 5xx and timeouts are retried by the next scheduled sync until they succeed; the record shows its last error in the admin UI meanwhile. Creating a record you do not hold yet is a queued job that retries with exponential backoff up to a bounded number of attempts, then surfaces for a human.
  • Permanent stop — a 4xx pauses the record and surfaces it in the admin UI. It stays paused until a human resumes it; we do not keep retrying a request that cannot succeed.
  • At-least-once delivery — a crash mid-delivery means you may receive the same update twice. Your endpoint must be idempotent. If you support an idempotency-key header, tell us its name and we send it.

How you should retry us

  • Webhook to us — 202 means accepted and queued, not processed; do not retry it. 400 is permanent; alert a human, do not retry. 5xx is transient; retry with backoff.
  • Sync / read calls — 429 and 5xx are transient; back off and retry. A 4xx other than 429 is a request problem; fix the request, do not hammer it.

Idempotency, both directions

Doing an operation twice must have the same effect as doing it once — this is required on both sides, because at-least-once delivery is the only delivery guarantee that survives crashes. Our sync trigger takes a lock and is safe to call concurrently; our outbound carries an idempotency key where you support one; your write endpoints must dedupe on their own key or on ours.

Where failures are visible

Every external call — request, response, latency, failure — is logged centrally by the logging service, and per-record sync errors are visible in the owning service's admin UI with single-row re-sync. "Did the call happen, and what came back?" is answerable on our side without asking you.