Retries, backoff and jitter
Lesson 2 taught you when to stop waiting. Now you try again. Done carelessly, retries turn a short outage into a long one, so this lesson is about trying again gently.
By the end you'll be able to decide which failures to retry, and spread retries out so a recovering server isn't knocked over again. You'll write src/retry.ts in the lab: a backoff formula, a retry budget and the retry loop that Lesson 6 builds on.
Read this first. Retrying a timeout is only safe if the operation is idempotent: doing it twice has the same effect as doing it once. A timeout is an unknown outcome (Lesson 1), so the first attempt may already have worked. That's how the naive policy in Lesson 1 charged 22 of 200 customers twice. This lesson is about how to retry. Whether it's safe to retry a charge is Lesson 4's job. Until then, retry only reads, or calls you know are idempotent.
What a retry can fix
A retry sends the identical request again. So ask: could the identical request succeed a moment later? Marc Brooker of AWS says client errors "should not be retried with the same request because they aren't going to succeed later"[1]. The lab's error classes, in lib/errors.ts, already tell you which is which:
| Error | What happened | Retry? |
|---|---|---|
TimeoutError | you stopped waiting; unknown outcome | yes, but only if the operation is idempotent |
TransientError · 503 | refused before any work (overloaded) | yes, after a wait |
ValidationError · 400 | the request itself is wrong | no: it will fail the same way |
Two more error types arrive in Lesson 4, with idempotency keys. One of them, ConflictError (409, "a request with this key is still in flight"), is also worth retrying after a wait, so include it in your isRetryable now; npm run test:03 checks it.
Retries are selfish
Brooker again: "when a client retries, it spends more of the server's time to get a higher chance of success"[1]. When failures are rare and random, that's a good trade. When the server is failing because it's overloaded, every retry adds load to the thing that's already drowning. And if several layers each retry on their own, the attempts multiply. His advice: "our best practice is to retry at a single point in the stack"[1].
The thundering herd
Picture the moment an outage ends. Hundreds of clients have been failing, and they all try again now. npm run l3 simulates exactly this: 500 clients retry against a server that can serve 50 requests per 10 ms tick, and each 503 it sends burns a tenth of the work of serving a request. Rejections cost real servers work too; the SRE book walks through a backend only 1% over capacity that retries push into crashing[2]. The simulation runs in virtual time with seed 1, so the numbers are reproducible:
policy total calls wasted (503) gave up ms until done
immediate 4722 4648 426 100
exponential 4722 4648 426 4560
full-jitter (yours) 2274 1774 0 260
Requests arriving per tick (first 40 ticks):
immediate █████████▇
exponential ██ █ █ █ █
full-jitter (yours) ██▄▅▂▃▃▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁ ▁ ▁"Immediate" finishes in 100 ms only because 426 clients have used up their 10 attempts and given up by then. That isn't a win.
Why do "immediate" and "exponential" lose exactly the same 426? This simulation has no randomness except the jitter you add: all 500 clients fail on the same tick, latency never varies, and the server starts each tick fresh. So without jitter every wave is the same crowd hitting the same server, with the same result. Backoff only moves the waves further apart in time. That makes these two rows a worst case. Real networks add a little natural spread, so plain backoff does somewhat better than this, but the clusters remain. Note also that the collapse depends on rejections costing work: change the knobs in the challenge and the ranking can change.
Mental model: a crowd at a door that just reopened. If everyone pushes again immediately, the door jams again. If everyone agrees to wait exactly 10 seconds, they all push again together, ten seconds later. Only if each person waits a random time do they trickle through. The model breaks down in one place: jitter doesn't reduce the number of people. If demand stays above capacity for good, no retry policy helps. You need fewer retries (a budget, below) or fewer requests.
Watch the three policies on the same recovery. Press ▶, use the chapters, or turn on 🔇 narration. The bars are the lab's real per-tick arrivals.
Now switch between them yourself. Red bars are ticks where more requests arrive than the server can serve.
Capped exponential backoff, with full jitter
Backoff doubles the wait after each failed attempt, so a struggling server gets more breathing room. A cap stops the wait from growing forever. But backoff alone still synchronises clients: they failed together, so they wait the same time and come back together. Brooker's simulations show that with backoff alone "there are still clusters of calls", and he concludes: "The no-jitter exponential backoff approach is the clear loser"[3].
His fix is full jitter: compute the capped ceiling, then wait a random time between zero and it. Jittered backoff, he writes, "should be considered a standard approach for remote clients"[3]. Full jitter isn't the only jittered option: among the variants he compares, "Equal Jitter" is "the loser", while the choice between "Decorrelated Jitter" and "Full Jitter" "is less clear". Full jitter is the simplest one that works well, so it's the one you'll build. The Google SRE book agrees: "Always use randomized exponential backoff when scheduling retries"[2].
Here is the whole formula, the first thing you'll write in src/retry.ts:
// attempt = 0 before the first retry
ceiling = min(capMs, baseMs × 2^attempt) 1
wait = random() × ceiling 2- Exponential, capped. 10, 20, 40, 80 ms… until the cap. Doubling gives a struggling server more room each time; the cap stops a client sleeping for minutes.
- Full jitter. Any point from 0 to the ceiling, uniformly. Take
randomas a parameter, so tests and the simulation can make it deterministic.
A retry budget
Backoff and jitter spread retries out in time, but they don't limit how many there are. The lab's RetryBudget is a token bucket, like the one Brooker's team uses[1]: it starts with 10 tokens, each retry spends one, and each success earns back 0.1. That allows about one retry per ten successes: plenty for rare blips, and almost nothing when the server is down, which is exactly when retries do the most harm. The tests pin down the details a budget gets wrong most often:
| Rule | Why |
|---|---|
| Spend a token only on an error you would actually retry | A 400 isn't a retry, so it mustn't drain the budget. |
| A retry needs a whole token (≥ 1) | Half a token is not a retry; otherwise the bucket goes negative. |
| Every success refills, including first-try successes | The budget measures health; healthy traffic earns retries. |
| Never refill above the capacity | After a calm hour, an uncapped bucket allows a retry storm. |
| Beware float drift | In JavaScript, 0.1 added ten times is 0.9999999999999999, not 1. Round when you add, or count in whole tenths. |
The retry loop
Your retry(work, options) calls work(attempt) until it succeeds. After a failure, it checks four things in this order, and rethrows the error at the first one that says stop:
catch (e):
① out of attempts? attempt + 1 ≥ maxAttempts → throw e
② can a retry fix it? !isRetryable(e) → throw e
③ does the budget allow it? budget && !budget.tryRetry() → throw e
④ is there time to wait? wait = backoffDelay(attempt)
wait ≥ deadline.remaining() → throw e (Lesson 2)
sleep(wait), then try again
on success: budget.onSuccess(), return the valueWhy this order? ① and ② are free and touch no shared state. ③ spends a token, so it comes after the free checks; if ④ then refuses, that one token is lost, which is acceptable this close to the deadline. ④ uses the deadline from Lesson 2: if the jittered wait would cross it, there's no point sleeping just to fail afterwards. Sleep the same wait you checked, and rethrow the last error itself (the same object, not wrapped in a new Error), so the caller can still tell a timeout from a 503. Lesson 6 relies on that. Run npm run test:03; it's red until all four pieces work.
Practice: from memory
No scrolling up.
A charge call to PayCo times out. Should your retry loop resend it?
The user, the API and the service each make up to 3 attempts at the next hop. How many attempts can one click cause at the database?
Why does exponential backoff without jitter still overload a recovering server?
Full jitter, base 10 ms, cap 100 ms, attempt = 5. How long does the client wait?
A call fails with ValidationError (400). What should retry() do?
Sort the failures
Match the sparklines
Fill the blanks: the backoff formula
function backoffDelay(attempt: number, o: BackoffOptions, random: () => number): number {
const ceiling = Math.(o.capMs,
o.baseMs * ** attempt);
return * ceiling;
}
// base 10, cap 1000, attempt 4:
// the wait is between 0 and msHints: the cap is an upper limit. Each attempt doubles the ceiling. Full jitter uses the whole range from zero.
🏆 Milestone challenge: Retries
Implement src/retry.ts, then use the simulation to make and test a prediction.
cd labs/payflow
npm run test:03 # red until src/retry.ts is done
npm run l3 # default world
cp results/03-retry-storm.json results/03-default.json # the next run overwrites it
CAPACITY=100 npm run l3 # or CLIENTS=…, or REJECT_COST=… (pick ONE knob)- Make it green. Implement
backoffDelay,isRetryable,RetryBudgetandretryuntilnpm run test:03passes. Don't opensolutions/first. - Explain, predict, run. In 3–4 sentences, explain the three rows of the default run: why "immediate" collapses, why "exponential" loses the same 426 clients just later, and why full jitter loses none with fewer calls. Then pick one knob (
CAPACITY,CLIENTSorREJECT_COST) and a new value, and before running, write down whether jitter's advantage (gave up, total calls) will grow or shrink, and why. Run it and compare. - Transfer it. Take an API you use (Stripe, GitHub, OpenAI, your own) and its list of HTTP error codes. Classify six of them as retry / don't retry, with one line of reasoning each. Mark any that are retryable only if the request is idempotent.
Send me back: the npm run test:03 output, both results files, your explanation, your prediction and what actually happened, and the classification table.
Pass criteria (check yourself before sending):
npm run test:03passes (all 15 tests) on your own code.- Your explanation names synchronisation as the reason backoff without jitter fails, and the shrinking capacity (rejections cost work) as the reason immediate retry collapses.
- The prediction was written before the run, the second results file shows the knob you changed, and you explain any difference between prediction and outcome. (If your knob makes "immediate" finish faster than jitter, that's a valid result: explain what changed.)
- The classification marks "the request is wrong" errors (400, 401, 403, 404, 422…) as don't-retry, "try later" errors (429, 503, and a 409 that means "same idempotency key still in flight") as retry-after-a-wait, and timeouts and 5xx errors on non-idempotent calls as unsafe to retry.
When I've reviewed it, I'll write a learning record and unlock Retries.
Read or watch next
Primary source: Exponential Backoff And Jitter, Marc Brooker, AWS Architecture Blog. Short, with the simulations behind full jitter. Compare his charts with the lab's sparklines.
- Exponential Backoff And JitterFull, equal and decorrelated jitter compared by total work and completion time. The source of this lesson's formula.
- Timeouts, retries, and backoff with jitterHow Amazon actually retries: "selfish" retries, retrying at one layer, token-bucket limits. Read the "Retries and backoff" section.
- Addressing Cascading FailuresA worked example of retries crashing a backend, and the checklist: randomised backoff, per-request limits, a server-wide budget.
No high-trust video on retries and jitter could be verified, so this lesson's primary source is an article.
💬 Unsure whether a specific error from your API is safe to retry, or how big your retry budget should be? Ask me in the terminal. Paste the API's error docs and we'll classify them together.