← claude.html
Code explainer

How our rate limiter works

A token-bucket walkthrough — diagram, the four steps, the actual code, and the things that bite you.

~4 min · written for someone reading it once

01At a glance

Each user has a virtual bucket that holds up to N tokens. Every request takes one token. The bucket refills at a fixed rate. If the bucket is empty when a request arrives, the request is rejected with HTTP 429.

Click Send request to see it.

refill: 1 token / sec Request →
5 / 5 tokens · refilling 1/sec

An HTML artifact can include a real demo. A markdown file can describe one.

02The algorithm in four steps

  1. A request arrives. We look up the user's bucket in Redis (or create one with N tokens if it's the first request).
  2. We refill the bucket: add (now − last_refill) × rate tokens, capped at N.
  3. If the bucket has at least 1 token, we decrement and let the request through.
  4. Otherwise we reject with 429 and a Retry-After header equal to the time until the next token.

03The implementation

Three increasingly correct versions. The first is what most people write, the third is what we actually run.

v1 — naive get/set Broken 
const current = await redis.get(key);
const count = current ? parseInt(current, 10) : 0;
if (count >= MAX) return reject();
await redis.set(key, count + 1, 'PX', WINDOW);
Looks fine. Has a race condition: two requests both read count = 99, both pass the check, both write 100. Under load you let through 2× the limit at every boundary.
v2 — atomic incr Better 
const count = await redis.incr(key);
if (count === 1) await redis.expire(key, WINDOW_S);
if (count > MAX) return reject();
incr is atomic so no more race. But the expire only sets on the first hit — if that call fails between incr and expire, the key is permanent and the user is locked out forever.
v3 — Lua script Correct 
-- INCR + EXPIRE in one atomic operation
local count = redis.call('INCR', KEYS[1])
if count == 1 then
  redis.call('EXPIRE', KEYS[1], ARGV[1])
end
return count
Lua runs as a single Redis command. incr and expire can't separate. Returns the count to the application, which checks against MAX. This is what's in scripts/rate-limit.lua.

04Gotchas

The things that bit us, in order of how much they hurt.