Build the mental model
Any service that accepts requests from external clients needs a defense against one client — malicious or simply buggy — sending requests fast enough to degrade performance for everyone else sharing the same backend. Rate limiting caps how many requests a client can make in a given time window. The token bucket algorithm is a popular way to implement this because it naturally balances two competing needs: allowing legitimate short bursts of activity (a user rapid-clicking through a few pages) while still enforcing a strict long-run average rate. The bucket holds up to a fixed capacity of tokens; tokens refill continuously at a fixed rate (e.g. 2 per second); each incoming request must consume one token to proceed. If the bucket is full, a burst of requests can all be served immediately, up to capacity. Once it's empty, requests are throttled to arrive no faster than the refill rate. This is a meaningfully different shape than a naive fixed-window counter (e.g. 'max 100 requests per minute'), which allows a client to send 100 requests in the last second of one window and 100 more in the first second of the next — a 200-request burst the token bucket would smooth out.
Connect it to a real scenario
As Thuta Learning's API opens up to third-party integrations — a partner app that syncs lesson progress, a browser extension that shows quiz results — the platform can no longer trust every caller to be well-behaved. A single misconfigured integration polling the lessons endpoint every 10 milliseconds instead of every 10 seconds could saturate the database and slow the site down for every real student browsing tutorials at the same time. By putting a token bucket in front of the API — say, 20 requests burst capacity refilling at 5 per second per API key — the platform lets normal usage patterns (a page loading several resources at once) through instantly, while a runaway script gets throttled down to a sustainable rate instead of taking the whole platform down.
Try the working example
import time
class TokenBucket:
def __init__(self, capacity: int, refill_rate: float):
self.capacity = capacity # max tokens the bucket can hold
self.refill_rate = refill_rate # tokens added per second
self.tokens = capacity # start full
self.last_check = time.monotonic()
def _refill(self):
now = time.monotonic()
elapsed = now - self.last_check
self.tokens = min(self.capacity, self.tokens + elapsed * self.refill_rate)
self.last_check = now
def allow_request(self) -> bool:
self._refill()
if self.tokens >= 1:
self.tokens -= 1
return True
return False
bucket = TokenBucket(capacity=5, refill_rate=2) # burst of 5, refills 2/sec
# Simulate 8 rapid requests with no delay (burst)
results = [bucket.allow_request() for _ in range(8)]
print("Rapid burst of 8:", results)
# Wait for tokens to refill, then try again
time.sleep(1.5)
print("After 1.5s wait:", bucket.allow_request())The script shows the first 5 rapid requests succeeding (draining the full bucket), the next 3 being rejected, and then a request succeeding again after waiting 1.5 seconds lets the bucket refill 3 tokens.5-minute try-it
Modify the TokenBucket class to track and print how many requests were rejected in total. Then experiment: with capacity=10 and refill_rate=1, simulate 15 requests sent instantly — how many succeed, and why does that number make sense given the parameters?
One important caution
Using wall-clock time.time() instead of a monotonic clock for refill calculations — if the system clock jumps backward (NTP adjustment), elapsed time can go negative and the bucket logic breaks; time.monotonic() is immune to clock adjustments.
Forgetting to cap tokens at capacity after refilling — without min(self.capacity, ...), a client that stays idle for a long time accumulates unlimited tokens and can then unleash an enormous burst the rate limiter was supposed to prevent.
Wikipedia — Rate limiting — System Design