Per-Tenant Rate Limiting: Containing Noisy Neighbors in a Shared API

When one customer's script floods a shared API, everyone else's requests wait in the same queues for the same workers. Latency climbs for customers who did nothing different, and the first sign is often a support ticket from a tenant who has never heard of the one causing it.

The usual first fix, a global rate limit, spreads the pain instead of containing it. This guide explains why the limit has to be keyed on the tenant, which algorithm to use, what the response should tell clients, and how to decide what happens when the limiter itself is unavailable. A lab compares no limit, a global limit, and per-tenant token buckets under the same flood.

Why limits keyed on IP address do not work for business tenants

IP-based limits are reasonable for anonymous traffic such as login attempts. They break down for business customers in two directions:

• Many people, one address. Employees of one company often reach you through a single corporate proxy or NAT gateway. A limit sized for one client throttles the whole company when one employee's script misbehaves.
• One tenant, many addresses. A job running across a pool of cloud functions or containers arrives from many addresses and slips under a per-address limit.

The unit you want to protect is the paying tenant, so key the primary limit on the tenant ID from authenticated context: the API key's owner, the session's organization, or the token's tenant claim. Never take it from a request header the client controls. Add narrower limits per user or per API key where one integration should not consume the whole tenant's allowance, and keep IP limits for unauthenticated endpoints.

AWS describes the same problem from the platform side in the SaaS Lens noisy-neighbor guidance, including throttling by tenant tier.

Choosing an algorithm

Four algorithms cover almost every rate limiter. They differ in how they handle bursts and how much state they keep.

The fixed-window weakness is easy to underestimate, so the lab below measures it directly. Stripe's engineering post Scaling your API with rate limiters describes running token-bucket request limiters alongside concurrency limiters and load shedders, which is a useful model: rate limits alone do not protect you from a small number of very slow requests.

Lab: no limit, a global limit, and per-tenant buckets

The program simulates an API with 10 workers and a 20 ms service time, so it can serve 500 requests per second. Twenty ordinary tenants each send 10 requests per second. One tenant runs a runaway script at 2,000 requests per second. Clients give up after 2 seconds. Requests that arrive in the same millisecond are shuffled with a fixed seed, so no tenant always wins ties and every run prints the same result.

The three admission policies are:

• none: every request joins the queue;
• global: one token bucket for the whole API, 500 per second with a burst of 500;
• per-tenant: one bucket per tenant, 20 per second with a burst of 40, rejecting with HTTP 429 when empty.

The second part sends 100 requests just before and 100 just after a minute boundary to compare a fixed 60-second window with a token bucket of the same nominal limit. Run it with go run .; it needs only the standard library.

// Lab: what a per-tenant token bucket changes when one tenant floods an API.
//
// A simulated API has 10 workers and each request takes 20 ms (capacity
// 500 req/s). Twenty ordinary tenants send 10 req/s each; one tenant runs
// a runaway script at 2,000 req/s. Clients abandon a request after 2 s.
// Simulated time in 1 ms steps; arrivals within a millisecond are shuffled
// with a fixed seed so no tenant is always first in line.
package main

import (
	"fmt"
	"math"
	"math/rand/v2"
	"sort"
)

type bucket struct {
	tokens, capacity, refillPerMs float64
	last                          int
}

// allow mirrors the Lua script: refill by elapsed time, then spend `cost`.
func (b *bucket) allow(now int, cost float64) bool {
	b.tokens = math.Min(b.capacity, b.tokens+float64(now-b.last)*b.refillPerMs)
	b.last = now
	if b.tokens >= cost {
		b.tokens -= cost
		return true
	}
	return false
}

type req struct {
	tenant  string
	arrived int
}

type stats struct {
	sent, ok, rejected, abandoned int
	lat                           []int
}

func run(mode string) map[string]*stats {
	const (
		workers, serviceMs, horizon, giveUpMs = 10, 20, 10_000, 2_000
	)
	st := map[string]*stats{}
	buckets := map[string]*bucket{}
	global := &bucket{tokens: 500, capacity: 500, refillPerMs: 0.5}
	var queue []req
	freeAt := make([]int, workers)
	rng := rand.New(rand.NewPCG(7, 7)) // fixed seed: same arrival order every run

	tenants := []string{"noisy"}
	for i := 0; i < 20; i++ {
		tenants = append(tenants, fmt.Sprintf("t%02d", i))
	}
	for _, t := range tenants {
		st[t] = &stats{}
		buckets[t] = &bucket{tokens: 40, capacity: 40, refillPerMs: 0.02} // 20 req/s, burst 40
	}

	for now := 0; now < horizon; now++ {
		// arrivals: noisy sends 2 per ms, others send one every 100 ms (offset per tenant)
		var arriving []req
		arriving = append(arriving, req{"noisy", now}, req{"noisy", now})
		for i, t := range tenants[1:] {
			if (now+i*5)%100 == 0 {
				arriving = append(arriving, req{t, now})
			}
		}
		rng.Shuffle(len(arriving), func(i, j int) { arriving[i], arriving[j] = arriving[j], arriving[i] })
		for _, r := range arriving {
			s := st[r.tenant]
			s.sent++
			admitted := true
			switch mode {
			case "global":
				admitted = global.allow(now, 1)
			case "per-tenant":
				admitted = buckets[r.tenant].allow(now, 1)
			}
			if !admitted {
				s.rejected++ // HTTP 429 returned immediately
				continue
			}
			queue = append(queue, r)
		}
		for w := range freeAt {
			for freeAt[w] <= now && len(queue) > 0 {
				r := queue[0]
				queue = queue[1:]
				if now-r.arrived > giveUpMs {
					st[r.tenant].abandoned++
					continue
				}
				st[r.tenant].ok++
				st[r.tenant].lat = append(st[r.tenant].lat, now-r.arrived+serviceMs)
				freeAt[w] = now + serviceMs
			}
		}
	}
	for _, r := range queue { // still waiting at the end of the run
		if horizon-r.arrived > giveUpMs {
			st[r.tenant].abandoned++
		}
	}
	return st
}

func pct(xs []int, p float64) int {
	if len(xs) == 0 {
		return -1
	}
	s := append([]int(nil), xs...)
	sort.Ints(s)
	return s[int(math.Ceil(p*float64(len(s))))-1]
}

func main() {
	fmt.Println("10 s simulation, capacity 500 req/s; noisy tenant 2,000 req/s; 20 ordinary tenants x 10 req/s")
	fmt.Printf("%-12s | %-38s | %s\n", "admission", "ordinary tenants (combined)", "noisy tenant")
	for _, mode := range []string{"none", "global", "per-tenant"} {
		st := run(mode)
		var o stats
		for t, s := range st {
			if t == "noisy" {
				continue
			}
			o.sent += s.sent
			o.ok += s.ok
			o.rejected += s.rejected
			o.abandoned += s.abandoned
			o.lat = append(o.lat, s.lat...)
		}
		n := st["noisy"]
		fmt.Printf("%-12s | served %5.1f%%, 429 %5.1f%%, p99 %5d ms | served %5.1f%%, 429 %5.1f%%\n", mode,
			100*float64(o.ok)/float64(o.sent), 100*float64(o.rejected)/float64(o.sent), pct(o.lat, 0.99),
			100*float64(n.ok)/float64(n.sent), 100*float64(n.rejected)/float64(n.sent))
	}

	// Fixed window vs token bucket at a window boundary: limit 100 per 60 s.
	fmt.Println("\nBurst at a window boundary (limit: 100 requests per 60 s)")
	fixedAllowed := 0
	windowCount := map[int]int{}
	tb := &bucket{tokens: 100, capacity: 100, refillPerMs: 100.0 / 60000}
	tbAllowed := 0
	for _, at := range []int{59_000, 61_000} { // client sends 100 requests at 59 s and 100 more at 61 s
		for i := 0; i < 100; i++ {
			w := at / 60_000
			if windowCount[w] < 100 {
				windowCount[w]++
				fixedAllowed++
			}
			if tb.allow(at, 1) {
				tbAllowed++
			}
		}
	}
	fmt.Printf("fixed 60 s window : %d of 200 allowed within 2 s\n", fixedAllowed)
	fmt.Printf("token bucket      : %d of 200 allowed within 2 s (capacity 100, refill 100/60 s)\n", tbAllowed)
}

Output recorded on November 3, 2026 (Go 1.26.5, windows/amd64):

10 s simulation, capacity 500 req/s; noisy tenant 2,000 req/s; 20 ordinary tenants x 10 req/s
admission    | ordinary tenants (combined)            | noisy tenant
none         | served  24.4%, 429   0.0%, p99  2020 ms | served  22.6%, 429   0.0%
global       | served  17.2%, 429  81.0%, p99  1011 ms | served  23.3%, 429  74.4%
per-tenant   | served 100.0%, 429   0.0%, p99    34 ms | served   1.2%, 429  98.8%

Burst at a window boundary (limit: 100 requests per 60 s)
fixed 60 s window : 200 of 200 allowed within 2 s
token bucket      : 103 of 200 allowed within 2 s (capacity 100, refill 100/60 s)

Reading the results

With no limit, the ordinary tenants had only 24.4% of their requests served, and those that were served waited up to the 2-second give-up point. They were sharing a queue that was four times oversubscribed.

A global limit was worse for ordinary tenants than no limit at all. It rejected 81% of their requests, because a global bucket hands out tokens to whoever asks, and the noisy tenant asks a hundred times more often. Their p99 latency dropped to about a second only because so few requests got in. A global limit protects the servers; it does not protect the customers.

Per-tenant buckets served 100% of ordinary traffic with a p99 of 34 ms, and rejected 98.8% of the runaway script's requests immediately with 429. The noisy tenant still got its 20 requests per second.

The boundary test shows the fixed-window problem: all 200 requests were accepted within two seconds under a "100 per minute" window, because they straddled two windows. The token bucket accepted 103: its 100-token burst plus the tokens refilled over those two seconds.

The model is simple on purpose. Real request costs vary, real clients retry, and real servers degrade before they saturate. Treat the numbers as an illustration of the mechanism and run your own traffic replay before choosing limits.

A Redis token bucket for several API servers

When several API servers share one limit, the bucket has to live in shared storage and the check-and-update has to be atomic. Reading the count, deciding in application code, and writing it back lets two servers spend the same token. Redis runs a Lua script atomically (see Scripting with Lua), which makes it a common place for the bucket.

The script below follows the same arithmetic as the lab's allow method. It reads the clock from Redis with TIME so that clock differences between your API servers do not change the refill; since Redis 7.0, scripts are always replicated by their effects, so this is allowed. We have not executed this script against a Redis server as part of the lab, so test it with your Redis version before use.

-- KEYS[1]  bucket key, e.g. rl:{tenant_id}:reads
-- ARGV[1]  capacity (maximum burst)
-- ARGV[2]  refill rate in tokens per second
-- ARGV[3]  cost of this request
local capacity = tonumber(ARGV[1])
local rate     = tonumber(ARGV[2])
local cost     = tonumber(ARGV[3])

local t   = redis.call('TIME')                       -- {seconds, microseconds}
local now = tonumber(t[1]) + tonumber(t[2]) / 1e6

local state  = redis.call('HMGET', KEYS[1], 'tokens', 'ts')
local tokens = tonumber(state[1]) or capacity
local ts     = tonumber(state[2]) or now
tokens = math.min(capacity, tokens + math.max(0, now - ts) * rate)

local allowed, wait = 0, 0
if tokens >= cost then
  tokens  = tokens - cost
  allowed = 1
else
  wait = (cost - tokens) / rate                       -- seconds until enough tokens
end

redis.call('HSET', KEYS[1], 'tokens', tokens, 'ts', now)
redis.call('PEXPIRE', KEYS[1], math.ceil(capacity / rate * 1000) + 1000)
return {allowed, math.floor(tokens), math.ceil(wait)}

The expiry is the time a bucket needs to refill completely, plus a second. After that, a missing key and a full bucket mean the same thing, so idle tenants cost no memory.

Choose the key carefully

Build the key from the tenant and a route class, such as reads, writes, or exports, not from the raw URL path. Paths contain object IDs (/invoices/9f1c...), so a key per path creates a separate bucket for every object a tenant touches. That is both a memory leak and a way to dodge the limit. Keep the number of route classes small and named in code.

Weight expensive requests

A request that scans a year of records costs far more than one that reads a single row. Give route classes different costs, measured from their typical database time or CPU, and pass the cost to the script. Review the weights when you change the endpoint, because a new index can make yesterday's expensive route cheap.

What the client should see

Reject with 429 Too Many Requests, defined in RFC 6585, and include a Retry-After header with the number of seconds to wait, as RFC 9110 defines it. Put a short machine-readable error in the body that names the limit that was hit, so a customer's engineer can tell a per-key limit from a tenant-wide one.

If you also want to advertise remaining quota on successful responses, the IETF HTTP API working group's RateLimit header fields draft defines RateLimit and RateLimit-Policy fields. It is still an Internet-Draft, and its field names changed across revisions, so many older articles show RateLimit-Limit and RateLimit-Remaining, which the current draft no longer uses. Document whichever headers you send.

When the limiter is unavailable

If Redis is unreachable, every request has to be either admitted or refused without a decision from the shared bucket. There is no universally right answer:

• Failing open keeps the API available but removes protection exactly when something is already wrong.
• Failing closed protects the database but turns a Redis outage into an API outage.

Decide per route class. A common split is to fall back to a local, per-server bucket set to the shared limit divided by the number of servers for ordinary reads, and to refuse expensive routes such as exports and bulk writes. Log every fallback decision and alert on it, because a silent fallback hides the outage that caused it. Google's SRE chapter on Handling Overload is a good companion on rejecting work early and cheaply.

Rate limits are not the whole answer

Request limits control how often a tenant asks. They do not control how long each request runs or how much background work it starts. Pair them with:

• concurrency limits per tenant on slow endpoints, so ten long exports cannot occupy every worker;
• fair scheduling for background jobs, covered with a lab in the job queue guide;
• database-level guardrails, such as statement timeouts for tenant-facing queries;
• placement decisions for tenants whose steady load is simply larger than a shared pool should carry, which the isolation-model guide discusses.

What to monitor

Record every 429 in your logs with the tenant ID, route class, and limit name. Do not put the tenant ID in a metrics label if you have many tenants; the observability guide explains why and what to do instead. From the logs, watch three things: which tenants are throttled most often, whether throttling is sustained (a runaway job) or brief (a legitimate burst that needs a larger bucket), and whether throttled clients honor Retry-After or retry immediately.

Where to start

Find the tenant ID in your request context, then add a per-tenant token bucket in front of your most expensive route class only. Measure how often it rejects for a week before you extend it to everything, and decide in writing what each route does when the limiter is unavailable. If you need a starting capacity number, the capacity planner can turn request rates and peaks into a planning envelope.

Frequently Asked Questions

How do I pick the bucket size and refill rate?

Start from observed traffic: set the refill near a high percentile of a well-behaved tenant's sustained rate and the bucket near its largest normal burst, such as one page load. Run in log-only mode for a week, count would-be rejections, and adjust before enforcing.

Should larger plans get larger limits?

Usually yes, if the plan is priced for more usage. Store limits with the plan or the tenant's entitlements rather than in code, and give support a way to raise one tenant's limit temporarily without a deployment.

Is 503 better than 429 when the whole service is overloaded?

Use 429 when a specific client exceeded its own allowance and 503 when the service is shedding load regardless of who is asking. Both can carry Retry-After. Mixing them up makes client retry logic and your own dashboards harder to read.

Can I run the limiter at the API gateway instead of in the application?

Yes, if the gateway can see the authenticated tenant ID. Many gateways only see IP addresses or API keys before authentication; in that case keep a tenant-level limit in the application as well.

Does a per-tenant limit stop one tenant from overloading the database?

Only partly. It limits how often the tenant can ask, not how expensive each request is. Add per-tenant concurrency limits and statement timeouts for heavy routes, and use cost weights so an export consumes more of the allowance than a single read.

Sources

Documents cited in this guide. Each link was opened and its title compared with the label below on the date shown. A source supports the specific statements it is linked from; design recommendations and lab results are Velynoriva's own.

• RFC 6585: Additional HTTP Status Codes (429 Too Many Requests) (Internet Engineering Task Force) • Link checked 2026-11-03

  Defines the 429 status code and allows a Retry-After header with it.

• RFC 9110: HTTP Semantics (Retry-After) (Internet Engineering Task Force) • Link checked 2026-11-03

  Defines the Retry-After header field.

• RateLimit header fields for HTTP (Internet-Draft) (IETF HTTPAPI Working Group) • Link checked 2026-11-03

  Work in progress defining RateLimit and RateLimit-Policy fields; not a published standard.

• Scaling your API with rate limiters (Stripe) • Link checked 2026-11-03

  Engineering account of token-bucket request limits, concurrency limits, and load shedding.

• Scripting with Lua (Redis) • Link checked 2026-11-03

  Atomic execution of scripts and effects replication.

• SaaS Lens: Noisy neighbor (Amazon Web Services) • Link checked 2026-11-03

  AWS guidance on noisy-neighbor effects and tier-based throttling in SaaS systems.

• Handling Overload (Site Reliability Engineering, chapter 21) (Google) • Link checked 2026-11-03

  Per-customer limits and rejecting excess work early.

Revision history

• 2026-11-03 — Added a reproducible Go lab. Replaced the rate-limit key built from the raw URL path, the universal fail-open rule, and the claim that RateLimit-Limit headers are an IETF standard. Updated the Lua script to read Redis server time and use HSET. Removed unsupported latency and size figures.
• 2026-10-12 — Endpoint-specific failure policy and header guidance revised.