API rate limiting is not a feature you add when you get popular. It is a foundational infrastructure decision that determines whether your API survives unexpected traffic spikes, malicious abuse, runaway clients, and misconfigured integrations. This is a production engineering guide — covering token bucket vs sliding window algorithm selection, Resilience4j in-process limiting, Redis-backed distributed limiting for horizontally scaled deployments, gateway vs application-layer strategy, proper RFC 9457 429 responses, and per-user/per-tier enforcement with JWT claims.
Every API I have built at scale has had a rate limiting incident. Not because rate limiting was not implemented — but because it was implemented incorrectly. In-memory limiters that reset on pod restart. IP-based limits that blocked an entire corporate network. 500 responses instead of 429 because nobody wired the fallback correctly. Limits enforced at the application layer that the API gateway bypassed entirely.
Rate limiting is not a one-liner. It is an architecture decision with operational consequences. This article is the guide I wish I had before I hit each of those incidents.
Rate limiting production failure modes — what goes wrong
1. Why API Rate Limiting Matters in 2026
Three categories of threats make rate limiting non-negotiable for any public or partner-facing API in 2026:
Abuse and scraping. Without rate limiting, a single client can drain your database, scrape your entire catalogue, or enumerate user IDs. These attacks do not require credentials — they require only an internet connection and patience. A search API with no rate limit is a full export tool for any competitor who wants your data.
Runaway clients. A misconfigured retry loop in a partner's SDK, a bug in a mobile app that retries every 100ms on failure, an automated test suite that runs against production — these are not malicious, but they are just as destructive as an attack. Every API I have operated at scale has been hit by a well-intentioned client sending 10,000 requests per minute when they expected to send 10.
Cost and fairness. In a multi-tenant system, one heavy user consuming 90% of API capacity degrades the experience for every other tenant. Rate limiting enforces fairness — premium tier gets more capacity, free tier gets less, and no single client can monopolise shared infrastructure.
A SaaS platform I reviewed had no rate limiting on their file export API. A data analysis firm discovered they could call it programmatically. Within 48 hours, 340GB of customer data had been exported through a single integration account. The account was authenticated and authorised — every request was legitimate. Rate limiting would have capped exports at a level that would have triggered an alert before significant data left the platform.
2. Throttling vs Rate Limiting — The Distinction That Matters Operationally
These terms are used interchangeably in documentation and then confused in implementation. They are architecturally different decisions with different resource implications.
| Aspect | Rate Limiting | Throttling |
|---|---|---|
| Behaviour | Reject requests over the limit with 429 | Accept all requests but artificially delay them |
| Server resource usage | Low — rejected requests consume minimal resources | High — threads held open for the delay duration |
| Client experience | Clear signal: retry after N seconds | Slow responses with no clear signal |
| Use case | Quota enforcement, abuse protection, fairness | Traffic shaping, preventing queue saturation |
| Production recommendation | ✓ Use for public and partner APIs | Use for internal message queues and batch jobs |
The practical reason to prefer rate limiting over throttling for APIs: throttling holds server-side threads open for the delay duration. Under sustained load, this exhausts thread pools faster than simply rejecting the request. A 429 uses virtually no server resources. A 5-second artificial delay on 1,000 concurrent requests holds 1,000 threads open for 5 seconds each.
3. Rate Limiting Algorithms — Which One to Choose and Why
Algorithm selection is not academic. It determines how the limiter behaves at the boundary — whether bursts are allowed, how window resets feel to clients, and how expensive the implementation is.
Token Bucket
Token bucket allows short bursts up to the bucket capacity while enforcing a sustained rate ceiling. A bucket with capacity 10 refilling at 10 tokens/second allows a user who has been idle to immediately fire 10 requests, then sustain 10/second thereafter. This matches real API usage patterns — users are not metronomically regular. Resilience4j's RateLimiter uses this algorithm.
Sliding Window Log
Maintains a log of every request timestamp. On each new request, counts how many requests occurred in the past N seconds. More precise than fixed window (no boundary burst), but memory-expensive at high volumes — each request entry is stored. Use for low-volume, high-precision scenarios (e.g., financial transaction APIs).
Sliding Window Counter (Hybrid)
Approximates the sliding window using two fixed-window counters (current window + previous window) weighted by elapsed time. Memory-efficient (only two counters per client), accurate to within a small margin. This is the algorithm used by Redis's built-in rate limiting and most production distributed rate limiters. Best choice for high-volume APIs at scale.
Token Bucket
Fixed capacity + steady refill. Allows bursts up to capacity. Memory: O(1) per client.
Best for: most APIsSliding Window Log
Exact precision. High memory cost (stores all request timestamps). Accurate at boundary.
Best for: low-volume, high-precisionSliding Window Counter
Approximated via two fixed-window counters. Efficient, scalable, small margin of error.
Best for: Redis-backed distributed4. Spring Boot Implementation with Resilience4j
Resilience4j is the standard rate limiting library for Spring Boot. Its RateLimiter uses the token bucket algorithm, integrates natively with Spring Boot 3.x, and supports annotation-based or programmatic configuration.
pom.xml · Resilience4j dependency for Spring Boot 3
<dependency>
<groupId>io.github.resilience4j</groupId>
<artifactId>resilience4j-spring-boot3</artifactId>
<version>2.2.0</version>
</dependency>application.yml · Resilience4j rate limiter configuration
resilience4j:
ratelimiter:
instances:
# Public unauthenticated endpoints — strict
publicApi:
limitForPeriod: 30 # 30 requests
limitRefreshPeriod: 1m # per minute
timeoutDuration: 0 # fail immediately if limit hit — don't queue
# Authenticated user endpoints — more generous
userApi:
limitForPeriod: 200
limitRefreshPeriod: 1m
timeoutDuration: 0
# Heavy write operations — very conservative
exportApi:
limitForPeriod: 5
limitRefreshPeriod: 1m
timeoutDuration: 0Java · annotation-based rate limiting with Resilience4j
@RestController
@RequestMapping("/api")
public class OrderController {
// Per-instance rate limit — NOT distributed (see Section 6 for Redis)
@GetMapping("/orders")
@RateLimiter(name = "userApi", fallbackMethod = "rateLimitFallback")
public ResponseEntity<List<Order>> getOrders(
@AuthenticationPrincipal UserDetails user) {
return ResponseEntity.ok(orderService.findByUser(user.getUsername()));
}
@PostMapping("/export")
@RateLimiter(name = "exportApi", fallbackMethod = "rateLimitFallback")
public ResponseEntity<Void> exportData(@AuthenticationPrincipal UserDetails user) {
exportService.queue(user.getUsername());
return ResponseEntity.accepted().build();
}
// Fallback — must match the annotated method's parameter types + RequestNotPermitted
public ResponseEntity<ProblemDetail> rateLimitFallback(
UserDetails user, RequestNotPermitted ex) {
return buildRateLimitResponse(60);
}
private ResponseEntity<ProblemDetail> buildRateLimitResponse(int retryAfter) {
ProblemDetail pd = ProblemDetail.forStatus(HttpStatus.TOO_MANY_REQUESTS);
pd.setType(URI.create("https://api.yourdomain.com/problems/rate-limit-exceeded"));
pd.setTitle("Rate Limit Exceeded");
pd.setDetail("Too many requests. Please retry after " + retryAfter + " seconds.");
pd.setProperty("retryAfterSeconds", retryAfter);
pd.setProperty("timestamp", Instant.now().toString());
return ResponseEntity.status(HttpStatus.TOO_MANY_REQUESTS)
.header("Content-Type", "application/problem+json")
.header("Retry-After", String.valueOf(retryAfter))
.header("X-RateLimit-Limit", "200")
.header("X-RateLimit-Remaining", "0")
.header("X-RateLimit-Reset", String.valueOf(
Instant.now().plusSeconds(retryAfter).getEpochSecond()))
.body(pd);
}
}With 4 pods and a limit of 100 requests/minute per pod, a single user can send 400 requests/minute by distributing requests across pods. For true per-user quota enforcement in horizontally scaled deployments, you need Redis-backed rate limiting. See Section 6.
5. Redis-Backed Distributed Rate Limiting
Redis solves the multi-instance problem by maintaining a single shared counter outside the JVM. Every pod increments and reads the same Redis key, making the limit global regardless of which pod handles the request. This is the correct architecture for any production deployment with more than one instance.
Two approaches: Bucket4j with Redis, or a custom Lua script. Bucket4j is more ergonomic. The Lua script is the lowest-latency option and is what large-scale systems use.
pom.xml · Bucket4j with Redis (Spring Boot 3)
<dependency>
<groupId>com.bucket4j</groupId>
<artifactId>bucket4j-redis</artifactId>
<version>8.10.1</version>
</dependency>Java · Bucket4j distributed rate limiter — per-user with Redis
@Component
public class UserRateLimiter {
private final RedisTemplate<String, byte[]> redisTemplate;
// Build a per-user bucket with token bucket algorithm
private Bucket buildBucket(String userId) {
RedisProxyManager<String> proxyManager = Bucket4jRedis.builderFor(redisTemplate)
.build();
BucketConfiguration config = BucketConfiguration.builder()
.addLimit(Bandwidth.classic(100, Refill.greedy(100, Duration.ofMinutes(1))))
// 100 tokens, refills 100 per minute — allows short bursts
.build();
return proxyManager.builder().build(userId, () -> config);
}
public boolean tryConsume(String userId) {
Bucket bucket = buildBucket("ratelimit:user:" + userId);
return bucket.tryConsume(1); // returns false if limit exceeded
}
public long getRemainingTokens(String userId) {
return buildBucket("ratelimit:user:" + userId).getAvailableTokens();
}
}
// In your controller/filter
@GetMapping("/api/orders")
public ResponseEntity<?> getOrders(@AuthenticationPrincipal UserDetails user) {
if (!userRateLimiter.tryConsume(user.getUsername())) {
return buildRateLimitResponse(60);
}
return ResponseEntity.ok(orderService.findByUser(user.getUsername()));
}Lua · Redis atomic sliding window — lowest latency for high-throughput APIs
-- rate_limit.lua — atomic sliding window counter in Redis
-- KEYS[1] = rate limit key (e.g. "rl:user:priya@example.com")
-- ARGV[1] = window size in seconds
-- ARGV[2] = max requests per window
-- ARGV[3] = current timestamp (epoch seconds)
local key = KEYS[1]
local window = tonumber(ARGV[1])
local limit = tonumber(ARGV[2])
local now = tonumber(ARGV[3])
local window_start = now - window
-- Remove expired entries outside the window
redis.call("ZREMRANGEBYSCORE", key, "-inf", window_start)
-- Count requests in current window
local count = redis.call("ZCARD", key)
if count < limit then
-- Add this request with current timestamp as score
redis.call("ZADD", key, now, now .. math.random())
redis.call("EXPIRE", key, window)
return {1, limit - count - 1} -- allowed, remaining
else
return {0, 0} -- rejected
endJava · call the Lua script from Spring Boot
@Component
public class RedisRateLimiter {
private final StringRedisTemplate redis;
private final RedisScript<List<Long>> script;
public RateLimitResult check(String key, int windowSeconds, int maxRequests) {
String rateLimitKey = "rl:" + key;
long now = Instant.now().getEpochSecond();
List<Long> result = redis.execute(script,
List.of(rateLimitKey),
String.valueOf(windowSeconds),
String.valueOf(maxRequests),
String.valueOf(now));
boolean allowed = result.get(0) == 1L;
long remaining = result.get(1);
return new RateLimitResult(allowed, remaining, windowSeconds);
}
}The Lua script sets a TTL on the sorted set equal to the window size. Expired keys are cleaned up automatically by Redis — you do not need a background job to remove old rate limit data. The atomic Lua script also prevents race conditions that a read-then-write approach would introduce.
6. API Gateway vs Application-Level Rate Limiting — Use Both
This is the most common architectural mistake I see: teams choose one or the other. Production systems need both, enforcing different concerns at different layers.
| Concern | Gateway layer | Application layer |
|---|---|---|
| DDoS / global IP flood | ✓ Gateway handles this | Should never reach app layer |
| Unauthenticated request burst | ✓ Gateway handles this | No auth context available |
| Per-user quota enforcement | ✗ No auth context | ✓ App layer with Redis |
| Per-tier limits (free vs premium) | ✗ Cannot read JWT claims | ✓ App layer reads JWT claims |
| Per-endpoint specific limits | Partial — route-based | ✓ Fine-grained per method |
| Business rule limits (export cap, API quota) | ✗ Not business-aware | ✓ Full business context |
7. Returning Proper 429 Responses with RFC 9457
This is the section most rate limiting implementations get wrong. The correct response to a rate limited request is 429 Too Many Requests with a Retry-After header and an RFC 9457 Problem Details body. What I find in production: 500 Internal Server Error, 503 Service Unavailable, or 200 OK with {"success": false}. All three are incorrect and all three break client handling.
HTTP · correct 429 response with RFC 9457 body
HTTP/1.1 429 Too Many Requests
Content-Type: application/problem+json
Retry-After: 47
X-RateLimit-Limit: 100
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1778336594
{
"type": "https://api.yourdomain.com/problems/rate-limit-exceeded",
"title": "Rate Limit Exceeded",
"status": 429,
"detail": "You have exceeded the 100 requests/minute limit. Retry after 47 seconds.",
"instance": "/api/orders",
"retryAfterSeconds": 47,
"limitPerMinute": 100,
"tier": "free",
"timestamp": "2026-05-09T14:23:14Z"
}Java · Spring Boot · complete 429 response builder
private ResponseEntity<ProblemDetail> buildRateLimitResponse(
String userId, String endpoint, RateLimitInfo info) {
ProblemDetail pd = ProblemDetail.forStatus(HttpStatus.TOO_MANY_REQUESTS);
pd.setType(URI.create("https://api.yourdomain.com/problems/rate-limit-exceeded"));
pd.setTitle("Rate Limit Exceeded");
pd.setDetail("You have exceeded the " + info.getLimit() + " requests/minute limit. "
+ "Retry after " + info.getRetryAfterSeconds() + " seconds.");
pd.setInstance(URI.create(endpoint));
pd.setProperty("retryAfterSeconds", info.getRetryAfterSeconds());
pd.setProperty("limitPerMinute", info.getLimit());
pd.setProperty("tier", info.getTier()); // free / pro / enterprise
pd.setProperty("timestamp", Instant.now().toString());
long resetAt = Instant.now().plusSeconds(info.getRetryAfterSeconds()).getEpochSecond();
return ResponseEntity.status(HttpStatus.TOO_MANY_REQUESTS)
.header("Content-Type", "application/problem+json")
.header("Retry-After", String.valueOf(info.getRetryAfterSeconds()))
.header("X-RateLimit-Limit", String.valueOf(info.getLimit()))
.header("X-RateLimit-Remaining","0")
.header("X-RateLimit-Reset", String.valueOf(resetAt))
.body(pd);
}Without Retry-After, every client that receives 429 will implement its own backoff strategy — and they will not coordinate. When the rate limit window resets, every client retries simultaneously: a thundering herd that immediately triggers the limit again. Retry-After tells all clients exactly when to retry, spreading them across the reset window and preventing the spike.
Test your 429 responses directly — verify the status code, Retry-After header, and RFC 9457 body structure before your clients discover a missing header in production. Use LearnHubly's REST API Tester to fire requests until the limit triggers and inspect the full response headers and body.
8. JWT, User, and IP-Based Rate Limiting Strategies
The identifier you rate limit against determines how precise and abuse-resistant your limiting is. Each strategy has different trade-offs and different appropriate use cases.
Per-Authenticated-User (JWT Subject) — Recommended for Most Endpoints
Java · rate limit by JWT user ID — most precise, abuse-resistant
@Component
public class UserRateLimitFilter extends OncePerRequestFilter {
private final RedisRateLimiter rateLimiter;
private final JwtDecoder jwtDecoder;
@Override
protected void doFilterInternal(HttpServletRequest request,
HttpServletResponse response, FilterChain chain)
throws ServletException, IOException {
String userId = extractUserId(request);
if (userId != null) {
// Tier-based limit from JWT claims
int limit = getTierLimit(request); // reads "tier" claim from JWT
RateLimitResult result = rateLimiter.check(
"user:" + userId, 60, limit);
if (!result.isAllowed()) {
writeRateLimitResponse(response, result);
return;
}
// Set remaining tokens header on every response (not just 429)
response.setHeader("X-RateLimit-Remaining",
String.valueOf(result.getRemainingTokens()));
}
chain.doFilter(request, response);
}
private int getTierLimit(HttpServletRequest request) {
// Read tier from JWT claim: free=60/min, pro=300/min, enterprise=2000/min
String tier = extractJwtClaim(request, "tier");
return switch (tier) {
case "pro" -> 300;
case "enterprise" -> 2000;
default -> 60; // free tier
};
}
}Per-IP (Unauthenticated Endpoints) — Secondary Protection Layer
Java · IP-based rate limiting — with proxy header awareness
private String extractClientIp(HttpServletRequest request) {
// Check forwarded headers first (behind load balancer / CDN)
String xForwardedFor = request.getHeader("X-Forwarded-For");
if (xForwardedFor != null && !xForwardedFor.isBlank()) {
// X-Forwarded-For can be a comma-separated list — take the first (client IP)
return xForwardedFor.split(",")[0].trim();
}
String xRealIp = request.getHeader("X-Real-IP");
if (xRealIp != null && !xRealIp.isBlank()) {
return xRealIp.trim();
}
return request.getRemoteAddr();
}Large organisations route all employees through a single NAT IP address. Setting an IP-based limit at 100 requests/minute on a login endpoint effectively blocks all 3,000 employees at a company whenever anyone at that IP hits the threshold. Use IP-based limiting conservatively (high thresholds) as a DDoS backstop, not as a per-user enforcement mechanism. For per-user enforcement, always use the authenticated identity.
Per-API-Key (Public API / Partner Access)
Java · API key rate limiting — for public API consumers
// Extract API key from header and use as rate limit identifier
String apiKey = request.getHeader("X-API-Key");
if (apiKey == null) {
// No API key — apply strict unauthenticated IP limit
return rateLimiter.check("ip:" + extractClientIp(request), 60, 10);
}
// Validate API key, look up associated quota
ApiKeyDetails keyDetails = apiKeyService.findByKey(apiKey);
if (keyDetails == null) {
throw new AuthenticationException("Invalid API key");
}
// Rate limit by API key identifier with key-specific quota
return rateLimiter.check(
"apikey:" + keyDetails.getId(),
60,
keyDetails.getMonthlyQuotaPerMinute()
);9. Common Rate Limiting Mistakes in Production
- In-memory limiter in a multi-pod deployment — per-pod counters let users exceed the global limit by a factor equal to the pod count. Use Redis-backed limiting for any horizontally scaled service.
- Rate limiting unauthenticated requests only — authenticated users can still exhaust downstream resources. Apply limits to all request paths, with different thresholds for authenticated vs unauthenticated traffic.
- Returning 500 instead of 429 from the fallback — Resilience4j throws
RequestNotPermittedwhich, without a fallback method, becomes an unhandled exception mapped to 500. Always wire the fallback and return 429 explicitly. - Missing Retry-After header — clients have no information about when to retry safely. They retry immediately and trigger the limit again. Thundering herd on every window reset.
- Rate limiting at the controller, not the filter — controller-level annotation rate limiting fires after the Spring Security filter chain, request parsing, and routing. An attacker sending malformed requests bypasses the annotation entirely. Apply rate limiting at the filter level for unauthenticated endpoints.
- No rate limit headers on successful responses — clients cannot know how close they are to their limit until they hit it. Return
X-RateLimit-Remainingon every response so clients can self-throttle before hitting 429. - One rate limit for all endpoints — a 100/min limit on your GET /users is appropriate; the same limit on POST /export (which generates a 50MB file) is far too generous. Set limits per operation cost, not per endpoint uniformly.
- No exception for health check endpoints — Kubernetes liveness probes and monitoring health checks should be exempt from rate limiting. A rate-limited health check causes false pod restarts.
10. Testing Rate Limits Before Production
Rate limiting bugs are invisible in normal testing and catastrophic in production. Test each of these explicitly before every release.
Java · Spring Boot Test · rate limit integration test
@SpringBootTest(webEnvironment = WebEnvironment.RANDOM_PORT)
@TestPropertySource(properties = {
"resilience4j.ratelimiter.instances.userApi.limitForPeriod=3",
"resilience4j.ratelimiter.instances.userApi.limitRefreshPeriod=10s"
})
class RateLimitIntegrationTest {
@Autowired TestRestTemplate restTemplate;
@Test
void shouldReturn429AfterLimitExceeded() {
HttpHeaders headers = new HttpHeaders();
headers.setBearerAuth(generateTestToken("user-1", "free"));
// First 3 requests should succeed
for (int i = 0; i < 3; i++) {
ResponseEntity<String> response = restTemplate.exchange(
"/api/orders", HttpMethod.GET,
new HttpEntity<>(headers), String.class);
assertThat(response.getStatusCode()).isEqualTo(HttpStatus.OK);
}
// 4th request should be rate limited
ResponseEntity<String> limited = restTemplate.exchange(
"/api/orders", HttpMethod.GET,
new HttpEntity<>(headers), String.class);
assertThat(limited.getStatusCode()).isEqualTo(HttpStatus.TOO_MANY_REQUESTS);
assertThat(limited.getHeaders().getFirst("Retry-After")).isNotNull();
assertThat(limited.getHeaders().getFirst("Content-Type"))
.contains("application/problem+json");
// Verify RFC 9457 body
JsonNode body = objectMapper.readTree(limited.getBody());
assertThat(body.get("status").asInt()).isEqualTo(429);
assertThat(body.get("type").asText()).contains("rate-limit-exceeded");
assertThat(body.get("retryAfterSeconds").asInt()).isGreaterThan(0);
}
@Test
void shouldTrackLimits separatelyPerUser() {
// User 1 hits limit
hitLimitForUser("user-1", 3);
// User 2 should still succeed — separate bucket
ResponseEntity<String> user2Response = makeRequest("user-2");
assertThat(user2Response.getStatusCode()).isEqualTo(HttpStatus.OK);
}
@Test
void shouldReturnRateLimitHeadersOnSuccessResponses() {
ResponseEntity<String> response = makeRequest("user-1");
assertThat(response.getHeaders().getFirst("X-RateLimit-Remaining")).isNotNull();
assertThat(response.getHeaders().getFirst("X-RateLimit-Limit")).isNotNull();
}
}For manual pre-release testing, use LearnHubly's REST API Tester — fire repeated requests and watch the X-RateLimit-Remaining header count down, then verify the 429 response body contains Retry-After and RFC 9457 fields. Takes 3 minutes and catches the most common misconfiguration before it reaches production.
11. Production Rate Limiting Checklist
- Redis-backed rate limiter used for all endpoints in multi-instance deployments Distributed
- Per-user (JWT subject) limit applied to all authenticated endpoints Identifier
- Per-IP limit applied as a secondary backstop for unauthenticated endpoints Identifier
- Tier-based limits enforced from JWT claims (free / pro / enterprise quotas) Business logic
- 429 response uses RFC 9457
application/problem+json— not 500 or 503 Status code Retry-Afterheader present on every 429 response HeadersX-RateLimit-Limit,X-RateLimit-Remaining,X-RateLimit-Resetreturned on all responses Headers- Resilience4j fallback method explicitly returns 429 — not left as unhandled exception Fallback
- Health check and liveness probe endpoints excluded from rate limiting Exceptions
- Different limits per operation cost (export ≠ list query ≠ health check) Granularity
- Rate limiting applied at filter level for unauthenticated endpoints, not controller annotation Architecture
- Integration test verifies 429 status, Retry-After header, and RFC 9457 body structure Testing
- API gateway rate limiting configured as Layer 1 (infrastructure protection) Gateway
- Monitoring alert configured for sustained 429 rate (possible abuse or misconfigured client) Observability
"A 429 with Retry-After is your API saying 'slow down.' A 500 without context is your API saying 'I give up.' The difference matters to every client that calls you."
12. FAQ — API Rate Limiting in Spring Boot
Rate limiting enforces a hard ceiling — requests over the limit are rejected with 429 Too Many Requests immediately. Throttling slows requests down rather than rejecting them — the server introduces artificial delay. Rate limiting is preferred for public APIs because it uses minimal server resources (rejected requests consume almost nothing), gives clients a clear signal with Retry-After, and protects infrastructure effectively. Throttling holds server threads open for the delay duration, which is expensive under load.
Token bucket gives each client a bucket of a fixed capacity. Each request consumes one token. Tokens refill at a steady rate. When the bucket is empty, requests are rejected. It allows short bursts up to the bucket capacity, then enforces a sustained rate ceiling. This matches real user behaviour — users are not metronomically regular. Resilience4j's RateLimiter uses this algorithm. It is the best choice for most API rate limiting use cases.
Resilience4j's default rate limiter is in-memory and per-JVM-instance. In a horizontally scaled deployment with multiple pods, each pod has its own counter — a user can send N × (pod count) requests before any single pod hits the limit. Redis-backed rate limiting (Bucket4j with Redis, or a Redis Lua script) maintains a single shared counter across all instances. Use Redis for any production deployment with more than one running instance.
Both — they serve different purposes. API gateway rate limiting (AWS API Gateway, Kong, NGINX) operates without authentication context and protects infrastructure from traffic floods, DDoS, and unauthenticated request bursts. Application-level rate limiting (Resilience4j + Redis in Spring Boot) has access to the authenticated user's identity, JWT claims, and subscription tier — enabling per-user quotas, tier-based limits, and per-endpoint business rules. Gateway alone cannot enforce per-authenticated-user limits.
Always 429 Too Many Requests — never 500, 503, or 200. Include a Retry-After header with the seconds until retry is safe. The response body should follow RFC 9457 Problem Details format with Content-Type: application/problem+json. Also include X-RateLimit-Limit, X-RateLimit-Remaining, and X-RateLimit-Reset headers. Without Retry-After, clients retry immediately when the window resets, causing a thundering herd that retriggeres the limit.
For authenticated endpoints: always by user ID (JWT subject). IP-based limiting for authenticated users is inaccurate — corporate NAT shares one IP among thousands of employees. For unauthenticated endpoints: IP-based limiting as a secondary backstop with generous thresholds (to avoid blocking corporate networks), combined with gateway-level global limits. For public APIs with API keys: rate limit by API key identifier with tier-specific quotas. Use all three strategies simultaneously at different thresholds for defence in depth.
13. Final Engineering Recommendations
Rate limiting is one of the most consequential infrastructure decisions in an API's lifetime — and one of the most frequently implemented incorrectly. Not because engineers do not know it matters, but because the failure modes are invisible until they happen at scale.
The architecture that works in production, consistently:
- Gateway layer for infrastructure protection. Global IP limits, DDoS mitigation, unauthenticated request filtering. This layer operates without authentication context and prevents floods from reaching your application.
- Redis-backed Bucket4j for per-user enforcement. Shared state across all pods, JWT-claim-aware, tier-based limits. This is where business quotas live.
- 429 with Retry-After and RFC 9457 body on every limit hit. Clients get a clear signal, retry at the right time, and parse a consistent error structure. Your monitoring dashboards show 429 rate, not 500 rate, for limit hits.
- Rate limit headers on every response. Clients self-throttle before hitting 429. The thundering herd on window reset is your problem to solve with Retry-After.
- Integration tests that trigger the limit explicitly. The most common production bug is the fallback returning 500 instead of 429. Test it before shipping.
The goal is an API that degrades gracefully under load — not one that falls over. Rate limiting, implemented correctly, is the mechanism that makes graceful degradation possible. — Priya
Test Your Rate Limiting Before It Triggers in Production
Fire repeated requests, count down the X-RateLimit-Remaining header, verify the 429 body structure and Retry-After value — all in the browser.