Token Bucket Rate Limiting Strategy

Lesson 10 of 15 12 min read Updated

Table of Contents

  1. Introduction
  2. How It Works
  3. Implementation
  4. Burst Mechanics
  5. Configuration Strategies
  6. Real-World Examples
  7. When to Use
  8. Complete Example

Introduction

Token Bucket is the most flexible rate limiting strategy. It allows controlled bursts while maintaining a sustainable long-term rate—perfect for APIs with naturally bursty traffic patterns.

The Problem It Solves:

User uploading 20 photos:

Fixed Window (30/min):
Upload 1-20: ✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓ (instant)
Upload 21-30: ❌ Wait 60 seconds
User frustrated 😞

Token Bucket (burst 20, refill 0.5/s):
Upload 1-20: ✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓ (instant burst)
Upload 21-30: ✓ Gradual (tokens refilling)
User happy 😊

Why Token Bucket?

  1. ✅ Excellent burst handling - Allow legitimate traffic spikes
  2. ✅ Natural flow - Matches real user behavior
  3. ✅ Sustained rate control - Prevents long-term abuse
  4. ✅ Flexible configuration - Tune burst size independently

How It Works

The Bucket Metaphor

Imagine a physical bucket that holds tokens:

🪣 Token Bucket
├─ Capacity: 20 tokens (max burst)
├─ Current: 15 tokens
├─ Refill Rate: 5 tokens per 10 seconds
└─ Cost per request: 1 token

Request arrives:
1. Check if bucket has ≥1 token
2. If yes: Remove 1 token, allow request ✓
3. If no: Reject request ❌
4. Background: Refill tokens at constant rate

Visual Timeline

Token Bucket Timeline (Capacity: 20, Refill: 5 per 10s)

00:00 → 🪣[████████████████████] 20/20 tokens (full)
00:01 → User makes 20 rapid requests
00:01 → 🪣[ ] 0/20 tokens (empty)
00:02 → User tries request 21 ❌ (no tokens)

00:11 → 🪣[█████ ] 5/20 tokens (refilled)
00:11 → User makes 5 requests ✓

00:21 → 🪣[██████████ ] 10/20 tokens (more refill)
00:31 → 🪣[███████████████ ] 15/20 tokens
00:41 → 🪣[████████████████████] 20/20 tokens (back to full)

Core Algorithm

Token Bucket State:
- capacity: maximum tokens (burst size)
- tokens: current available tokens
- refill_rate: tokens added per period
- refill_period: time between refills
- last_refill: timestamp of last refill

For each request at time T:
// Step 1: Calculate tokens to add since last refill
elapsed_time = T - last_refill
periods_elapsed = elapsed_time / refill_period
tokens_to_add = periods_elapsed × refill_rate
// Step 2: Refill bucket (capped at capacity)
tokens = min(capacity, tokens + tokens_to_add)
last_refill = T
// Step 3: Check if request can be served
if tokens >= 1:
tokens -= 1
return ALLOW ✓
else:
return REJECT ❌

Key Properties

1. Burst Allowance:

Burst capacity = Token capacity
Example: 20 tokens = 20 instant requests possible

2. Sustained Rate:

Long-term rate = Refill rate
Example: 5 tokens per 10s = 30 tokens/min sustained

3. Recovery Time:

Time to full = (Capacity / Refill rate) × Refill period
Example: (20 / 5) × 10s = 40 seconds

Implementation

Complete C# Implementation

using Microsoft.AspNetCore.RateLimiting;

namespace SecurityApp.API.RateLimiting;

/// <summary>
/// Token Bucket Rate Limiter
/// Best for APIs with bursty traffic patterns (file uploads, batch operations).
/// </summary>
public static class TokenBucketRateLimiter
{
public const string PolicyName = "TokenBucket";

/// <summary>
/// Configuration:
/// - TokenLimit: 20 tokens (maximum burst size)
/// - TokensPerPeriod: 5 tokens added per refill
/// - ReplenishmentPeriod: 10 seconds between refills
/// - AutoReplenishment: Automatic background refilling
///
/// Behavior:
/// - Initial burst: 20 requests instantly
/// - Sustained rate: 5 requests per 10s = 30 requests/minute
/// - Recovery time: 40 seconds to refill from empty to full
///
/// Example Timeline:
/// 00:00 → 20 requests instantly ✓
/// 00:01 → Requests rejected (empty bucket)
/// 00:10 → 5 new tokens → 5 requests ✓
/// 00:20 → 5 more tokens → 5 requests ✓
/// 00:40 → Back to 20 tokens
/// </summary>
public static void Configure(RateLimiterOptions rateLimiterOptions)
{
rateLimiterOptions.AddTokenBucketLimiter(PolicyName, options =>
{
options.TokenLimit = 20;
options.TokensPerPeriod = 5;
options.ReplenishmentPeriod = TimeSpan.FromSeconds(10);
options.QueueProcessingOrder = System.Threading.RateLimiting.QueueProcessingOrder.OldestFirst;
options.QueueLimit = 5;
options.AutoReplenishment = true; // Important: Enable auto-refill
});
}
}

Registration & Configuration

// Program.cs
var builder = WebApplication.CreateBuilder(args);

builder.Services.AddRateLimiter(options =>
{
// Register Token Bucket
TokenBucketRateLimiter.Configure(options);
// Rejection handler
options.OnRejected = async (context, cancellationToken) =>
{
context.HttpContext.Response.StatusCode = 429;
if (context.Lease.TryGetMetadata(MetadataName.RetryAfter, out var retryAfter))
{
context.HttpContext.Response.Headers.RetryAfter =
((int)retryAfter.TotalSeconds).ToString();
}
await context.HttpContext.Response.WriteAsJsonAsync(new
{
error = "rate_limit_exceeded",
message = "Token bucket empty. Tokens refill at 5 per 10 seconds.",
retry_after_seconds = (int)retryAfter.TotalSeconds,
strategy = "token_bucket",
hint = "Burst of 20 requests allowed, then gradual refill"
}, cancellationToken);
};
});

var app = builder.Build();
app.UseRateLimiter();

// Apply to bursty endpoints
app.MapPost("/api/upload", async (IFormFile file) =>
{
// File upload logic
return Results.Ok(new { uploaded = file.FileName });
})
.RequireRateLimiting(TokenBucketRateLimiter.PolicyName)
.DisableAntiforgery();

app.Run();

Configuration Patterns

Pattern 1: Large Burst, Slow Refill

// Use case: File upload batches
options.AddTokenBucketLimiter("batch-upload", opt =>
{
opt.TokenLimit = 100; // Large burst
opt.TokensPerPeriod = 10; // Slow refill
opt.ReplenishmentPeriod = TimeSpan.FromSeconds(30);
opt.AutoReplenishment = true;
});

// Behavior:
// - Upload 100 files instantly
// - Then 10 files every 30 seconds
// - Sustained: 20 files/minute

Pattern 2: Small Burst, Fast Refill

// Use case: Real-time updates
options.AddTokenBucketLimiter("realtime", opt =>
{
opt.TokenLimit = 10; // Small burst
opt.TokensPerPeriod = 5; // Fast refill
opt.ReplenishmentPeriod = TimeSpan.FromSeconds(1);
opt.AutoReplenishment = true;
});

// Behavior:
// - Initial burst: 10 requests
// - Then 5 requests per second
// - Sustained: 300 requests/minute

Pattern 3: Balanced Configuration

// Use case: General API (recommended)
options.AddTokenBucketLimiter("balanced", opt =>
{
opt.TokenLimit = 50; // Moderate burst
opt.TokensPerPeriod = 10; // Moderate refill
opt.ReplenishmentPeriod = TimeSpan.FromSeconds(5);
opt.AutoReplenishment = true;
});

// Behavior:
// - Initial burst: 50 requests
// - Sustained: 120 requests/minute
// - Good balance between burst and sustained rate

Burst Mechanics

Understanding Burst Capacity

Burst vs Sustained Rate:

Token Bucket Configuration:
- Capacity: 20 tokens
- Refill: 5 tokens per 10 seconds
- Sustained rate: 30 tokens/minute

Timeline Analysis:

Minute 1 (with full bucket):
00:00 → 20 tokens available (burst)
00:00-00:01 → Use 20 tokens instantly
00:10 → +5 tokens
00:20 → +5 tokens
00:30 → +5 tokens
00:40 → +5 tokens
00:50 → +5 tokens
Total in minute 1: 20 + 25 = 45 requests ✓

Minute 2 (starting empty):
01:00 → 5 tokens (from 00:50 refill)
01:10 → +5 tokens
01:20 → +5 tokens
01:30 → +5 tokens
01:40 → +5 tokens
01:50 → +5 tokens
Total in minute 2: 30 requests ✓

Conclusion:
- Short term (1st minute): 45 requests (burst benefit)
- Long term (average): 30 requests/minute (sustained)

Burst Use Cases

Use Case 1: Photo Upload

// Scenario: User uploads vacation photos

// Bad: Fixed Window
options.AddFixedWindowLimiter("photos-bad", opt =>
{
opt.PermitLimit = 30;
opt.Window = TimeSpan.FromMinutes(1);
});

// User experience:
// Upload 30 photos → ✓ Success
// Upload photo 31 → ❌ Wait 60 seconds
// User: "This is annoying!" 😞

// Good: Token Bucket
options.AddTokenBucketLimiter("photos-good", opt =>
{
opt.TokenLimit = 50; // Allow burst
opt.TokensPerPeriod = 10;
opt.ReplenishmentPeriod = TimeSpan.FromSeconds(10);
opt.AutoReplenishment = true;
});

// User experience:
// Upload 50 photos → ✓ Instant success
// Upload more → ✓ Gradual (10 every 10s)
// User: "Works great!" 😊

Use Case 2: Report Generation

// Scenario: Generate multiple reports

options.AddTokenBucketLimiter("reports", opt =>
{
opt.TokenLimit = 5; // Small burst
opt.TokensPerPeriod = 1; // Slow refill
opt.ReplenishmentPeriod = TimeSpan.FromMinutes(5);
opt.AutoReplenishment = true;
});

// Behavior:
// - Generate 5 reports instantly
// - Then 1 report every 5 minutes
// - Prevents overwhelming report service

Mathematical Analysis

Effective Rate Over Time:

Configuration:
- Token Limit (L): 20
- Tokens Per Period (T): 5
- Replenishment Period (P): 10 seconds

Short-term rate (first minute):
Burst + Refills = L + (60/P × T)
= 20 + (6 × 5)
= 20 + 30
= 50 requests/minute

Long-term rate (sustained):
Sustained = (60/P) × T
= (60/10) × 5
= 6 × 5
= 30 requests/minute

Burst multiplier:
Short-term / Long-term = 50/30 = 1.67x
User can burst 67% above sustained rate initially

Configuration Strategies

Strategy 1: High Burst, Low Sustained

// Use case: Batch imports, bulk operations

options.AddTokenBucketLimiter("bulk-import", opt =>
{
opt.TokenLimit = 1000; // Very high burst
opt.TokensPerPeriod = 50; // Moderate sustained
opt.ReplenishmentPeriod = TimeSpan.FromSeconds(30);
opt.AutoReplenishment = true;
});

// Analysis:
// Burst: 1000 requests instantly
// Sustained: 100 requests/minute
// Recovery: 600 seconds (10 minutes) to full
// Best for: Data import, migrations

Strategy 2: Low Burst, High Sustained

// Use case: Streaming, continuous operations

options.AddTokenBucketLimiter("streaming", opt =>
{
opt.TokenLimit = 10; // Low burst
opt.TokensPerPeriod = 50; // High sustained
opt.ReplenishmentPeriod = TimeSpan.FromSeconds(1);
opt.AutoReplenishment = true;
});

// Analysis:
// Burst: 10 requests instantly
// Sustained: 3000 requests/minute
// Recovery: 0.2 seconds to full
// Best for: Real-time feeds, websockets

Strategy 3: Adaptive Configuration

// Dynamic configuration based on user tier

public class AdaptiveTokenBucket
{
public static void Configure(
RateLimiterOptions options,
string tier)
{
var config = tier.ToLower() switch
{
"free" => (Limit: 10, Refill: 2, Period: 10),
"basic" => (Limit: 50, Refill: 10, Period: 10),
"pro" => (Limit: 200, Refill: 50, Period: 10),
"enterprise" => (Limit: 1000, Refill: 200, Period: 10),
_ => (Limit: 10, Refill: 2, Period: 10)
};
options.AddTokenBucketLimiter($"adaptive-{tier}", opt =>
{
opt.TokenLimit = config.Limit;
opt.TokensPerPeriod = config.Refill;
opt.ReplenishmentPeriod = TimeSpan.FromSeconds(config.Period);
opt.AutoReplenishment = true;
});
}
}

// Usage
AdaptiveTokenBucket.Configure(options, "pro");

Real-World Examples

Example 1: File Upload API

// Scenario: Photo sharing app

public class FileUploadConfiguration
{
public static void ConfigureRateLimiting(RateLimiterOptions options)
{
// Small files (< 1MB): Generous burst
options.AddTokenBucketLimiter("small-files", opt =>
{
opt.TokenLimit = 100; // 100 files burst
opt.TokensPerPeriod = 20; // 20 files per 10s
opt.ReplenishmentPeriod = TimeSpan.FromSeconds(10);
opt.AutoReplenishment = true;
});
// Large files (> 1MB): Limited burst
options.AddTokenBucketLimiter("large-files", opt =>
{
opt.TokenLimit = 10; // 10 files burst
opt.TokensPerPeriod = 2; // 2 files per 10s
opt.ReplenishmentPeriod = TimeSpan.FromSeconds(10);
opt.AutoReplenishment = true;
});
}
}

// Apply based on file size
app.MapPost("/api/upload", async (HttpContext context, IFormFile file) =>
{
var policy = file.Length < 1_000_000 ? "small-files" : "large-files";
// Check rate limit programmatically
var limiter = context.RequestServices
.GetRequiredService<RateLimiter>();
using var lease = await limiter.AcquireAsync(context, 1);
if (!lease.IsAcquired)
{
return Results.StatusCode(429);
}
// Process upload
await SaveFile(file);
return Results.Ok(new { uploaded = file.FileName });
});

Example 2: API Integration

// Scenario: Third-party API with rate limits

public class ExternalApiClient
{
private readonly RateLimiter _rateLimiter;
public ExternalApiClient()
{
// Match external API limits
var limiter = new TokenBucketRateLimiter(new TokenBucketRateLimiterOptions
{
TokenLimit = 100, // Their burst limit
TokensPerPeriod = 10, // Their sustained rate
ReplenishmentPeriod = TimeSpan.FromSeconds(1),
AutoReplenishment = true
});
_rateLimiter = new RateLimiter(limiter);
}
public async Task<ApiResponse> CallApiAsync(ApiRequest request)
{
// Wait for token before calling external API
using var lease = await _rateLimiter.AcquireAsync(1);
if (!lease.IsAcquired)
{
throw new RateLimitException("Rate limit exceeded for external API");
}
// Make actual API call
return await _httpClient.PostAsJsonAsync("/api/endpoint", request);
}
}

Example 3: Email Sending

// Scenario: Email service with send limits

options.AddTokenBucketLimiter("email", opt =>
{
opt.TokenLimit = 50; // 50 emails burst
opt.TokensPerPeriod = 10; // 10 emails per minute
opt.ReplenishmentPeriod = TimeSpan.FromMinutes(1);
opt.AutoReplenishment = true;
});

app.MapPost("/api/email/send", async (EmailRequest request) =>
{
// Process email
await _emailService.SendAsync(request);
return Results.Ok();
})
.RequireRateLimiting("email");

// Behavior:
// - Newsletter blast: Send 50 immediately
// - Ongoing emails: 10 per minute sustained
// - Prevents email provider throttling

When to Use

✅ Perfect For

1. File Upload/Download

// Users upload photos in bursts
app.MapPost("/api/photos/upload", UploadPhotos)
.RequireRateLimiting("TokenBucket");

// Why: Allows natural burst behavior

2. Batch Operations

// Bulk data import
app.MapPost("/api/import/bulk", ImportData)
.RequireRateLimiting("TokenBucket");

// Why: Large initial burst, then steady rate

3. Report Generation

// Generate multiple reports
app.MapPost("/api/reports/generate", GenerateReport)
.RequireRateLimiting("TokenBucket");

// Why: Burst for multiple reports, prevent abuse

4. Image Processing

// Process multiple images
app.MapPost("/api/images/process", ProcessImages)
.RequireRateLimiting("TokenBucket");

// Why: Burst for batch, sustained for ongoing

❌ Avoid When

1. Need Strict Enforcement

// Payment processing (no bursts allowed)
app.MapPost("/api/payment", ProcessPayment)
.RequireRateLimiting("SlidingWindow"); // Better choice

// Why: Token Bucket allows bursts, payments need strict control

2. Extremely Simple APIs

// Internal health check
app.MapGet("/health", () => "OK")
.RequireRateLimiting("FixedWindow"); // Simpler choice

// Why: Overhead not worth it for simple internal APIs

3. Long-Running Operations

// Video transcoding (takes minutes)
app.MapPost("/api/video/transcode", TranscodeVideo)
.RequireRateLimiting("Concurrency"); // Better choice

// Why: Need to limit concurrent operations, not rate

Complete Example

Full Production Setup

// Program.cs
using Microsoft.AspNetCore.RateLimiting;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddRateLimiter(options =>
{
// Photo uploads: Allow bursts
options.AddTokenBucketLimiter("photo-upload", opt =>
{
opt.TokenLimit = 50;
opt.TokensPerPeriod = 10;
opt.ReplenishmentPeriod = TimeSpan.FromSeconds(10);
opt.QueueLimit = 10;
opt.AutoReplenishment = true;
});
// API calls: Balanced
options.AddTokenBucketLimiter("api-calls", opt =>
{
opt.TokenLimit = 100;
opt.TokensPerPeriod = 20;
opt.ReplenishmentPeriod = TimeSpan.FromSeconds(10);
opt.QueueLimit = 20;
opt.AutoReplenishment = true;
});
// Batch operations: Large burst
options.AddTokenBucketLimiter("batch", opt =>
{
opt.TokenLimit = 1000;
opt.TokensPerPeriod = 50;
opt.ReplenishmentPeriod = TimeSpan.FromSeconds(30);
opt.QueueLimit = 100;
opt.AutoReplenishment = true;
});
// Rejection handler with token info
options.OnRejected = async (context, ct) =>
{
var logger = context.HttpContext.RequestServices
.GetRequiredService<ILogger<Program>>();
logger.LogWarning(
"Token bucket empty: {Path} from {IP}",
context.HttpContext.Request.Path,
context.HttpContext.Connection.RemoteIpAddress);
context.HttpContext.Response.StatusCode = 429;
if (context.Lease.TryGetMetadata(
System.Threading.RateLimiting.MetadataName.RetryAfter,
out var retryAfter))
{
context.HttpContext.Response.Headers.RetryAfter =
((int)retryAfter.TotalSeconds).ToString();
}
await context.HttpContext.Response.WriteAsJsonAsync(new
{
error = "rate_limit_exceeded",
message = "Token bucket empty. Tokens refill gradually.",
strategy = "token_bucket",
details = new
{
retry_after_seconds = (int?)retryAfter?.TotalSeconds ?? 10,
explanation = "Burst capacity used. Tokens refilling at configured rate.",
hint = "Implement exponential backoff or cache results"
}
}, ct);
};
});

var app = builder.Build();
app.UseRateLimiter();

// Photo upload endpoint
app.MapPost("/api/photos", async (IFormFileCollection files) =>
{
var results = new List<string>();
foreach (var file in files)
{
var path = await SavePhoto(file);
results.Add(path);
}
return Results.Ok(new { uploaded = results.Count, files = results });
})
.RequireRateLimiting("photo-upload")
.DisableAntiforgery();

// Batch import endpoint
app.MapPost("/api/import", async (ImportRequest request) =>
{
await ProcessImport(request);
return Results.Ok(new { imported = request.Items.Count });
})
.RequireRateLimiting("batch");

// Standard API calls
app.MapGet("/api/data", () => GetData())
.RequireRateLimiting("api-calls");

app.Run();

async Task<string> SavePhoto(IFormFile file)
{
var path = Path.Combine("uploads", Guid.NewGuid() + Path.GetExtension(file.FileName));
using var stream = File.Create(path);
await file.CopyToAsync(stream);
return path;
}

async Task ProcessImport(ImportRequest request)
{
// Simulate batch processing
await Task.Delay(100);
}

object GetData() => new { data = "value" };

record ImportRequest(List<string> Items);