howdy/rate_limit

Rate limiting middleware.

Create a limiter once at startup, then attach it at any level with a choice of how to identify the client.

let api = rate_limit.fixed_window(limit: 100, per_seconds: 60)
let writes = rate_limit.token_bucket(capacity: 10, refill_per_second: 1)

howdy.new()
|> howdy.middleware(rate_limit.by_ip(api))

controller.post("/", create |> middleware.wrap(rate_limit.by_header(writes, "x-api-key")))

Counters live in an ETS table owned by the process that called the constructor. Create limiters in main, not inside a handler. A linked cleanup process expires old windows every window or minute, whichever is shorter. It exits when the constructor’s process exits. Token buckets are evicted once they have refilled to capacity, which is the same as not existing; a bucket is never evicted while it is short of tokens. Sweeps run at most a minute apart.

Each limiter tracks at most default_max_identities distinct keys, which max_identities adjusts. The bound is exact: a slot is reserved atomically before a row is inserted, so concurrent first hits cannot overshoot it. Keys over 64 bytes are stored as their SHA-256, so row size is bounded too. Keys already in the table keep being limited even when the table is full. A key not yet in the table is refused with Saturated, so an attacker who can mint keys (a spoofed header, a large IPv6 range) cannot grow memory without bound, and cannot switch limiting off either. Trade-off: while saturated, new legitimate clients are refused too, and room only returns when a sweep finds rows to evict. Size the cap for your identity space.

Behind a proxy every request shares the proxy’s IP, so use by_header with the header your proxy sets rather than by_ip.

Those counters belong to one node: behind a load balancer each node allows the full limit, and a restart forgets it. shared keeps a fixed window’s counts in a Store every node can reach, such as a database table, at the cost of one round trip per request.

Types

Decision

</>

The outcome of checking a key against a limiter.

pub type Decision {
  Allowed(limit: Int, remaining: Int, reset_seconds: Int)
  Denied(limit: Int, retry_after_seconds: Int)
  Saturated(retry_after_seconds: Int)
}

Constructors

```
Allowed(limit: Int, remaining: Int, reset_seconds: Int)
```
The request may proceed. remaining is how many more are allowed before the limit; reset_seconds is when that number goes back up.
```
Denied(limit: Int, retry_after_seconds: Int)
```
The request is over the limit. Try again after retry_after_seconds.
```
Saturated(retry_after_seconds: Int)
```
The key is new and the limiter already tracks its maximum number of identities. Nothing was counted. The next sweep runs within retry_after_seconds; it frees room only if some rows have expired or refilled by then.

Limiter

opaque </>

pub opaque type Limiter

Store

</>

Counts kept outside this node. increment(key, window, expires_at_ms) adds a hit to key in fixed window number window and returns how many that window has counted, this one included. It must be atomic per key. A hit for a window older than the one a key holds (a node with a slow clock) should count against the newer window rather than reset it. The count is not needed after expires_at_ms, Unix time in milliseconds.

pub type Store {
  Store(increment: fn(String, Int, Int) -> Result(Int, Nil))
}

Constructors

Store(increment: fn(String, Int, Int) -> Result(Int, Nil))

Values

by

</>

pub fn by(
  limiter: Limiter,
  key: fn(context.Context(Nil)) -> option.Option(String),
) -> fn(
  context.Context(Nil),
  fn(context.Context(Nil)) -> response.Response(content.Content),
) -> response.Response(content.Content)

Limit by any key you derive from the request. None skips limiting.

by_header

</>

pub fn by_header(
  limiter: Limiter,
  header: String,
) -> fn(
  context.Context(Nil),
  fn(context.Context(Nil)) -> response.Response(content.Content),
) -> response.Response(content.Content)

Limit by the value of a request header. Requests without the header are not limited; combine with an auth middleware that requires it.

by_ip

</>

pub fn by_ip(
  limiter: Limiter,
) -> fn(
  context.Context(Nil),
  fn(context.Context(Nil)) -> response.Response(content.Content),
) -> response.Response(content.Content)

Limit by the client’s socket address. Behind a proxy, prefer by_header.

check

</>

pub fn check(limiter: Limiter, key: String) -> Decision

Record a hit for key and decide whether it is allowed.

check_at

</>

pub fn check_at(
  limiter: Limiter,
  key: String,
  now_ms: Int,
) -> Decision

Like check but with the current time supplied in milliseconds. Exposed so tests can drive the clock without sleeping. Automatic cleanup uses the VM monotonic clock; use its time domain and do not replay times whose state has already expired. Use check for live traffic: it resamples time when retrying after concurrent updates or eviction. A token bucket treats timestamps older than its last update as that last update’s time, so delayed concurrent requests cannot reverse its clock.

default_max_identities

</>

pub const default_max_identities: Int

How many distinct keys a limiter tracks unless max_identities says otherwise. At roughly a hundred bytes per row this is a few tens of megabytes at worst.

fixed_window

</>

pub fn fixed_window(
  limit limit: Int,
  per_seconds per_seconds: Int,
) -> Limiter

Allow limit requests per key in each window of per_seconds. Simple and predictable; a client can make up to twice limit requests around a window boundary.

max_identities

</>

pub fn max_identities(limiter: Limiter, count: Int) -> Limiter

Track at most count distinct keys instead of default_max_identities. Counts rows in the table: for a fixed window a key’s previous window also occupies a row until the sweep after the boundary, so allow for that.

rate_limit.fixed_window(limit: 100, per_seconds: 60)
|> rate_limit.max_identities(1_000_000)

shared

</>

pub fn shared(
  limit limit: Int,
  per_seconds per_seconds: Int,
  name name: String,
  store store: Store,
) -> Limiter

Allow limit requests per key in each window of per_seconds, counted in store so every node shares them. name separates this limiter’s keys from any other limiter using the same store; give each a different one. Windows follow the system clock, so keep nodes’ clocks synchronised.

If the store fails, the request is allowed and a warning logged: a limiter is abuse protection, and one that failed closed would turn an outage of its store into an outage of every route behind it.

token_bucket

</>

pub fn token_bucket(
  capacity capacity: Int,
  refill_per_second refill_per_second: Int,
) -> Limiter

Each key has a bucket holding up to capacity tokens, refilled at refill_per_second. A request takes one token. Allows short bursts while holding the long-run rate.