Rate limits behind carrier-grade NAT

Per-IP rate limiting is the default almost everywhere: one bucket per client address, usually enforced at the reverse proxy. It is cheap, it needs no application code, and for many audiences it works. For a mobile app whose users are largely on Bangladeshi carrier networks, it is the wrong unit.

The problem

Mobile carriers commonly use carrier-grade NAT (CGNAT): many subscribers share a single public IPv4 address. To the server, hundreds of unrelated people can look like one very busy client. A per-IP limit sized for one person throttles all of them; a limit sized for all of them protects nothing.

The fix

• –Authenticated routes are limited per user ID, not per address. Each person gets their own budget regardless of which network they are on.
• –Expensive routes, such as AI parsing, get their own tighter per-user budgets because they cost money per call.
• –The limiter fails open. If the cache backing the counters is unavailable, the request proceeds and the failure is logged, rather than turning a cache outage into a full outage.

export function limitByUser(opts: { key: string; limit: number; windowSec: number }) {
  return async (req: Request, res: Response, next: NextFunction) => {
    const userId = req.auth?.userId;
    if (!userId) return next(); // unauthenticated traffic is handled separately

    try {
      const bucket = `rl:${opts.key}:${userId}`;
      const count = await cache.incr(bucket);
      if (count === 1) await cache.expire(bucket, opts.windowSec);
      if (count > opts.limit) return res.status(429).json({ error: "rate_limited" });
    } catch (err) {
      log.warn({ err }, "rate limiter unavailable, failing open");
    }
    next();
  };
}

Trade-offs

Limits now live in application code rather than in the proxy, which is more code to own. Traffic before sign-in still needs protection, and that part cannot be keyed by user. Failing open also means a cache outage briefly removes the limits; for this product, that was a better failure than refusing every request.

Verification

The behaviours worth testing are concrete: two users on the same address get independent budgets; one user exceeding a budget gets a 429 while others do not; and a simulated cache failure lets requests through and emits a warning instead of an error response.