Systems Design

API Gateway Patterns That Prevent Cascading Failures

Published 2026-08-11 · 8 min read · By Stellar Forge

The most common distributed systems failure isn't one service crashing — it's one slow service quietly taking every other service down with it. An API gateway is your first and best place to prevent that, if it's designed for resilience rather than just routing.

The cascading failure, step by step

It usually goes like this: a downstream service (say, a third-party payment provider or a reporting database) starts responding slowly instead of failing outright. Requests to it pile up, holding open connections and threads. Upstream services waiting on those responses start queuing too. Within minutes, a slowdown in one non-critical service has consumed the connection pool for your entire API layer, and unrelated requests start timing out. Nothing "crashed" — the whole system just seized up.

Timeouts: the pattern everyone skips

The single highest-leverage fix is also the simplest: every outbound call from your gateway needs an explicit timeout, shorter than your own service's SLA allows. Without one, a hung downstream call can block a request indefinitely. This sounds obvious and is still, in practice, the most commonly missing piece in systems we've reviewed.

Circuit breakers: stop calling what's already broken

A circuit breaker tracks failure rates to a given downstream service and, past a threshold, stops sending it requests for a cooldown period — failing fast instead of piling up more waiting requests. This does two things: it protects your system from wasting resources on calls likely to fail, and it gives the struggling downstream service room to recover instead of being hit with a continued flood of retries.

Bulkheads: don't let one tenant sink the ship

Named after the compartments in a ship's hull, bulkheads isolate resources (connection pools, thread pools, rate limits) per downstream dependency or per tenant, so a problem in one doesn't exhaust capacity needed by everything else. In a multi-tenant system specifically, this also prevents one high-traffic or misbehaving customer from degrading service for everyone else — a failure mode that's easy to overlook until it happens.

Putting it together at the gateway layer

• Set explicit timeouts on every outbound call, tuned below your own response-time SLA.
• Add circuit breakers around dependencies with a history of intermittent slowness or failure — third-party APIs are the most common candidate.
• Apply bulkheads per major dependency and, in multi-tenant systems, per tenant tier.
• Return fast, honest errors when a circuit is open — a clear 503 with a retry hint is more useful to a client than a request that hangs for thirty seconds and then fails anyway.

Resilience is a design decision, not a library

These patterns are available in most API gateway and service mesh tooling, but adopting the library doesn't automatically get you the benefit — it requires deciding, per dependency, what the right timeout and failure threshold actually is. That decision is where the design work lives, and it's worth doing deliberately rather than accepting defaults.

---