Load balancer vs API gateway

Load balancerPicks a healthy copy

Delivery in10 minutes

Home · Indiranagar, BLR-07

Search for milk, bread…

Buy again

Fresh milk500 ml₹32
Whole wheat bread400 g₹52
Farm eggs6 pieces₹100

Bot5,000/s

Load balancerHealthy instance

App 1Zepto API
App 2Zepto API
App 3Zepto API

API gatewayReads every request

Delivery in10 minutes

Home · Indiranagar, BLR-07

Search for milk, bread…

Buy again

Fresh milk500 ml₹32
Whole wheat bread400 g₹52
Farm eggs6 pieces₹100

Bot5,000/s

Load balancerhealthy copy

API gatewayone copy

Token
Limit
Route

2% busy

Search/search
Checkout/orders

One Zepto shopper, two front doors: an expired login, a search, a bot and a crash. Illustrative numbers: one copy serves about 1,000 requests a second, the bot sends 5,000, and the gateway allows 100 a minute per user.

Compare next

Stateful vs Stateless servers Consistent hashing vs Modulo hashing

Load balancer or API gateway?

Use both, for different jobs. The load balancer spreads traffic over healthy copies; the gateway checks who is asking, how often, and which service it’s for. Most systems put the load balancer first, in front of the gateway’s copies.

See it in a real system

Watch Zepto’s real services Every call behind a Zepto order

Compare next

Stateful vs Stateless servers Consistent hashing vs Modulo hashing

Difference	Load balancer	API gateway
Its question	Which copy is healthy and free?	Who are you, are you allowed, and which service?
Sends to	Identical copies of one app	Different services, by path
Reads	The connection, or the URL at layer 7	The whole request: path, token, headers
Login	Not checked: each app checks for itself	Checked once, at the door
A flood	Spread evenly, so everyone slows down	Counted per user; past the limit, 429
A dead server	Its health check takes it out of rotation	Needs a load balancer in front of its own copies
Too big a flood	Passes it all on: the copies drown first	Can be swamped too: DDoS protection or a WAF goes in front

Load balancer vs API gateway, explained step by step

1. Open the app

One Zepto shopper, two different front doors. On the left, a load balancer: it spreads requests across identical copies of the app. On the right, an API gateway: it reads each request, checks who sent it, and sends it to the right service. Let’s put both through four situations.

First, a shopper opens Zepto after a week away. Their login has expired.

The load balancer doesn’t check logins, so it passes the request to App 1.

App 1 has to check the token itself and turn it away. The phone says: Log in again. Without a gateway, every service has to carry that same check.

The shopper logs in again.

On the right, the gateway checks the token at the door. It’s expired, so the gateway answers four oh one, and no service is ever touched.

2. Search for milk

Logged in again, the shopper searches for milk. The load balancer picks the least busy of three identical copies, App 2 here. Any copy can answer anything, so it doesn’t need to know what’s being asked.

App 2 checks the token again, then sends back the results.

On the right, the token is good this time. The gateway passes the user’s ID along, so no service checks it again.

Then it reads the path. Slash search goes to the Search service. An order, slash orders, would go to Checkout instead.

3. A bot floods search

Then a bot starts hammering search: five thousand requests a second. The load balancer can’t tell the bot from a shopper, so it spreads all five thousand evenly across the three copies.

In this example, each copy serves about a thousand a second. Three copies serve three thousand, so all of them are overloaded.

A real shopper’s search joins the same queue, behind the bot.

It waits behind thousands of bot requests, then times out. The shopper sees an error.

On the right, the gateway keeps a separate count for every user. Each user gets a hundred requests a minute.

Now the same five thousand a second reach the gateway. The bot uses up its hundred in a blink. After that, the gateway answers every bot request itself, four two nine, too many requests, without running a single search.

Saying no is cheap: it’s one counter check, so the gateway keeps up easily, and Search only sees real shoppers.

The shopper has their own count, still almost empty, so their search goes straight through, and the results come right back.

4. A copy crashes

Finally, a crash. App 2 dies. For a moment, the load balancer still sends it requests, so the shopper’s search fails.

Its health check fails, so the load balancer stops sending it traffic. App 1 and App 3 carry on, and the shopper’s next search works. That’s the load balancer’s main job.

On the right, the gateway runs on a server too, and it crashes. It can’t route around itself, so every shopper gets an error.

That’s why Zepto runs several copies of the gateway, with a load balancer in front. It picks a healthy copy, and the gateway checks and routes as before. The results come back the same way.

In short

So which should you pick? Both, for different jobs. A load balancer spreads traffic across healthy copies. An API gateway reads every request: who sent it, how often, and which service it’s for. Most systems put the load balancer first, then the gateway.