“Scale” covers two different problems:
- Traffic scale: handling more requests per second.
- Team scale: letting more engineers ship without stepping on each other.
A monolith is one deployable application whose modules call each other through plain function calls. A microservice deploys on its own and talks to the others over the network. Splitting into microservices mostly helps team scale, not traffic scale. Deploying separately gives no request more CPU. It lets the payments team release on Tuesday without waiting for the search team. A stateless monolith already scales out: run ten copies behind a load balancer. Split it only when one part needs very different resources, or when teams keep blocking each other’s deploys.
The same request, two ways#
Take an order that touches orders, payments and users. In the monolith the client makes one network call, and the rest are function calls that cost nanoseconds and can’t time out. Split the code into three services and every arrow becomes a network hop with its own timeout and retry policy. The animation runs both versions side by side.
Figure 1: The same request as one process (top) and as three services (bottom). Only the bottom row pays for hops and a retry.
Why splitting doesn’t add capacity#
Capacity comes from machines. Ten copies of a stateless monolith give you ten copies of orders, payments and users, as far as the database allows. That’s plain horizontal scaling, and it doesn’t care whether the code inside is one module or forty.
Splitting adds work on top. Every call gets serialized, sent and parsed, and then the response makes the same trip back. Inside one data center a hop costs around a millisecond or a few, plus the tail.1 As the Latency vs Throughput note explains, a request that crosses five services waits on the slowest of five p99s. Services that skip connection pooling also pay for TCP and TLS handshakes again and again, as the URL to first byte note shows.
Failure multiplies too. If each service is up 99.9% of the time and a request needs four in a row, it succeeds 99.6% of the time,2 which is about four times the downtime.
Retries are worse. If every caller retries a failed call twice, each layer multiplies the calls below it by three. When the last of four services goes down, it takes 3 × 3 × 3 = 27 calls for one user click, right when it’s trying to recover. The animation shrinks this to three services and one retry each:
Figure 2: With users down, a retry at each layer repeats everything below it, so the calls reaching users double with every layer.
The usual fix is to retry at one layer only, often the edge. A retry budget per client and a circuit breaker help too, so callers stop hitting a service that’s already down.
The Prime Video case#
In March 2023 the Prime Video tech blog described one service, the tool that monitors stream quality. Its first version ran on AWS Step Functions and Lambda, and passed video frames between steps through S3. Step Functions bills per state transition, and every frame meant S3 calls, so both costs grew with load. The team moved everything into one process on Amazon ECS, passed frames in memory, and cut infrastructure cost by over 90%. To grow, they ran several copies of that process. That’s scaling out a monolith.3
The lesson is narrow. When parts hand each other a lot of data, moving the data can cost more than processing it. It isn’t a verdict on microservices in general.
When splitting is the right call#
The strongest traffic argument for services is that you scale only the busy part. For a typical web app the idle modules cost each copy some memory, not CPU, so it only wins when the busy part needs different hardware.
Very different resources. Video transcoding wants CPU or a GPU and can wait in a queue. Checkout wants low latency and must stay up. In one process you size every copy for the worse of the two, and a transcoding spike slows checkout. Split, each scales and fails on its own.
Teams blocking each other. With a few dozen engineers in one codebase you get long merge queues, release trains, and one team’s broken test holding up everyone’s deploy. Separate services with separate pipelines fix exactly that.
A middle step is the modular monolith: one deployable with enforced boundaries inside it. Shopify has written about doing this with its large Rails codebase.4 You get most of the ownership with none of the hops, and the boundary is already drawn if a component later needs to move out.
When it’s the wrong call#
Splitting backfires when the team is small. Every service needs a pipeline, dashboards, alerts, tracing and someone on call, and with five engineers that work lands on the people meant to ship features. It also backfires when domain boundaries aren’t clear yet, or when services share one database. Then two services must change and deploy together for every feature. The microservices fundamentals note covers why database-per-service keeps deploys independent. Get it wrong and you’ve built a distributed monolith: the network costs without the independence.
What to measure#
Before splitting, find out which scale hurts. For traffic, check CPU and memory per copy and the database, which is usually the real limit. For teams, check deploy frequency, merge-to-production time, and how often one team’s change blocks another’s release.
After splitting, track hops on the critical path, p99 per hop, retry rate per client and pages per on-call engineer. If those climb and deploy frequency doesn’t, the split cost more than it gave back.
References#
Case studies
- Reduce costs by 90% by moving from microservices to monolith, DevClass, May 2023. Coverage of the Prime Video post. The original on primevideotech.com now redirects.
- Prime Video monolith debate, Adrian Cockcroft in The Stack. The counter-reading of the same case.
- Deconstructing the Monolith, Kirsten Westeinde, Shopify Engineering, 2019.
Essays
- Monolith First, Martin Fowler.
- Microservices, James Lewis and Martin Fowler, 2014.
A rough range, not a measurement. Payload size, serialization, connection reuse and service-mesh sidecars all move it. ↩︎
0.999⁴ ≈ 0.996, so the failure rate goes from 0.1% to about 0.4%. ↩︎
Adrian Cockcroft reads it differently: the team reused most of its code as one long-running service, scaled horizontally on ECS, which he calls the normal next step after a serverless prototype. Both readings agree the savings came from passing frames in memory. ↩︎
In the 2019 post, Shopify tracked boundary violations with a tool (Wedge) rather than blocking them outright. Strict enforcement was still a goal. ↩︎
