In the previous article, you learned about API design with REST, GraphQL, and gRPC. Now let us talk about how to structure your entire application: as one big service (monolith) or many small services (microservices).
This is one of the most debated topics in software engineering. The answer is not always microservices. Many successful companies run monoliths. The right choice depends on your team size, system complexity, and stage of growth.
What is a Monolith?
A monolith is a single application that contains all your business logic. The user service, order service, payment service, and notification service all live in one codebase and deploy as one unit.
Monolith Architecture:
┌─────────────────────────────────────────────┐
│ Single Application │
│ │
│ ┌──────────┐ ┌──────────┐ ┌─────────────┐ │
│ │ User │ │ Order │ │ Payment │ │
│ │ Module │ │ Module │ │ Module │ │
│ └──────────┘ └──────────┘ └─────────────┘ │
│ ┌──────────┐ ┌──────────┐ ┌─────────────┐ │
│ │ Email │ │ Search │ │ Analytics │ │
│ │ Module │ │ Module │ │ Module │ │
│ └──────────┘ └──────────┘ └─────────────┘ │
│ │
│ Shared Database │
│ ┌──────────┐ │
│ │ PostgreSQL│ │
│ └──────────┘ │
└─────────────────────────────────────────────┘
One codebase. One deployment. One database.
Monolith Advantages
1. Simplicity. One codebase, one build, one deployment. A new developer can clone one repository and run the entire application locally.
2. Easy debugging. When something goes wrong, you have one set of logs. You can step through the code in a single debugger. There are no network calls between modules — everything is a function call.
3. Easy data access. All modules share one database. Need to join users with orders? It is a simple SQL JOIN. No network calls, no data consistency issues.
4. Fast development at small scale. With a small team (1-10 developers), a monolith is faster to develop. No need for service discovery, API contracts between teams, or distributed tracing.
5. Simple testing. You can run end-to-end tests on a single application. No need to spin up 20 services for integration testing.
6. Lower operational cost. One application to deploy, monitor, and scale. No Kubernetes cluster needed.
Monolith Disadvantages
1. Scaling is all-or-nothing. If the search module needs more CPU but the user module does not, you still scale the entire application. You cannot scale individual components.
Monolith Scaling Problem:
Search module needs 10 servers.
User module needs 1 server.
Payment module needs 2 servers.
With monolith: deploy 10 copies of the ENTIRE app.
All modules get 10 servers, even if they do not need it.
Wasted resources for User and Payment modules.
2. Deployment risk. A small change in the email module requires redeploying the entire application. If the deployment has a bug, everything goes down — not just the email feature.
3. Technology lock-in. The entire application uses the same language, framework, and database. You cannot use Python for the machine learning module and Go for the API module.
4. Slower development at large scale. With 100+ developers, merge conflicts are constant. Build times grow. A change in one module can break another. Teams step on each other.
5. Single point of failure. If the application crashes, everything is down. A memory leak in the analytics module can take down the payment system.
What are Microservices?
Microservices split the application into small, independent services. Each service owns a specific business capability, has its own codebase, its own database, and deploys independently.
Microservices Architecture:
┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐
│ User │ │ Order │ │ Payment │ │ Email │
│ Service │ │ Service │ │ Service │ │ Service │
│ │ │ │ │ │ │ │
│ [MySQL] │ │ [Postgres]│ │ [Postgres]│ │ [Redis] │
└──────────┘ └──────────┘ └──────────┘ └──────────┘
| | | |
└──────────────┴──────────────┴──────────────┘
|
[API Gateway]
|
[Clients]
Each service: own codebase, own database, own deployment.
Microservices Advantages
1. Independent scaling. Scale each service based on its own load. The search service gets 10 instances. The user service gets 2. The payment service gets 3. No wasted resources.
2. Independent deployment. Deploy the email service without touching the payment service. If the email deployment fails, only email is affected.
3. Technology freedom. Each team picks the best language and database for their service. The machine learning team uses Python. The API team uses Go. The search team uses Java with Elasticsearch.
4. Team autonomy. Each team owns a service end-to-end. They can develop, test, deploy, and operate independently. This works well for large organizations (50+ developers).
5. Fault isolation. If the email service crashes, the rest of the system keeps running. Users can still place orders and make payments.
6. Easier to understand. Each service is small and focused. A new developer on the email team only needs to understand the email service, not the entire system.
Microservices Disadvantages
1. Distributed systems complexity. Network calls fail. Latency adds up. Partial failures are hard to handle. A simple function call in a monolith becomes an HTTP/gRPC request that can timeout, fail, or return unexpected results.
2. Data consistency. Each service has its own database. You cannot do a SQL JOIN across services. Maintaining consistency across services requires patterns like Saga (explained below).
3. Operational overhead. Instead of one application, you have 20, 50, or 100 services. Each needs monitoring, logging, alerting, deployment pipelines, and health checks. You need Kubernetes, service mesh, distributed tracing.
4. Debugging is harder. A single user request might pass through 5 services. When something fails, you need distributed tracing (like OpenTelemetry) to follow the request across services.
5. Network latency. A monolith makes function calls (nanoseconds). Microservices make network calls (milliseconds). A request that touches 10 services has 10x the latency of network overhead.
6. Testing complexity. Integration testing requires running multiple services. Contract testing between services adds work. End-to-end tests are slow and brittle.
Service Discovery
In a microservices architecture, services need to find each other. When the order service needs to call the user service, how does it know the address?
Client-Side Discovery
The client queries a service registry to get the address of the service it needs.
Client-Side Discovery:
1. Order Service asks Service Registry: "Where is User Service?"
2. Service Registry returns: "10.0.1.5:8080, 10.0.1.6:8080"
3. Order Service picks one (load balancing) and makes the request.
┌───────────────┐
│ Service │ <-- User Service registers here on startup
│ Registry │ <-- Order Service registers here on startup
│ (Consul/etcd)│
└───────────────┘
^ ^
| |
┌───────┐ ┌───────┐
│ Order │ │ User │
│Service│--│Service│
└───────┘ └───────┘
Tools: Netflix Eureka, HashiCorp Consul, etcd.
Server-Side Discovery
The client sends requests to a load balancer or router. The router queries the service registry and forwards the request.
Server-Side Discovery:
1. Order Service sends request to Load Balancer
2. Load Balancer queries Service Registry
3. Load Balancer forwards request to User Service
┌───────────────┐
│ Load Balancer │ <-- knows where all services are
└───────────────┘
^ |
| v
┌───────┐ ┌───────┐
│ Order │ │ User │
│Service│ │Service│
└───────┘ └───────┘
Tools: Kubernetes (built-in service discovery), AWS ALB, Nginx.
DNS-Based Discovery
Services register with a DNS server. Other services use DNS names to find them.
DNS-Based Discovery:
Order Service calls: http://user-service:8080/users/123
DNS resolves "user-service" to the correct IP address.
Kubernetes does this automatically with internal DNS.
Inter-Service Communication
Synchronous Communication
Services call each other directly and wait for a response.
Synchronous (REST/gRPC):
[Order Service] --HTTP/gRPC--> [User Service]
| |
|<----response----------------|
Pros: Simple, immediate response.
Cons: Tight coupling, cascading failures.
If User Service is slow, Order Service is also slow.
If User Service is down, Order Service fails.
Asynchronous Communication
Services communicate through message queues. No waiting for a response.
Asynchronous (Message Queue):
[Order Service] --event--> [Kafka] --event--> [User Service]
Pros: Loose coupling, resilient to failures.
Cons: Eventual consistency, harder to debug.
If User Service is down, the message waits in the queue.
Order Service is not affected.
Which to Choose?
Decision:
Need immediate response (get user data)?
--> Synchronous (gRPC for internal, REST for external)
Fire-and-forget (send email, log event)?
--> Asynchronous (message queue)
Long-running operation (process video, generate report)?
--> Asynchronous (message queue)
Most systems use a mix of both. Synchronous for reads, asynchronous for writes and events.
Database Per Service
In a microservices architecture, each service should own its own database. No other service can access it directly.
Database Per Service:
┌──────────┐ ┌──────────┐ ┌──────────┐
│ User │ │ Order │ │ Product │
│ Service │ │ Service │ │ Service │
└──────────┘ └──────────┘ └──────────┘
| | |
┌──────────┐ ┌──────────┐ ┌──────────┐
│ User DB │ │ Order DB │ │Product DB│
│ (MySQL) │ │(Postgres)│ │(MongoDB) │
└──────────┘ └──────────┘ └──────────┘
Each service picks the best database for its use case.
No direct database access between services.
Services communicate through APIs.
Why Separate Databases?
- Loose coupling: changing the user database schema does not affect the order service.
- Independent scaling: the order database can be sharded independently.
- Technology freedom: use PostgreSQL for orders (ACID transactions) and MongoDB for products (flexible schema).
The Challenge: Cross-Service Queries
With separate databases, you cannot do a SQL JOIN across services. How do you get a user’s name on the order page?
1. API Composition: The order service calls the user service API to get the user’s name. Simple but adds latency.
2. Data Duplication: The order service stores a copy of the user’s name in its own database. Fast reads but the data may become stale.
3. CQRS (Command Query Responsibility Segregation): Separate the write model from the read model. Writes go to individual service databases. A read-optimized database (like Elasticsearch) aggregates data from multiple services for queries.
The Saga Pattern
In a monolith, a transaction can span multiple tables: “create order AND charge payment AND reserve inventory” in one database transaction. With microservices and separate databases, this is impossible.
The Saga pattern solves this by breaking a distributed transaction into a sequence of local transactions. Each service performs its local transaction and publishes an event. If any step fails, compensating transactions undo the previous steps.
Saga: Order Processing
Step 1: Order Service creates order (status: "pending")
--> publishes "OrderCreated" event
Step 2: Payment Service charges customer
--> publishes "PaymentCompleted" event
Step 3: Inventory Service reserves items
--> publishes "InventoryReserved" event
Step 4: Order Service updates order (status: "confirmed")
If Step 2 fails (payment declined):
--> Payment Service publishes "PaymentFailed"
--> Order Service receives "PaymentFailed"
--> Order Service cancels order (compensating transaction)
If Step 3 fails (out of stock):
--> Inventory Service publishes "InventoryInsufficient"
--> Payment Service refunds customer (compensating transaction)
--> Order Service cancels order (compensating transaction)
Saga Orchestration vs Choreography
Choreography: Each service listens for events and decides what to do. No central coordinator. Simpler but harder to track the overall flow.
Choreography:
Order Service --"OrderCreated"--> Event Bus
Payment Service hears "OrderCreated" --> charges customer --> "PaymentCompleted"
Inventory Service hears "PaymentCompleted" --> reserves items --> "InventoryReserved"
Order Service hears "InventoryReserved" --> confirms order
Orchestration: A central orchestrator tells each service what to do. Easier to understand and monitor but the orchestrator is a single point of failure.
Orchestration:
[Saga Orchestrator]
1. Tell Order Service: "Create order"
2. Tell Payment Service: "Charge customer"
3. Tell Inventory Service: "Reserve items"
4. Tell Order Service: "Confirm order"
If step 2 fails:
Orchestrator tells Order Service: "Cancel order"
Observability
With microservices, you need three pillars of observability to understand what is happening in your system:
1. Metrics (Prometheus + Grafana)
Numbers that tell you how the system is performing: request rate, error rate, latency, CPU usage, memory usage.
Key Metrics:
Request rate: 500 requests/second
Error rate: 0.5% (5xx errors)
P99 latency: 200ms (99% of requests complete in under 200ms)
CPU usage: 65%
Memory usage: 70%
2. Logs (ELK Stack or Loki)
Text records of what happened. Each service writes structured logs (JSON format) with a request ID that links logs across services.
Structured Log:
{
"timestamp": "2026-05-22T10:15:30Z",
"service": "order-service",
"request_id": "abc-123",
"level": "error",
"message": "Failed to create order",
"user_id": 456,
"error": "inventory insufficient"
}
3. Distributed Tracing (OpenTelemetry + Jaeger)
A trace follows a single request across multiple services. Each service adds a span to the trace showing what it did and how long it took.
Distributed Trace:
Request: "Create Order" (request_id: abc-123)
├── API Gateway [2ms]
├── Order Service [5ms]
│ ├── User Service [3ms] (get user details)
│ ├── Payment Service [150ms] (charge customer)
│ └── Inventory Service [8ms] (reserve items)
└── Email Service [2ms] (send confirmation)
Total: 170ms
The trace shows Payment Service is the bottleneck (150ms).
Monolith-First Approach
Many experts recommend starting with a monolith, even if you think you will need microservices later.
The Evolution:
Stage 1: Startup (1-10 developers)
--> Monolith. Move fast. Ship features.
Stage 2: Growth (10-30 developers)
--> Modular monolith. Separate modules with clear boundaries.
Still one deployment but organized for future extraction.
Stage 3: Scale (30-100+ developers)
--> Extract high-traffic or independently-scaling modules
into microservices. Keep the rest as a monolith.
Stage 4: Full microservices (100+ developers, proven scale needs)
--> Most modules are services. Heavy investment in infrastructure.
The Modular Monolith
A middle ground: keep one deployment but organize code into well-defined modules with clear boundaries. Each module has its own database schema (logical separation, even if it is the same physical database). Modules communicate through internal APIs, not direct database access.
Modular Monolith:
┌─────────────────────────────────────────────┐
│ Single Application │
│ │
│ ┌──────────┐ ┌──────────┐ ┌─────────────┐ │
│ │ User │ │ Order │ │ Payment │ │
│ │ Module │ │ Module │ │ Module │ │
│ │ [API] │ │ [API] │ │ [API] │ │
│ └──────────┘ └──────────┘ └─────────────┘ │
│ | | | │
│ [user_schema] [order_schema] [payment_schema] │
│ Single Database │
└─────────────────────────────────────────────┘
Modules talk through APIs, not direct DB access.
Easy to extract into a microservice later.
Real-World Migration: Netflix
Netflix is the classic example of a monolith-to-microservices migration.
2007: Netflix was a monolithic Java application. One database. One deployment.
2008: A database corruption caused a 3-day outage. Netflix could not ship DVDs. This was the turning point.
2009-2012: Netflix gradually moved to microservices on AWS. They built custom tools: Eureka (service discovery), Zuul (API gateway), Hystrix (circuit breaker), Ribbon (client-side load balancing).
Today: Netflix runs over 1,000 microservices handling over 2 billion API requests per day. Each team owns their services end-to-end.
Key lesson: Netflix did not rewrite everything at once. They extracted services one at a time over 3+ years. The migration was gradual and driven by specific pain points.
When to Choose What
Choose Monolith when:
- Small team (1-20 developers)
- Simple domain
- Starting a new project (you do not know the domain boundaries yet)
- Speed of development matters more than independent scaling
- You do not have the infrastructure expertise for microservices
Choose Microservices when:
- Large team (30+ developers working on the same system)
- Different parts of the system have very different scaling needs
- Teams need to deploy independently and frequently
- Different parts need different technology stacks
- You have the infrastructure (Kubernetes, CI/CD, monitoring) to support them
Choose Modular Monolith when:
- Medium team (10-30 developers)
- You want the simplicity of a monolith with the organizational benefits of services
- You plan to extract microservices in the future but do not need them yet
Interview Tips
When discussing architecture in a system design interview:
Do not default to microservices. Many candidates immediately say “microservices” to sound advanced. Instead, say “I would start with a monolith for simplicity, then extract services as the system grows.”
Justify your choice. “I recommend microservices here because the payment service and the video processing service have very different scaling requirements.”
Mention the Saga pattern. “Since services have separate databases, I would use the Saga pattern with choreography for distributed transactions.”
Discuss service discovery. “Services find each other through Kubernetes’ built-in DNS-based service discovery.”
Talk about observability. “I would use OpenTelemetry for distributed tracing, Prometheus for metrics, and structured logging with correlation IDs.”
Know the trade-offs. Be ready to discuss the downsides of your choice. If you choose microservices, acknowledge the operational complexity.
Related Articles
- System Design #8: API Design — REST, GraphQL, gRPC
- System Design #7: Message Queues — Kafka, RabbitMQ, SQS
- System Design #3: Load Balancers — Traffic distribution
- Go Tutorial #10: Project Structure — Organizing Go projects
What’s Next?
In the next article, System Design #10: Rate Limiting and Throttling, you will learn:
- Why every API needs rate limiting
- Token bucket, leaky bucket, sliding window algorithms
- Distributed rate limiting with Redis
- HTTP headers for rate limiting
- How to implement a rate limiter
This is part 9 of the System Design Tutorial series. Follow along to learn system design from scratch.