Lesson 7: Microservices

A reference-style deep dive into service boundaries, RPC vs REST, service mesh, observability of many small services. Read this as an article, not a transcript. The accompanying audio is the spoken companion; the article below is the canonical written reference.

Audience. Engineers designing, building, or operating distributed systems who want a clear mental model rather than a checklist of tools.

Prerequisites. Working knowledge of HTTP, basic SQL, and the idea of running more than one server behind a load balancer.

Table of contents

1. Why microservices matters
2. The core mental model and where to start 3-12. In-depth sections below (see "Lesson body")

Lesson diagram

Figure 1. The canonical microservices topology and control flow covered in this lesson.

1. Why microservices matter

A monolith is the right choice until it isn't. Microservices let independent teams ship independently; the cost is operational complexity and distributed-system failure modes. The question is not "monolith vs microservices" but "what service boundaries match my team topology?". Conway's Law still applies.

Three motivations drive microservices adoption:

• Independent deployability. Team A ships feature X without coordinating with Team B.
• Independent scaling. The image-processing service scales to 1000 instances; the user-profile service stays at 10.
• Technology heterogeneity. Different services use different languages, databases, frameworks.

The cost: every cross-service call is a network call. Every service is a deployment unit. Every team needs to understand distributed systems.

2. Conway's Law — your org shapes your system

Conway's Law (1968): "Any organization that designs a system... will inevitably produce a design whose structure is a copy of the organization's communication structure."

Practical implication: if you have 3 teams, you will end up with 3 services. If you have 30 teams, you will end up with 30 services. The boundaries of your system are determined by the boundaries of your org.

The corollary: do not refactor your org and expect the system to stay the same. Do not refactor the system and expect the org to stay the same. They co-evolve.

3. Bounded context — the right service boundary

In Domain-Driven Design, a bounded context is the boundary inside which a domain model is consistent. The "User" concept means different things in different bounded contexts:

• In authentication, a User has credentials and roles.
• In billing, a User has a payment method and invoice history.
• In marketing, a User has preferences and engagement metrics.

Three contexts; three different "User" objects. They share an ID but not a schema. Trying to model them as one User is how monoliths become unmaintainable.

The right service boundary aligns with the bounded context. Each service owns its data model and exposes it via an API.

4. Communication styles — REST, gRPC, events

Three styles, each with a different trade-off:

REST (HTTP + JSON)

GET /orders/42 → 200 {order_id: 42, total: 99.99}
POST /orders → 201 {order_id: 43}

Properties:

• Human-readable; easy to debug with curl.
• Loose coupling; any client can consume.
• Slower than gRPC; JSON parsing overhead.

Used for: public APIs, browser-to-service, simple internal APIs.

gRPC (HTTP/2 + protobuf)

service OrderService {
    rpc GetOrder(GetOrderRequest) returns (Order);
    rpc CreateOrder(CreateOrderRequest) returns (Order);
}

Properties:

• Typed; client and server share a schema.
• Fast; binary serialization, HTTP/2 multiplexing.
• Supports streaming (client, server, bidirectional).
• Harder to debug (binary payload).

Used for: service-to-service RPC, internal microservices.

Events (queue or pub/sub)

kafka.publish('order.placed', {'order_id': 42, 'total': 99.99})

Properties:

• Async; producer does not wait for consumer.
• Decoupled; producer does not know who consumes.
• Eventual consistency; consumers process at their own pace.

Used for: domain events, integration between services, audit logs.

5. Service mesh — cross-cutting policies as infrastructure

A service mesh (Istio, Linkerd) handles cross-cutting concerns in a sidecar proxy that runs next to every service.

# Istio: mTLS for all traffic in the mesh
apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: default
spec:
  mtls:
    mode: STRICT

What the mesh handles:

• mTLS — every service-to-service call is encrypted and authenticated.
• Retries — automatic retry with exponential backoff.
• Circuit breaking — stop sending traffic to failing services.
• Telemetry — distributed traces, request metrics, error rates.
• Traffic shifting — canary, blue-green, A/B.

The cost: every request has two extra hops (in + out of the sidecar). For most services, this is negligible. For ultra-low-latency services, it can be 10-20% overhead.

6. The saga pattern — distributed transactions

Microservices forbid cross-service transactions. The saga pattern is the work-around.

A saga is a sequence of local transactions, each in its own service, with a compensating action for each step.

# Order saga
def place_order(order):
    # Step 1: create order in order_service
    order_service.create(order)
    try:
        # Step 2: charge payment in payment_service
        payment_service.charge(order)
    except PaymentFailed:
        # Compensate step 1
        order_service.cancel(order.id)
        return
    try:
        # Step 3: reserve inventory in inventory_service
        inventory_service.reserve(order)
    except OutOfStock:
        # Compensate steps 1 and 2
        payment_service.refund(order)
        order_service.cancel(order.id)
        return
    # Success

Sagas are eventually consistent: between steps, the system is in an inconsistent state. The right pattern for most business processes; the wrong pattern for money-critical operations (use a synchronous transaction system for those).

7. Service registration and discovery — how services find each other

A service does not hardcode the IPs of other services. It registers itself with a service registry and queries the registry to find others.

# Registration
registry.register('orders', '10.0.0.42', port=8080)

# Discovery
peer = registry.discover('billing')
response = http.post(f"{peer}/charge", ...)

Service registries:

• Consul — health checks, KV store, DNS interface.
• etcd — Kubernetes-native; strong consistency via Raft.
• ZooKeeper — mature, complex, used by Kafka and HBase.
• Kubernetes DNS — built-in; service-name → cluster-IP.

8. API versioning

Services evolve; consumers must not break. Three strategies:

URL path versioning

GET /v1/orders/42
GET /v2/orders/42

Easy to debug; clear which version a client uses.

Header versioning

GET /orders/42
Accept: application/vnd.myapi.v2+json

Cleaner URLs; harder to debug (the version is in the header).

No versioning (additive only)

GET /orders/42 → always includes new fields; old fields are never removed

The cleanest if you can do it. Requires careful schema design; never remove a field, only add.

9. Observability across services

You cannot debug a microservices system without three things:

• Centralized logs — every service logs to the same backend (ELK, Loki).
• Distributed traces — a request is one trace, with spans per service (Jaeger, Zipkin).
• Metrics per service — RED metrics (Rate, Errors, Duration) for every service.

The single most important practice: propagate trace context across every boundary. The trace ID and span ID must be passed in every HTTP header, every gRPC metadata, every Kafka message header.

# Outgoing HTTP request — propagate trace context
traceparent = f"00-{trace_id}-{span_id}-01"
requests.post(url, headers={'traceparent': traceparent, ...})

10. Common anti-patterns

Distributed monolith

Multiple services that must deploy together. The worst of both worlds: operational complexity of microservices + coupling of a monolith.

Symptoms: a deploy of service A requires a coordinated deploy of service B, C, D.

Chatty services

Services that make many small calls per request. Latency amplifies; failure cascades.

Fix: batch APIs, coarser-grained endpoints, async events.

Shared database

Multiple services writing to the same database. Breaks service ownership; creates hidden coupling.

Fix: each service owns its data; cross-service data goes through APIs or events.

11. The team-size rule

Microservices are not for every team. Rule of thumb:

• < 5 engineers: monolith. You cannot afford the operational cost.
• 5-20 engineers: modular monolith (one deploy, multiple modules). Bridges the gap.
• > 20 engineers: microservices, aligned to team boundaries.

The two-pizza team (small team that can be fed by two pizzas) should own one service. Larger teams can own multiple, but each team owns a small number.

12. Key takeaways

• Microservices buy independent deployability and scaling; cost operational complexity.
• Service boundaries should match bounded contexts and team boundaries.
• gRPC for service-to-service RPC; REST for public APIs; events for async decoupling.
• Service mesh handles mTLS, retries, telemetry in a sidecar.
• Saga is the pattern for cross-service workflows; not for money-critical paths.
• Observability (centralized logs, traces, metrics) is not optional.
• Avoid the distributed monolith — services must be independently deployable.

Appendix: terms

• Bounded context — the boundary inside which a domain model is consistent.
• Conway's Law — system design mirrors org structure.
• Service mesh — infrastructure that handles cross-cutting concerns via sidecars.
• Saga — a sequence of local transactions with compensating actions.
• Service discovery — dynamic lookup of service endpoints via a registry.
• Distributed monolith — microservices that must deploy together.

Appendix: source dialogue excerpt

The audio for this lesson was synthesized from the following Cantonese dialogue (verbatim, not translated):

• M: 各位同學早晨, 我係子謙。歡迎收聽系統架構課程第七課。今日嘅主題係 Microservices。…
• F: 大家好, 我係曉晴。Microservices 係將 monolithic application 拆成多個 independent service 嘅 architecture pattern, 每個 service 維護自己嘅 data 同 deploy cycle。今日我哋會拆解 microservices 嘅 …
• M: 首先講解基本概念。Microservices 嘅核心係 single responsibility, 即係每個 service 負責一個 bounded context。Context 嘅 definition 嚟自 domain driven design, 即係 business domain 入面嘅 concep…
• F: Microservices 嘅 motivation 包括 independent deploy, 即係每個 service 可以獨立 release, 唔需要 synchronize 全部 team。Independent scale, 即係 hot service 例如 checkout 可以 scale up, …
• M: Microservices 嘅 cost 包括 operational complexity, 即係以前 deploy 一個 application, 而家要 deploy 幾十個 service。Network failure, 即係 service 之間用 network call, network 一定 unre…
• F: 好, 第一個 important concept 係 service boundary design。即係點樣 split monolithic 入面嘅 module 做 service。Design heuristic 包括 business capability, 即係跟住 business function 例如…

Full dialogue contains 31 segments; see script_raw.json in the source folder.