1. What It Is
Proxies sit between clients and your services — handling TLS termination, routing, rate limiting, and auth. We distinguish forward proxies (client-side) from reverse proxies (server-side gateway).
What:
Intermediary network proxy servers that intercept client-server packets to execute security, caching, routing, or identity masking policies.
Primary purpose:
Decoupling client applications from exact physical backend servers, enforcing centralized routing policies, and audit trails.
Usually used for:
API Gateways, corporate firewalls, reverse proxy caches, and TLS encrypt/decrypt terminators.
2. Core Mental Model
Forward proxy sits with the client; reverse proxy sits with your servers — same intermediary idea, opposite direction:
🛡️ Forward Client Shield (rare in HLD)
Sits between clients and the open internet for egress filtering — mention briefly; most interview designs focus on reverse proxies at the server edge.
🧱 Reverse Server Shield
Reverse Proxy sits between clients and backend microservices, masking server identities and managing path-based API routes.
🔒 SSL Terminator
Offload security by decrypting SSL handshakes at the edge reverse proxy, routing plain HTTP back to private application nodes.
In the room
In interviews, "API gateway" and "reverse proxy" often overlap. Clarify what the gateway owns: routing, auth, rate limits, request transformation. Don't put business logic in the proxy unless you explain why.
3. Why It Matters in HLD
Proxies terminate TLS, route traffic, and enforce policy at the edge — before requests hit your services. Three lenses:
Needed When:
Designing microservices mesh ingress boundaries, enforcing API gateway security, or monitoring corporate internet outbound ports.
Avoids:
Exposing internal server IP addresses to public attacks, duplicate SSL encryption handshakes, and uncoordinated path APIs.
Optimizes For:
System edge security, payload routing speed, SSL handshake CPU offloading, and transaction logging.
4. Architecture & Data Flow
Walk reverse-proxy flow as interview steps. Step 1 — Client TLS: HTTPS terminates at the proxy (nginx, Envoy, API gateway). Step 2 — Auth/rate limit: validate JWT, apply WAF rules, throttle abusive clients. Step 3 — Route: path-based routing to internal service pools. Step 4 — Protocol translate: public REST to internal gRPC if needed. Step 5 — Observability: inject trace headers before upstream hop.
In the room
Draw one gateway box that terminates TLS and routes /v1/users to the user service — interviewers want to see where auth and rate limiting live, not scattered across every microservice.
5. Key Characteristics
Forward vs reverse proxy, sidecar vs centralized gateway — we compare roles:
- Forward proxies govern client egress; reverse proxies protect server ingress — most HLD discussions focus on the latter:
| Dimension | Forward Proxy (Client Shield) | Reverse Proxy (Server Shield) |
|---|---|---|
| Primary Target | Shields and governs Clients (corporate workers, browsers). | Shields and coordinates Servers (web instances, microservices). |
| IP Visibility | Masks client IPs from public destination websites. | Masks backend database/server IPs from incoming clients. |
| Common Use Cases | Content filtering, corporate outbound logs auditing, cache proxy. | SSL termination, API gateway path routing, rate limiting, DDoS shield. |
6. Strategic Tradeoffs
Centralized policy enforcement trades latency and single-point complexity for security — we state both:
| Benefit | Cost |
|---|---|
| Edge Isolation (shields private VPC networks from direct client exposures, shielding against DDoS and direct port scans) | Operational Gateway Latency (adding another network proxy hop introduces small packet routing latencies) |
| Central Security Control (terminate SSL/TLS keys centrally at the gateway, avoiding CPU handshakes on app nodes) | Single Point of Failure (gateway crash cuts off all client connections, requiring active replica LB pairs) |
7. Failure / Bottleneck Awareness
Proxy as bottleneck, misconfigured timeouts, and SSL termination CPU — we name mitigations:
Problem: A single reverse proxy or API gateway fronts all inbound traffic — OOM or CPU saturation on that tier drops every client connection.
Mitigation: Run a pool of proxy nodes behind an L4 load balancer (HAProxy, AWS NLB).
Problem: Decrypting TLS at the gateway is CPU-heavy; at very high QPS the proxy tier can become the latency bottleneck.
Mitigation: Hardware SSL offload, or terminate TLS at CDN edge PoPs closer to users.
8. Common HLD Usage
API gateways and service meshes are where proxy patterns appear in interviews:
| Production System | Proxy Topology Type | Architectural Rationale |
|---|---|---|
| Netflix Streaming API Gateway | Zuul / Envoy (Reverse Proxy) | Authenticates user keys, decrypts SSL, routes video queries by URL path, and implements fallback rate limits. |
| Kubernetes Ingress | NGINX / Envoy (Reverse Proxy) | Terminates TLS at the cluster edge, routes by hostname and path to pod services, and enforces auth middleware. |
9. Decision Signals
Add a reverse proxy when TLS termination, routing, or auth must happen before app logic:
- You need to build a secure ingress boundary (API Gateway) terminating SSL, managing authorization tokens, and rate-limiting user requests.
- You must mask internal server networks or private VPC IP addresses from public client exposures.
- You want to execute path-based routing (e.g. route requests by URL subdirectory paths to isolated backend fleets).
11. Deep Dive (Optional)
Gateway Ingress Security: TLS Bridging vs TLS Termination
When deploying a reverse proxy API Gateway, systems select between two security patterns:
1. TLS Termination (Offloading)
The client establishes a TLS connection to the reverse proxy. The proxy decrypts the packets, inspects the HTTP headers, and forwards *unencrypted* plain HTTP traffic to private backend servers inside the secure VPC.
**Advantage**: Minimal CPU load on internal application servers, simple header lookups.
2. TLS Bridging (End-to-End Encryption)
The reverse proxy decrypts client packets, inspects them, and immediately *re-encrypts* them to establish a secondary TLS connection to the backend server nodes (using internal CA certificates).
**Advantage**: Zero plaintext leakage, strict corporate compliance (e.g. PCI-DSS), preventing packet interception on internal service networks.
API Gateway Responsibilities (Beyond TLS)
In microservice interviews, "reverse proxy" often means a full API gateway — Kong, Envoy, AWS API Gateway, or NGINX with plugins. Core duties beyond TLS termination:
- Authentication & authorization: Validate JWT/OAuth tokens once at the edge; forward trusted identity headers to internal services.
- Rate limiting & quotas: Per API key, per IP, per tenant — return 429 with Retry-After before traffic hits fragile backends.
- Request routing & versioning: Route
/v1/*to legacy cluster,/v2/*to new deployment; A/B traffic splits by header or percentage. - Protocol translation: Terminate public REST/JSON and fan in to internal gRPC — browsers cannot speak gRPC natively without gRPC-Web.
- Observability injection: Attach trace IDs, emit access logs, and aggregate latency metrics at the edge.
Keep business logic out of the gateway — it should enforce cross-cutting policies, not implement domain rules. See API Contract & Integration Design for error and versioning conventions the gateway exposes.
Review
How helpful was this walkthrough?
Click a star to rate. We actively use this feedback to refine and update our system design content.
Discussion
Share your thoughts, ask questions, or help others.