CDN System Design | Caching, Anycast DNS & Edge Routing

High-Level Architecture

A CDN places Points of Presence (PoPs) geographically closer to users to terminate TCP connections faster and serve cached static content.

User (US)

↔

Edge
(US-East)

⬇ ⬆

Origin
Server

⬆ ⬇

Edge
(EU-West)

↔

User (EU)

Push vs Pull Caching

How does data get to the edge?

Strategy	Concept	Pros	Cons
Pull CDN (Standard)	Edge fetches functionality from Origin on Cache Miss.	Low maintenance. Zero redundant storage.	First user suffers high latency ("Cold Start").
Push CDN	Engineer manually uploads content to CDN.	Content is always close to user.	High maintenance. Complex invalidation.

Routing: Anycast vs Geo-DNS

How do we route a user to the nearest Edge server?

🌍 Geo-DNS

DNS server checks Client IP -> Looks up Geo Database -> Returns IP of nearest server.

Slow (DNS Caching issues)

📡 Anycast DNS

All Edge servers share the Same IP Address. BGP routing automatically sends packets to the topological nearest node.

Fast & Resilient

Cache Eviction Policies

Edge storage is finite. What do we delete when it's full?

LRU (Least Recently Used): Discard items not accessed for the longest time. (Standard).
LFU (Least Frequently Used): Discard items with lowest total hit count. (Good for stable popularity).
TTL (Time To Live): Expire content based on `Cache-Control` header max-age.

Nginx Proxy Cache Example

Configuring a simple "Pull" CDN node using Nginx.

proxy_cache_path /data/nginx/cache keys_zone=mycache:10m;

server {
    listen 80;
    
    location / {
        # 1. Forward to Origin if miss
        proxy_pass http://origin_server;
        
        # 2. Store response in cache zone
        proxy_cache mycache;
        
        # 3. Cache valid rules
        proxy_cache_valid 200 302 60m; # Cache SUCCESS for 60 mins
        proxy_cache_valid 404 1m;      # Cache Not Found briefly
        
        # 4. Add header for debugging status (HIT/MISS)
        add_header X-Cache-Status $upstream_cache_status;
    }
}

Summary

Reduce Latency: Terminate SSL/TCP closer to the user to reduce Round Trip Time (RTT).
Pull Caching: Most common. Fetch on demand.
Anycast: Use BGP for network-level routing to nearest PoP.
Tiered Caching: Edge -> Regional -> Origin. Protects Origin from "Thundering Herd".