Webhooks vs APIs System Design | Event-Driven vs Polling, Security & HMAC

The "Polling" vs "Push" Analogy

The classic "Don't call us, we'll call you."

Imagine you are waiting for a package to arrive.

🔄 API Polling

You go to the post office every 5 minutes and ask, "Is it here yet?"

Wasted Effort: 99% of requests say "No".
Latency: Package arrives at 1:00, you check at 1:05. Use old data for 5 mins.

⚡ Webhook (Push)

The post office calls your phone immediately when the package arrives.

Efficient: Zero wasted trips.
Real-time: You know the second it happens.

Comparison: Polling Strategies

Pattern	How it works	Pros	Cons
Short Polling	Client requests every X seconds.	Simple to check. Standard HTTP.	Wastes resources. "Chatty". High Latency.
Long Polling	Client requests, Server holds connection open until data arrives (or timeout).	Near real-time. Less chatty.	Holds server threads open. Complex to scale.
Webhooks	Server sends POST to Client URL on event.	Real-time. Server efficient.	Client must have public URL. Security risks.

Securing Webhooks (HMAC)

Since your webhook URL is public, how do you know the request actually came from Stripe/GitHub/Slack and not a hacker?

1. Shared Secret

You and the provider share a secret key (never sent over network).

2. Digital Signature

The provider hashes the payload with the secret (using HMAC-SHA256) and sends the signature in a header: `X-Hub-Signature-256`.

3. Verification

You re-calculate the hash of the received body + your secret. If `Calculated Hash == Header Hash`, it's legitimate.

Resilience & Best Practices

The Internet is Unreliable: Webhooks will fail. You must handle it.

🔁 Retries & Exponential Backoff

If your server returns `500` or times out, the provider should retry.

Immediate Retry: 0s delay.
Backoff 1: 5s delay.
Backoff 2: 30s delay.
Backoff 3: 5m delay.

🆔 Idempotency

Retries mean you might receive the same event twice (e.g., "Payment Charged").

Fix: Track `Event-ID` in a database. If you've seen ID `evt_123` before, ignore it (return 200 OK).

Python Implementation: Secure Receiver

A Flask example showing how to verify an HMAC signature (like GitHub/Stripe using SHA256).

import hmac
import hashlib
from flask import Flask, request, abort

app = Flask(__name__)
SECRET_KEY = b'my_super_secret_key_123'

def verify_signature(payload_body, header_signature):
    """
    Verify that the payload was sent by us by checking SHA256 HMAC.
    """
    if not header_signature:
        return False
        
    # Calculate HMAC signature of the payload using our secret
    mac = hmac.new(SECRET_KEY, msg=payload_body, digestmod=hashlib.sha256)
    expected_signature = 'sha256=' + mac.hexdigest()
    
    # Use compare_digest for timing-attack safe comparison
    return hmac.compare_digest(expected_signature, header_signature)

@app.route('/webhook', methods=['POST'])
def handle_webhook():
    signature = request.headers.get('X-Hub-Signature-256')
    
    # 1. SECURITY: Verify HMAC before processing!
    if not verify_signature(request.data, signature):
        abort(403, description="Invalid Signature")
        
    event = request.json
    event_id = event.get('id')
    
    # 2. IDEMPOTENCY: Check if processed
    # if db.exists(event_id): return "OK", 200
    
    # 3. Process Event
    print(f"Received verified event: {event['type']}")
    
    # 4. ACKNOWLEDGE: Always return 200 OK quickly
    return "Received", 200

if __name__ == '__main__':
    app.run(port=5000)

Summary

Use **APIs** for data fetching, **Webhooks** for asynchronous notifications.
Always verify **HMAC Signatures** to prevent spoofing.
Implement **Idempotency** to handle retries gracefully.
Return `200 OK` **immediately** (before processing heavy logic).

Webhooks vs APIs: Event-Driven Architecture