Email Service Architecture

The SREDSimplify email system utilizes Redis Streams as a robust, asynchronous message broker for high-throughput email delivery. It replaces standard SQL outbox polling with real-time stream consumption, providing At-Least-Once (ALO) delivery guarantees, delayed retries, and Dead Letter Queue (DLQ) routing. We use Resend as the actual email delivery provider.

Architecture & Event-Driven Messaging Flow

Instead of sending emails synchronously during a request (which could block the HTTP thread for seconds), the application pushes the email payload into a Redis stream.

Core Components

• RedisStreamEmailDispatchService: The entry point. Handles idempotency checking against the PostgreSQL database and queues messages.
• RedisEmailStreamProducer: The publisher to the primary Redis stream (sred:email:send).
• EmailStreamConsumer: The long-polling worker within a Redis consumer group that executes the actual Resend API HTTP calls.
• RedisEmailRetryService: A Redis Sorted Set (ZSET) based delay engine for exponential backoff retries if the Resend API is temporarily unavailable.

Flow Diagram

sequenceDiagram
    participant App
    participant SQL_DB as PostgreSQL
    participant Redis_Stream as Stream (sred:email:send)
    participant Redis_ZSET as ZSET (sred:email:retry)
    participant Consumer as EmailStreamConsumer
    participant Provider as Resend API

    App->>App: buildIdempotencyKey()
    App->>SQL_DB: Check idempotency (skip if duplicate)
    App->>SQL_DB: Save EmailSendLog (Status: QUEUED)
    App->>Redis_Stream: XADD (EmailStreamMessage)
    
    Consumer->>Redis_Stream: XREADGROUP (Listen for jobs)
    Redis_Stream-->>Consumer: Delivers Message
    
    alt Resend API Success
        Consumer->>Provider: HTTP POST (Send Email)
        Provider-->>Consumer: 200 OK
        Consumer->>SQL_DB: Update EmailSendLog (Status: SUCCESS)
        Consumer->>Redis_Stream: XACK (Acknowledge message)
    else Resend API Failure (e.g. 429 Rate Limit)
        Consumer->>Provider: HTTP POST (Send Email)
        Provider-->>Consumer: 429 Too Many Requests
        Consumer->>Redis_ZSET: ZADD (Schedule Retry for future timestamp)
        Consumer->>Redis_Stream: XACK (Acknowledge original to remove from pending)
    end

Use Cases

• Verification Emails (Welcome & Confirm): Sent immediately upon registration.
• Password Reset Links: Time-sensitive emails requiring high deliverability.
• System Notifications: E.g., Subscription changes, Quota limit warnings.