SREDSimplify Engineering

Appearance

Runbook: Redis High Memory Usage ​

Trigger: Alert RedisMemoryCritical (>85% utilization). Impact: Rate limiters fail open or close, user sessions drop, and Email Streams halt.

Steps ​

  1. Identify the bloated keys:
    bash
    redis-cli --bigkeys
  2. Check for stalled Stream consumers:
    bash
    redis-cli XINFO GROUPS sred:email:send
    If pending is extremely high, restart the EmailStreamConsumer pods.
  3. Emergency Eviction (If 99% usage): Temporarily change the eviction policy to drop volatile keys:
    bash
    redis-cli config set maxmemory-policy volatile-lru

Escalation ​

If memory does not stabilize within 5 minutes, page the @backend-oncall team.

Post-action ​

  1. File an incident ticket.
  2. Schedule a Postmortem to identify why the keys were not expiring.