Redis Patterns¶

Knowing Redis commands isn't the same as knowing Redis patterns Any developer can SET key value and GET key value. The real skill is knowing which data structure to use for which problem and how to avoid the distributed systems pitfalls that turn your "fast cache layer" into a source of stale data, lost updates, and race conditions that only manifest under production load when three microservices try to update the same key simultaneously

Caching patterns¶

Cache-Aside (lazy loading)¶

The most common caching pattern: when reading data, check Redis first. If found (cache hit), return it. If not (cache miss), read from the database, store in Redis with a TTL, then return

async function getUser(id) {
    // Try cache first
    const cached = await redis.get(`user:${id}`);

    if (cached) {
        return JSON.parse(cached);  // Cache hit - return fast
    }

    // Cache miss - read from database
    const user = await db.query('SELECT * FROM users WHERE id = $1', [id]);

    if (user) {
        // Store in cache with TTL (3600 seconds = 1 hour)
        await redis.setex(`user:${id}`, 3600, JSON.stringify(user));
    }

    return user;
}

Pros: only caches data that's actually requested, handles cache misses gracefully, simple to implement. Cons: first request after TTL expiry always hits the database (thundering herd problem), stale data until TTL expires

Write-Through¶

Data is written to cache AND database simultaneously. The cache always has fresh data but writes are slower because both systems must complete

async function updateUser(id, data) {
    // Write to database first (source of truth)
    await db.query('UPDATE users SET username = $1 WHERE id = $2', [data.username, id]);

    // Write to cache
    await redis.setex(`user:${id}`, 3600, JSON.stringify(data));

    return data;
}

Pros: cache always consistent with database. Cons: every write is slower (two operations), cache stores data that may never be read again - wasting memory

Write-Behind (write-back)¶

Write to Redis immediately, then asynchronously write to the database. Extremely fast writes but risky - if Redis crashes before the async write completes, data is lost

async function logPageView(pageId) {
    // Atomic increment in Redis
    const count = await redis.incr(`pageviews:${pageId}`);

    // Every 100 views, persist to database
    if (count % 100 === 0) {
        // Don't await - fire and forget
        db.query('UPDATE page_stats SET views = $1 WHERE id = $2', [count, pageId])
            .catch(err => console.error('Failed to persist view count', err));
    }

    return count;
}

Pros: extremely fast writes (memory speed). Cons: data loss risk, complexity of background sync, reconciliation logic needed when Redis restarts

Cache invalidation¶

The hardest problem in computer science (after naming things and off-by-one errors). When data changes in the database, the cache must either update or be deleted

// Invalidation strategy 1: delete on update
async function updateUser(id, data) {
    await db.query('UPDATE users SET username = $1 WHERE id = $2', [data.username, id]);
    await redis.del(`user:${id}`);  // Delete cache - next read will repopulate
    // This is usually better than trying to update the cache with the new data
    // because it avoids consistency issues with concurrent updates
}

// Invalidation strategy 2: publish invalidation event
async function updateUser(id, data) {
    await db.query('UPDATE users SET username = $1 WHERE id = $2', [data.username, id]);
    await redis.publish('cache:invalidate', `user:${id}`);
    // Any service subscribed to 'cache:invalidate' knows to drop this key
    // This is essential in microservice architectures where multiple services cache user data
}

Rate limiting patterns¶

Sliding Window Counter¶

Track request counts in a sorted set where each request adds an entry with a timestamp score. Count requests within the time window by removing entries outside the window and checking the remaining count

const WINDOW_SIZE = 60;  // 60 seconds
const MAX_REQUESTS = 100;

async function slidingWindowRateLimit(userId) {
    const key = `ratelimit:${userId}`;
    const now = Date.now();
    const windowStart = now - (WINDOW_SIZE * 1000);

    // Remove old entries outside the window (pipeline for atomicity)
    const multi = redis.multi();
    multi.zremrangebyscore(key, 0, windowStart);
    multi.zadd(key, now, `${now}:${Math.random()}`);  // Unique member per request
    multi.zcard(key);   // Count remaining entries
    multi.expire(key, WINDOW_SIZE * 2);  // Cleanup TTL
    const results = await multi.exec();

    const currentCount = results[2];  // zcard result

    if (currentCount > MAX_REQUESTS) {
        return { allowed: false, retryAfter: calculateWaitTime(results[0]) };
    }

    return { allowed: true, remaining: MAX_REQUESTS - currentCount };
}

Token Bucket¶

A fixed number of tokens are added to a bucket at a steady rate. Each request consumes a token. If no tokens remain, the request is denied. This is simpler to implement and handles burst traffic better than fixed window

async function tokenBucketRateLimit(userId, maxTokens = 100, refillRate = 10, refillInterval = 1) {
    const key = `tokenbucket:${userId}`;
    const now = Math.floor(Date.now() / 1000);

    // Lua script for atomic token bucket logic
    const script = `
        local key = KEYS[1]
        local maxTokens = tonumber(ARGV[1])
        local refillRate = tonumber(ARGV[2])
        local now = tonumber(ARGV[3])

        local bucket = redis.call('HMGET', key, 'tokens', 'lastRefill')
        local tokens = tonumber(bucket[1]) or maxTokens
        local lastRefill = tonumber(bucket[2]) or now

        local elapsed = now - lastRefill
        local refill = math.floor(elapsed * refillRate)
        if refill > 0 then
            tokens = math.min(maxTokens, tokens + refill)
            lastRefill = now
        end

        if tokens >= 1 then
            tokens = tokens - 1
            redis.call('HMSET', key, 'tokens', tokens, 'lastRefill', lastRefill)
            redis.call('EXPIRE', key, 3600)
            return {1, tokens}
        else
            return {0, tokens}
        end
    `;

    const result = await redis.eval(script, 1, key, maxTokens, refillRate, now);
    return { allowed: result[0] === 1, remainingTokens: result[1] };
}

Distributed locks (Redlock)¶

When multiple application instances need to ensure only one runs a critical section at a time (cron jobs, resource-intensive computations, payment processing), distributed locks prevent duplicate execution

const LOCK_TTL = 30000;  // 30 seconds
const lockKey = 'lock:payment:processor';

async function acquireLock(resource) {
    const identifier = `${process.env.HOSTNAME}:${Date.now()}:${Math.random()}`;

    // SET NX - only sets if key doesn't exist
    // PX - expiry in milliseconds (auto-release if crash)
    const acquired = await redis.set(resource, identifier, 'NX', 'PX', LOCK_TTL);

    if (!acquired) {
        return null;  // Lock held by another instance
    }

    return identifier;  // Return so we can verify when releasing
}

async function releaseLock(resource, identifier) {
    // Lua script ensures we only delete OUR lock (not someone else's)
    const script = `
        if redis.call('GET', KEYS[1]) == ARGV[1] then
            return redis.call('DEL', KEYS[1])
        else
            return 0
        end
    `;

    await redis.eval(script, 1, resource, identifier);
}

// Usage
async function processPayments() {
    const lock = await acquireLock(lockKey);
    if (!lock) {
        console.log('Payment processor already running on another instance');
        return;
    }

    try {
        await processPendingPayments();
    } finally {
        await releaseLock(lockKey, lock);  // Always release in finally
    }
}

Warning: Redlock has been debated by distributed systems experts (see Martin Kleppmann's critique). For most applications the simple lock above is sufficient. For absolute correctness guarantees (financial transactions), use a consensus-based system like ZooKeeper or etcd

Pub/Sub - real-time messaging¶

Redis Pub/Sub is fire-and-forget messaging: publishers send messages to channels, subscribers receive them. There's no message persistence - if a subscriber isn't connected, it misses the message

// Publisher
async function notifyUser(userId, event) {
    await redis.publish(`user:${userId}:events`, JSON.stringify(event));
    // Any subscriber listening on this channel receives the event
}

// Subscriber (separate process or service)
const subscriber = redis.duplicate();  // Separate connection for pub/sub
await subscriber.connect();

await subscriber.subscribe('user:100:events', (message) => {
    const event = JSON.parse(message);
    console.log('User event received:', event);
    // Send WebSocket notification, trigger email, update dashboard, etc.
});

Queues with Lists and Streams¶

Simple queue with Lists¶

// Producer - add work to queue
async function enqueueEmail(userId, template, data) {
    const job = JSON.stringify({ userId, template, data, createdAt: Date.now() });
    await redis.lpush('queue:email', job);
}

// Consumer - process jobs
async function processEmailQueue() {
    while (true) {
        // BRPOP - blocking right pop (waits for items)
        const result = await redis.brpop('queue:email', 5);  // 5 second timeout
        if (result) {
            const [queue, jobData] = result;
            const job = JSON.parse(jobData);
            await sendEmail(job);
        }
    }
}

Streams - Redis' answer to Kafka¶

Redis Streams (5.0+) provide persistent messaging with consumer groups, message acknowledgment, and message history - making it suitable for reliable job processing where you can't afford to lose messages

// Producer - add to stream
async function addJob(type, payload) {
    return await redis.xadd(
        'job:stream',
        '*',  // Auto-generate ID (timestamp-sequence)
        'type', type,
        'payload', JSON.stringify(payload),
        'createdAt', Date.now().toString()
    );
}

// Consumer group - reliable processing
async function setupConsumer() {
    // Create consumer group (run once)
    await redis.xgroup('CREATE', 'job:stream', 'email-workers', '$', {
        MKSTREAM: true  // Create stream if doesn't exist
    });

    while (true) {
        // Read new messages
        const results = await redis.xreadgroup(
            'GROUP', 'email-workers', 'worker-1',
            'BLOCK', 2000,  // Block for 2 seconds
            'COUNT', 10,
            'STREAMS', 'job:stream', '>'  // '>' = never-before-read messages
        );

        if (results) {
            for (const [stream, messages] of results) {
                for (const [id, fields] of messages) {
                    try {
                        await processJob(fields);
                        await redis.xack('job:stream', 'email-workers', id);
                    } catch (err) {
                        // Failed to process - message stays in pending list
                        console.error(`Job ${id} failed:`, err);
                    }
                }
            }
        }
    }
}

Session storage¶

// Store session with user data
async function createSession(userId, userData) {
    const sessionId = crypto.randomUUID();
    await redis.hset(`session:${sessionId}`, {
        userId,
        email: userData.email,
        role: userData.role,
        ipAddress: userData.ip,
        userAgent: userData.userAgent,
        createdAt: Date.now().toString()
    });
    await redis.expire(`session:${sessionId}`, 86400);  // 24 hour expiry
    return sessionId;
}

// Validate session
async function getSession(sessionId) {
    const session = await redis.hgetall(`session:${sessionId}`);
    if (!session || Object.keys(session).length === 0) {
        return null;  // Session expired or invalid
    }

    // Touch - extend session TTL on active use
    await redis.expire(`session:${sessionId}`, 86400);
    return session;
}

// Destroy session (logout)
async function destroySession(sessionId) {
    await redis.del(`session:${sessionId}`);
}

prerequisites¶

db_10_redis_intro.md - you need Redis running and the redis-cli to experiment with the data structures. Understanding the basic types (Strings, Lists, Sets, Sorted Sets, Hashes) is required because every pattern builds on them

next → db_12_redis_security.md