Redis

Most people meet Redis as "a cache," but that undersells it. Redis is an in-memory data structure store: a server that holds rich data types — strings, hashes, lists, sets, sorted sets, streams — in RAM and exposes atomic operations on them over the network. That combination of data structures plus microsecond latency is why Redis ends up behind caching, session stores, rate limiters, leaderboards, queues, and pub/sub, often all in the same system. This is the technology our caching and leaderboard articles keep reaching for; here's how it actually works.

Why In-Memory and Single-Threaded

Two design choices explain Redis's speed. First, it keeps data in RAM, so reads and writes avoid disk entirely — latencies are microseconds, not milliseconds. Second, command execution is single-threaded: one command runs at a time on one core, processed through an event loop. That sounds like a limitation, but it's a feature: there are no locks, no race conditions between commands, and every operation is naturally atomic. You never get a half-applied increment. The CPU is rarely the bottleneck for an in-memory store — the network and memory bandwidth are — so a single thread comfortably handles 100K+ operations per second. (Modern Redis does use threads for I/O and background tasks, but the core command execution remains single-threaded.)

The Data Structures

What sets Redis apart from a plain key-value cache like Memcached is its typed values with operations that run server-side:

• Strings — bytes, integers (atomic INCR), or blobs. The basic cache value.
• Hashes — field→value maps; store an object without serializing the whole thing.
• Lists — linked lists; LPUSH/RPOP make simple queues and stacks.
• Sets — unordered unique members; membership, union/intersection (e.g. mutual friends).
• Sorted sets (ZSET) — members ordered by score; the engine behind leaderboards and rate limiters.
• Streams — append-only log with consumer groups; a lightweight Kafka-like primitive.
• Bitmaps / HyperLogLog / Geo — space-efficient counters, approximate cardinality, and geospatial queries.

SET   session:42 "..."  EX 3600     # string + 1h TTL
INCR  views:home                    # atomic counter
HSET  user:42 name "Ada" tier gold  # object as a hash
ZADD  board 9820 alice              # sorted set → leaderboard
ZREVRANK board alice                # O(log n) rank
LPUSH jobs "task1"  /  BRPOP jobs 0  # simple queue

Persistence: RDB vs AOF

In-memory doesn't have to mean ephemeral. Redis offers two persistence mechanisms so it can survive restarts:

RDB is great for backups and fast restarts; AOF is better when you can't afford to lose more than a second of writes. The common production setup runs both — AOF for durability, RDB for quick backups — and many use a hybrid format. If you're using Redis purely as a cache, you might disable persistence entirely and treat data as recomputable.

Eviction and Expiration

To use Redis as a cache, you cap its memory with maxmemory and pick an eviction policy for when it's full: allkeys-lru (evict least-recently-used), allkeys-lfu (least-frequently-used — better for skewed access), volatile-ttl (evict soonest-to-expire among keys with a TTL), or noeviction (reject writes). Separately, keys can carry a TTL via EXPIRE. Redis expires keys both lazily (on access) and actively (a background sampler removes expired keys), balancing CPU against memory. Eviction (note: Redis approximates LRU/LFU by sampling, not a perfect global ordering) is what lets it serve as the cache layer described in our caching deep dive.

Replication and High Availability

A single Redis node is a single point of failure. Replication adds one or more replicas that asynchronously copy the primary's data — useful for read scaling and as warm standbys. For automatic failover, Redis Sentinel monitors the primary and replicas; if the primary goes down, Sentinel reaches a quorum, promotes a replica, and reconfigures clients. Because replication is asynchronous, a failover can lose the last few writes that hadn't reached the promoted replica — the same durability-vs-availability tradeoff seen across distributed systems (see our DDIA notes on replication).

Redis Cluster: Horizontal Scaling

When the dataset or write throughput outgrows one node, Redis Cluster shards the keyspace across multiple primaries. It divides the key space into 16,384 hash slots; each key maps to a slot via CRC16(key) mod 16384, and each primary owns a range of slots. Adding a node means moving some slots to it — only the affected keys migrate, not the whole keyspace.

slot(key) = CRC16(key) mod 16384

  node A: slots     0 – 5460      (+ replica)
  node B: slots  5461 – 10922     (+ replica)
  node C: slots 10923 – 16383     (+ replica)

  multi-key ops must share a slot → use a {hashtag}:
     MGET {cart:42}:items {cart:42}:total   # both hash on "cart:42"

The catch: multi-key operations (and transactions) only work if all keys live in the same slot, which you force with a hash tag — the part in {} is what's hashed. This is the main thing that makes app code Cluster-aware.

What Redis Is Actually Used For

The data structures map directly onto a surprising range of jobs:

• Cache — the classic: cache-aside in front of a database.
• Session store — fast, TTL'd user sessions shared across stateless app servers.
• Rate limiter — atomic INCR with an EXPIRE, or a sorted set for sliding windows.
• Leaderboard — sorted sets give O(log n) rank and top-k (see leaderboard design).
• Queues — lists (LPUSH/BRPOP) or streams with consumer groups.
• Pub/Sub — lightweight fan-out messaging for real-time features.
• Distributed lock — SET key val NX EX as a lease; the Redlock algorithm extends this across nodes (though it's debated — single-instance locks with fencing are often enough).

Redis vs Memcached

Pitfalls

• Big keys — a single huge value (a million-element list) makes every operation on it slow and stalls the thread; split or restructure.
• Hot keys — one key getting disproportionate traffic concentrates load on its node; client-side caching or key-splitting helps.
• Blocking commands — avoid KEYS, large SMEMBERS/HGETALL in production; use SCAN for iteration.
• Memory & persistence interaction — RDB's fork can transiently double memory; size headroom accordingly.