System Designer
Back to Reference

Caching Strategies

Cache patterns, hit rates, and invalidation

Not Started
Loading...

Caching is the single most effective way to improve performance. A well-designed cache can reduce database load by 90%, cut response times from seconds to milliseconds, and save thousands of dollars in infrastructure costs. The challenge is choosing the right caching pattern and managing cache invalidation—famously one of the two hard things in computer science.

The golden rule: cache at the highest level possible. Browser cache beats CDN, CDN beats application cache, application cache beats database cache. Use cache-aside for flexibility, write-through for consistency, and write-behind for performance. Always set TTLs—stale data is better than no data, and infinite caches always break.

Caching Strategies & Patterns

Cache patterns, eviction policies, and performance characteristics

Cache Performance by Layer
CPU L1 Cache
0.5 nshit
100 nsmiss
Redis (Local)
1 mshit
50 msmiss
CDN Edge
10 mshit
150 msmiss
Cache hits are orders of magnitude faster than misses • Hit rate is critical for performance
Caching Patterns
Cache-Aside (Lazy)
Read Flow
App checks cache, on miss reads DB + writes cache
Write Flow
App writes to DB, optionally invalidates cache
✓ Simple, cache only what's needed
✗ Cache miss penalty, stale data possible
Write-Through
Read Flow
App reads from cache
Write Flow
App writes to cache, cache writes to DB
✓ Always consistent, fast reads
✗ Slower writes, cache all data
Write-Behind (Write-Back)
Read Flow
App reads from cache
Write Flow
App writes to cache, async write to DB
✓ Fast writes, batched DB operations
✗ Data loss risk, eventual consistency
Refresh-Ahead
Read Flow
Cache proactively refreshes before expiry
Write Flow
Standard write-through or write-behind
✓ No cache miss penalty
✗ Complex logic, may refresh unused data
Eviction Policies
LRU (Least Recently Used)
Evicts oldest accessed item
✓ Good temporal locality
✗ Vulnerable to scans
LFU (Least Frequently Used)
Evicts least accessed item
✓ Protects hot data
✗ Slow to adapt
FIFO (First In, First Out)
Evicts oldest inserted item
✓ Simple, low overhead
✗ Ignores access patterns
Random
Evicts random item
✓ O(1), no metadata
✗ May evict hot data
Cache Technologies Performance
Redis (Memory)
Single-threaded, persistence optional
1M ops/sec
Memcached
Multi-threaded, memory-only, simple protocol
1.2M ops/sec
Hazelcast (JVM)
Distributed, strongly consistent, complex
800K ops/sec
DynamoDB DAX
Managed, microsecond latency, AWS only
2M ops/sec
Application Memory
In-process, no network, limited by GC
10M ops/sec
Cache Sizing Guidelines
Working Set
• 80/20 rule: 20% of data = 80% of requests
• Size cache for hot working set
• Monitor hit rates by time of day
Hit Rate Targets
• Web content: 95%+ hit rate
• Database query: 80-90%
• Session data: 99%+
Memory Planning
• Leave 25% headroom for growth
• Account for Redis overhead (3x data size)
• Plan for traffic spikes

📝 Test Your Knowledge

6 questions • Progress: 0/6