Skip to main contentSkip to user menuSkip to navigation
System Designer
Back to Fundamentals

Scalability Basics

20 min read•Beginner
Not Started
Loading...

What is Scalability?

Scalability is a system's ability to maintain or improve performance as workload increases. It's about handling growth—more users, more data, more requests—without breaking, slowing down, or becoming prohibitively expensive.

Why Scalability Matters

💰 Business Impact

Instagram: 13 employees supported millions of users. WhatsApp: 55 engineers for 2B users. Good scalability = lean teams.

📈 Growth Enabler

Go viral without crashing. Handle Black Friday traffic. Scale from 100 to 100M users without rewriting everything.

⚡ Cost Efficiency

Pay for what you use. Auto-scale during peaks. Scale down at night. Good scalability = optimized costs.

🎯 The Scalability Test: A truly scalable system maintains or improves performance as load increases. If doubling your users doubles your response time or costs, your system isn't scalable—it's just growing linearly. True scalability means sub-linear cost growth and stable (or better) performance.

The Scalability Challenge

Your startup launches on Monday with 100 users. By Friday, a celebrity tweet sends you to 100,000 users. Does your system survive or crash? This is the scalability challenge every successful product faces.

💥 What Happens Without Scalability

  • • Pokémon GO (2016): Crashed on launch day, couldn't handle viral success
  • • HealthCare.gov (2013): $2B spent, failed under load on day one
  • • Knight Capital (2012): Lost $440M in 45 minutes due to poor deployment scaling
  • • Common pattern: Slow pages → frustrated users → lost revenue → damaged reputation

🚀 What Good Scalability Enables

  • • Netflix: Went from DVDs to 230M+ global streaming users seamlessly
  • • Zoom: Handled 10x traffic spike during COVID-19 without major issues
  • • Discord: Scaled from gaming chat to 150M+ users across all communities
  • • Common pattern: Stable performance → happy users → sustainable growth → market dominance

Two Approaches to Scaling

🤔 The Fundamental Question: When your system hits capacity, you have two choices: make your existing machines more powerful (vertical) or add more machines (horizontal). Each has profound implications for cost, complexity, and maximum scale.

Visual Comparison

Max Capacity
100 unitsvertical
1000 unitshorizontal
Initial Cost
100 $vertical
300 $horizontal
Complexity
20/100vertical
80/100horizontal
⬆️

Vertical Scaling

(Scale Up)

Add more power to your existing machine: more CPU, RAM, disk, or network bandwidth.

✅ Advantages
  • • Simple - no architecture changes needed
  • • Low complexity - same code runs on bigger machine
  • • No distributed system challenges
  • • Easy to implement and maintain
❌ Disadvantages
  • • Hardware limits (can't scale infinitely)
  • • Expensive at high end (non-linear cost increase)
  • • Single point of failure
  • • Downtime during upgrades
Example:

Upgrading AWS EC2 from t3.medium (2 vCPU, 4GB) to m5.4xlarge (16 vCPU, 64GB) - $34/mo → $560/mo

↔️

Horizontal Scaling

(Scale Out)

Add more machines to your pool of resources and distribute load across them.

✅ Advantages
  • • Nearly unlimited scaling potential
  • • Linear cost growth (add capacity as needed)
  • • High availability (redundant machines)
  • • No downtime during scaling
❌ Disadvantages
  • • Higher complexity (load balancing, state management)
  • • Code changes required (stateless design)
  • • Distributed system challenges (consistency, latency)
  • • More operational overhead
Example:

Going from 1 server → 10 servers with load balancer. Same unit cost ($34/mo each), 10x capacity

🎯 When to Use Which?

Choose Vertical Scaling When:
  • • You're in early stages (100s to low 1000s of users)
  • • Your application isn't designed for distribution
  • • You need quick wins without code changes
  • • Database performance is the bottleneck
Choose Horizontal Scaling When:
  • • You're planning for massive growth (>100K users)
  • • You need high availability and fault tolerance
  • • You want cost-effective scaling at large scale
  • • Your workload is stateless and parallelizable

Zero to Millions: The Complete Journey

Alex Xu's Framework: This progression shows how systems evolve from a single user to millions, with each stage introducing specific challenges and architectural solutions.

Stage 1: Single Server (1-1,000 users)

Architecture

  • • Web app, database, cache on one server
  • • DNS points to single IP address
  • • Users connect directly to server

What Breaks First

  • • Server CPU/memory limits (500-1K users)
  • • Database connection pool exhaustion
  • • Single point of failure
Stage 2: Database Separation (1K-10K users)

Architecture Changes

  • • Separate web server and database server
  • • Vertical scaling for both servers
  • • Network connection between tiers

Benefits

  • • Independent scaling of web/DB tiers
  • • Better resource utilization
  • • Supports 5-10x more users
Stage 3: Load Balancer + Cache (10K-100K users)

New Components

  • • Load balancer for multiple web servers
  • • Cache layer (Redis/Memcached)
  • • CDN for static assets
  • • Session store (Redis/Database)

Performance Gains

  • • 80-90% cache hit ratio
  • • 10x faster static asset delivery
  • • Horizontal scaling capability
Stage 4: Database Scaling (100K-1M users)

Database Strategy

  • • Master-slave replication
  • • Read replicas for scaling reads
  • • Write-through cache for consistency
  • • Connection pooling

Key Decisions

  • • Split read/write traffic (90/10 ratio)
  • • Choose SQL vs NoSQL
  • • Plan for eventual consistency
Stage 5: Microservices (1M+ users)

Service Decomposition

  • • Break monolith into services
  • • Service mesh for communication
  • • Message queues for async processing
  • • Database per service

New Challenges

  • • Distributed system complexity
  • • Service discovery and config
  • • Monitoring and logging
  • • Data consistency across services

💡 Key Takeaways

  • • Scale incrementally: Don't build for 1M users when you have 1K
  • • Database is usually the bottleneck: Plan your data layer carefully
  • • Caching wins: 80% of requests can be served from cache
  • • Stateless design: Enables horizontal scaling
  • • Monitor everything: You can't optimize what you can't measure

Real-world Scaling Examples

Netflix

Challenge: 230M+ subscribers, 15,000+ titles, global streaming

Solution: Microservices (700+), AWS auto-scaling, CDN (Netflix Open Connect)

Click to explore detailed architecture →

WhatsApp

Challenge: 2B+ users, 100B+ messages/day with 55 engineers

Solution: Erlang (massive concurrency), minimal features, FreeBSD optimization

Click to explore detailed architecture →

Discord

Challenge: Real-time messaging, voice chat, 150M+ active users

Solution: Elixir clusters, Rust for performance-critical parts, global edge nodes

Click to explore detailed architecture →

Identifying Bottlenecks

Before scaling, you need to identify what's limiting your system's performance.

CPU
Warning: High CPU usage (>80%)
Solution: More cores or better algorithms
Memory
Warning: High RAM usage, swapping
Solution: More RAM or data structure optimization
Storage
Warning: Slow disk I/O, high latency
Solution: SSD, caching, or database optimization
Network
Warning: High bandwidth usage, packet loss
Solution: CDN, compression, or load balancing
Database
Warning: Slow queries, connection limits
Solution: Indexing, read replicas, or sharding

Load Testing & Metrics

Key Metrics to Monitor

  • • Response Time: How long requests take
  • • Throughput: Requests handled per second
  • • Error Rate: Percentage of failed requests
  • • Resource Usage: CPU, memory, disk, network
  • • Concurrent Users: Active users at same time

Load Testing Types

  • • Load Test: Expected normal traffic
  • • Stress Test: Beyond normal capacity
  • • Spike Test: Sudden traffic increases
  • • Endurance Test: Extended periods
  • • Volume Test: Large amounts of data

Scaling Timeline

1-1K Users
Monolith app, single database, simple deployment
Single Server
1K-10K Users
Redis/Memcached, CDN for static assets
Add Caching
10K-100K Users
Read replicas, connection pooling
Database Scaling
100K-1M Users
Multiple app servers, session management
Load Balancing
1M+ Users
Service decomposition, message queues
Microservices

Common Scaling Mistakes

Premature Optimization

Building for 1M users when you have 100. Start simple, scale when needed.

Not Measuring

Guessing at bottlenecks instead of using monitoring and profiling tools.

Only Vertical Scaling

Throwing money at bigger servers instead of architecting for horizontal scale.

🧮 Scaling Cost Calculator

Compare the costs of vertical vs horizontal scaling strategies

Inputs

users
users
$/month

Scaling Comparison

Vertical Scaling
3162 $/month

Vertical scaling becomes 31.6x more expensive due to premium hardware costs

Scale Factor:10x
Cost Multiplier:31.6x
Total Servers:1 (bigger)
Horizontal Scaling
1000 $/month

Horizontal scaling requires 10 servers but costs scale linearly

Scale Factor:10x
Number of Servers:10
Cost per Server:$100
💡 Quick Comparison: Horizontal scaling is 3.2x cheaper for this scenario

🎯 Real-world Scaling Scenarios

Explore how different companies handle scaling challenges

Metrics
Normal Traffic
1K RPS
Peak Traffic
10K RPS
Response Time
2.5s
Error Rate
15%
Outcome

Without auto-scaling, servers crashed. Lost $2M in sales during 3-hour outage.

Lessons Learned
  • Pre-scale infrastructure before known traffic spikes
  • Implement auto-scaling with aggressive scaling policies
  • Use CDN and caching to reduce server load
  • Load test with 10x expected traffic
No quiz questions available
Quiz ID "scalability-basics" not found