Skip to main contentSkip to user menuSkip to navigation
System Designer
Back to Practice Hub

Design a RAG (Retrieval-Augmented Generation) System

Problem: Design an enterprise RAG system that combines document retrieval with large language models to provide accurate, contextual responses. Handle 10M documents, 100K users, 1M daily queries with <3s latency while maintaining high accuracy and preventing hallucinations.

Q: What types of documents will the RAG system need to handle?
Interviewer: Text documents, PDFs, web pages, code repositories, and some structured data like tables.
Analysis: This requires multi-modal processing pipelines and different chunking strategies per document type.
Q: What's the expected scale in terms of document volume and user queries?
Interviewer: 10 million documents totaling ~500GB, with 100K daily active users generating 1M queries/day.
Analysis: Peak load: 2K QPS requiring distributed architecture with sub-3-second response times.
Q: What accuracy and quality requirements do we have?
Interviewer: Retrieval recall >85%, response relevance >90%, with factual accuracy being critical.
Analysis: Need hallucination detection, source attribution, and confidence scoring for enterprise use.
Q: Are there real-time requirements or is batch processing acceptable?
Interviewer: Real-time query responses needed ({'<'}3s), but document ingestion can be near real-time (5-10 minutes).
Analysis: Async document processing pipeline with incremental index updates and embedding generation.
Q: What are the privacy and security requirements?
Interviewer: Enterprise-grade: data isolation, access controls, PII detection, and audit logging required.
Analysis: Multi-tenant architecture with role-based access control and data residency compliance.

Summary of Requirements

Functional Requirements:

  • • Multi-document type processing (PDF, text, code)
  • • Semantic search with hybrid retrieval
  • • Context-aware response generation
  • • Source attribution and citation

Non-Functional Requirements:

  • • <3s query response time
  • • Recall >85%, relevance >90%
  • • 10M documents, 100K DAU
  • • Enterprise security & compliance
No quiz questions available
Quiz ID "rag-system" not found

Interview Practice Questions

Practice these open-ended questions to prepare for system design interviews. Think through each scenario and discuss trade-offs.

1

Scale Challenge: Your RAG system needs to handle 10x more queries (100M/day) while maintaining <200ms response time. Walk through your scaling strategy including retrieval, LLM inference, and data pipeline optimization.

2

Data Freshness: How would you design a real-time update system where new documents need to be searchable within 30 seconds of ingestion? Consider embedding generation, vector indexing, and consistency challenges.

3

Multi-tenancy: Design a RAG system serving 1000+ enterprise customers, each with their own private knowledge base. How do you ensure data isolation, optimize costs, and handle varying query volumes per tenant?

4

Failure Scenarios: Your vector database cluster goes down during peak traffic. How do you maintain service availability? Design your fallback mechanisms and recovery procedures.

5

Quality Control: How would you build an evaluation system to continuously monitor RAG answer quality? Include metrics, testing frameworks, and feedback loops for model improvement.

6

Cost Optimization: Your RAG system costs are growing rapidly due to LLM inference and vector storage. Design a cost optimization strategy while maintaining quality and performance SLAs.