Skip to main contentSkip to user menuSkip to navigation

Apache RocketMQ

Master Apache RocketMQ: transactional messaging, scheduled delivery, distributed queues, and high-throughput messaging.

35 min readIntermediate
Not Started
Loading...

What is Apache RocketMQ?

Apache RocketMQ is a distributed messaging and streaming platform originally developed by Alibaba Group. It's designed for high-throughput, low-latency messaging with strong consistency guarantees. RocketMQ powers critical systems at Alibaba, handling trillions of messages daily during events like Singles' Day.

Unlike other message queues, RocketMQ provides unique features like transactional messaging, scheduled message delivery, and message filtering. It combines the best of traditional message queues with modern streaming capabilities, making it ideal for e-commerce, financial services, and IoT applications.

🧮 RocketMQ Performance Calculator

Estimate RocketMQ resource requirements and performance characteristics for your workload.

📊 Calculated Requirements

Memory Needed
1802 MB
Storage Needed
2472 GB
Throughput
10 MB/s
Bandwidth
80 Mbps
Consumer Lag
0s
Scheduled Msg/s
1000

🏗️ RocketMQ Architecture

RocketMQ follows a distributed architecture with four main components working together to provide reliable, high-performance messaging capabilities.

📋 NameServer

Registry and routing center

  • • Manages broker information
  • • Provides routing tables to clients
  • • Stateless and clustered
  • • No single point of failure

🗄️ Broker

Message storage and delivery

  • • Stores and forwards messages
  • • Manages message queues
  • • Handles replication
  • • Supports master-slave setup

📤 Producer

Message publishing clients

  • • Sends messages to brokers
  • • Supports sync/async sending
  • • Handles load balancing
  • • Transaction message support

📥 Consumer

Message consumption clients

  • • Pull model message consumption
  • • Consumer group management
  • • Message filtering support
  • • Automatic rebalancing

⭐ Unique Features

💳Transactional Messages

Ensures exactly-once delivery with two-phase commit protocol.

Transaction Message Example
// Send transactional message
TransactionMQProducer producer = new TransactionMQProducer("transaction_producer");
producer.setTransactionListener(new LocalTransactionExecuter());
producer.start();

Message message = new Message("TopicTest", "TagA", "Hello RocketMQ".getBytes());
SendResult result = producer.sendMessageInTransaction(message, null);

🔍Message Filtering

SQL-based message filtering on the broker side.

Message Filtering
// Producer sends with properties
Message message = new Message("TopicTest", "TagA", "Hello".getBytes());
message.putUserProperty("category", "electronics");
message.putUserProperty("price", "100");

// Consumer filters messages
consumer.subscribe("TopicTest", MessageSelector.bySql("category = 'electronics' AND price > 50"));

Scheduled Messages

Send messages to be delivered at a specific future time.

Scheduled Message
// Schedule message for 10 seconds later
Message message = new Message("TopicTest", "TagA", "Delayed message".getBytes());
message.setDelayTimeLevel(3); // 10 seconds delay

SendResult result = producer.send(message);

📊Message Ordering

FIFO ordering within message queues using message keys.

Ordered Messages
// Send ordered messages
String orderId = "ORDER_001";
for (int i = 0; i < 5; i++) {
    Message message = new Message("OrderTopic", ("Order step " + i).getBytes());
    // Use orderId as selector to ensure same queue
    producer.send(message, new MessageQueueSelector() {
        public MessageQueue select(List<MessageQueue> queues, Message msg, Object arg) {
            String orderId = (String) arg;
            int index = orderId.hashCode() % queues.size();
            return queues.get(Math.abs(index));
        }
    }, orderId);
}

🌍 Real-World Examples

🛒 Alibaba E-commerce

Scale: Trillions of messages daily during Singles' Day

Use case: Order processing, payment, inventory, logistics

Benefits: Transaction messages ensure payment consistency

Implementation: Order → Payment → Inventory → Shipping pipeline with transactional guarantees

🏦 Financial Services

Scale: Millions of transactions per day

Use case: Payment processing, fraud detection, audit logs

Benefits: Exactly-once delivery prevents double charges

Implementation: Account debit → Credit → Notification with transaction rollback capability

📱 IoT & Smart Devices

Scale: Billions of sensor messages

Use case: Device telemetry, command dispatch, alerts

Benefits: Message filtering reduces consumer load

Implementation: Sensor data → Filter by device type → Route to specific processing services

🎮 Gaming Platforms

Scale: Hundreds of millions of events daily

Use case: Player actions, leaderboards, rewards

Benefits: Ordered messages maintain game state consistency

Implementation: Player action → Game state update → Leaderboard → Achievement notifications

💡 Best Practices

Do's

  • Use appropriate sending mode: Sync for critical messages, async for high throughput
  • Implement proper retry logic: Handle failed sends with exponential backoff
  • Monitor consumer lag: Set up alerts for queue buildup and processing delays
  • Use consumer groups effectively: Distribute load across multiple consumer instances
  • Set appropriate retention: Balance storage costs with replay requirements

Don'ts

  • Don't ignore dead letter queues: Monitor and handle failed messages properly
  • Don't create too many topics: Each topic consumes resources, design thoughtfully
  • Don't send large messages: Keep messages under 4MB, use external storage for large data
  • Don't forget message ordering: Use message keys when order matters within a topic
  • Don't ignore security: Enable authentication and encrypt sensitive messages

⚡ Performance Comparison

FeatureRocketMQKafkaRabbitMQ
Peak Throughput1M+ msg/sec2M+ msg/sec20K msg/sec
Latency1-5ms2-10ms1-5ms
Transactional Messages✓ Native~ Limited✗ None
Scheduled Delivery✓ Built-in✗ None~ Plugin
Message Filtering✓ SQL-based~ Client-side✓ Routing
Operational ComplexityMediumHighLow-Medium
No quiz questions available
Questions prop is empty