iwantcoding.com
🔥 Daily 👥 Rooms 🏆 Top Log in Sign up

SQS

By · Reviewed & last updated · Part of AWS Tutorial

SQS is AWS’s managed message queue. Two flavours: Standard (at-least-once, possibly-duplicated, possibly-reordered) and FIFO (exactly-once-processing, ordered). Great default for async work, retries, decoupling.

Send, receive, ack, DLQ

EXAMPLE
# 1) Create a queue + a dead-letter queue
aws sqs create-queue --queue-name jobs-dlq.fifo \
    --attributes 'FifoQueue=true,ContentBasedDeduplication=true'

aws sqs create-queue --queue-name jobs.fifo \
    --attributes file://attrs.json
# attrs.json
{
    "FifoQueue":         "true",
    "ContentBasedDeduplication": "true",
    "VisibilityTimeout": "60",                # seconds a worker has to ack
    "MessageRetentionPeriod": "345600",       # 4 days
    "RedrivePolicy": "{\\"deadLetterTargetArn\\":\\"arn:aws:sqs:us-east-1:123:jobs-dlq.fifo\\",\\"maxReceiveCount\\":\\"5\\"}"
}

# 2) Producer — Node SDK v3
import { SQSClient, SendMessageCommand, ReceiveMessageCommand, DeleteMessageCommand } from '@aws-sdk/client-sqs';
const sqs = new SQSClient({ region: 'us-east-1' });

await sqs.send(new SendMessageCommand({
    QueueUrl:         JOBS_URL,
    MessageBody:      JSON.stringify({ kind: 'send_email', userId: 42 }),
    MessageGroupId:   'user-42',                    // FIFO: same group → ordered
    MessageDeduplicationId: crypto.randomUUID(),    // FIFO: dedupe key
    MessageAttributes: { priority: { DataType: 'Number', StringValue: '1' } },
}));

# 3) Consumer — long-poll loop
while (true) {
    const { Messages = [] } = await sqs.send(new ReceiveMessageCommand({
        QueueUrl:            JOBS_URL,
        MaxNumberOfMessages: 10,
        WaitTimeSeconds:     20,       // long poll — cheaper + lower latency
        AttributeNames:      ['All'],
        MessageAttributeNames: ['All'],
    }));

    for (const m of Messages) {
        try {
            await handle(JSON.parse(m.Body));
            await sqs.send(new DeleteMessageCommand({
                QueueUrl: JOBS_URL,
                ReceiptHandle: m.ReceiptHandle,
            }));
        } catch (err) {
            console.error('handler error — will be re-delivered', err);
            // Don't delete: SQS re-delivers after VisibilityTimeout.
            // After maxReceiveCount, message → DLQ.
        }
    }
}

# 4) Patterns
#    - Lambda + SQS event source — managed polling, no infrastructure to run
#    - Long polling (WaitTimeSeconds=20) — kills empty receive spam
#    - VisibilityTimeout > 2 × handler runtime — avoid duplicate processing
#    - DLQ alarms — page on-call when messages land there

Why it matters

SQS is the cheapest reliable message queue you can adopt. Pair Standard queues with idempotent handlers, or use FIFO when ordering matters — both with a DLQ + alarms.

Tip: Tweak the snippet with Try it Yourself », then sit the quiz at the bottom of the page.

Example

Example
# Reliable queue — sender doesn't have to wait for receiver.
# At-least-once delivery; design idempotent consumers.
Try it Yourself »

Discussion

Loading…