BullMQ 5 job queue for reliably handling Node.js async tasks — covering retry strategies, Dead Letter Queue, Sandboxed Processor, and Prometheus metrics integration

When building a user registration API, you eventually find yourself wondering: "Should the HTTP response really be blocked while a welcome email is being sent?" I used to just put await sendEmail() before the response, but the moment a SendGrid timeout hit, every registration request started returning 500. When you factor in image resizing, webhook processing, and external API calls, an async job queue isn't optional — it's essential.

In the Node.js ecosystem, BullMQ is the de facto standard for job queue libraries that use Redis as a backend. It's a complete TypeScript-first rewrite of the original Bull library, and as of 2025–2026 it's actively maintained in the v5.x range. In this article, we'll walk through four topics you'll actually need in production — retry strategies (Exponential Backoff + Jitter), building your own Dead Letter Queue (DLQ), Sandboxed Processors for CPU-intensive tasks, and Prometheus metrics integration — with code throughout.

By the end of this article, you'll be able to answer questions like "A job failed — why didn't I get an alert?" and "How do I know how many retries have been made?" on your own. This is written for Node.js v20 LTS, BullMQ 5, and Redis 7.

Core Concepts

BullMQ Building Blocks

When you first encounter BullMQ, it can be confusing why Queue, Worker, and QueueEvents each exist separately. Honestly, I once tried to handle everything in a single Worker and got into a mess. Each component has a clearly defined role.

Here's the overall architecture at a glance. Moving jobs to the DLQ is not a built-in BullMQ feature — it's implemented directly at the application layer via QueueEvents.

flowchart LR
    A[API Server] -->|Add Job| B[(Redis)]
    B -->|Pull Job| C[Worker]
    C -->|Success| D[completed]
    C -->|Failure| E{Retries Remaining?}
    E -->|Yes| B
    E -->|No| F[failed]
    G[QueueEvents] -.->|Subscribe to Events| B
    G -->|Detect Retries Exhausted, Move at App Layer| H[DLQ Queue]

Job State Machine

BullMQ transitions jobs through waiting → active → completed/failed states atomically using Lua scripts. This minimizes the risk of job loss or duplicate execution. If a Worker crashes abnormally, jobs can get stuck in the active state as "stalled jobs" — the stalledInterval setting enables automatic detection and restart. However, since this can cause a job to run twice, idempotency design is a must.

Installation and Basic Setup

npm install bullmq ioredis

// src/queue/config.ts
import IORedis from 'ioredis';
 
const connection = new IORedis({
  host: process.env.REDIS_HOST || 'localhost',
  port: 6379,
  maxRetriesPerRequest: null, // Required setting for BullMQ
});
 
export { connection };

maxRetriesPerRequest: null is a required option for BullMQ. Internally, BullMQ uses blocking commands like BRPOPLPUSH, which conflict with ioredis's default maxRetriesPerRequest setting. Without this option, BullMQ throws an explicit error at startup and blocks Worker initialization.

Retry Strategy: Backoff and Jitter

BullMQ retries are controlled with the attempts and backoff options.

Fixed Backoff retries at a fixed interval. It works well for transient errors, but if multiple Workers fail simultaneously, they all retry at the same moment — this is the "Thundering Herd" problem.

Exponential Backoff increases the retry interval progressively. With delay: 3000ms and attempts: 5, four retries fire at intervals of 3s → 6s → 12s → 24s. This is especially useful when hitting external API rate limits.

Jitter adds random variance to the retry interval to prevent Thundering Herd. With jitter: 0.5, a random value in the range of 50%–100% of the delay is used. See Scenario 1 below for how to apply it in practice.

Custom Backoff is used when you need to determine the retry interval dynamically based on an external signal, such as the Retry-After header from an HTTP 429 response. Define it with a settings.backoffStrategy function on the Worker, and reference it with backoff: { type: 'custom' } when adding a job.

import { Worker, Job } from 'bullmq';
import { connection } from './config';
 
const worker = new Worker(
  'email',
  async (job) => { /* processing logic */ },
  {
    connection,
    concurrency: 10,
    settings: {
      // Example: prefer the Retry-After header value from an HTTP 429 response
      backoffStrategy: (attemptsMade: number, type: string, err: any, job: Job) => {
        if (err?.retryAfter) {
          return err.retryAfter * 1000;
        }
        return Math.pow(2, attemptsMade) * 1000;
      },
    },
  }
);
 
// Reference the function above with type 'custom' when adding a job
await emailQueue.add('jobName', data, {
  attempts: 5,
  backoff: { type: 'custom' },
});

Practical Application

Scenario 1: Email Delivery Pipeline

The most typical use case. A pattern that decouples email sending from the registration API so responses aren't delayed.

// src/queue/email.queue.ts
import { Queue } from 'bullmq';
import { connection } from './config';
 
const emailQueue = new Queue('email', { connection });
 
await emailQueue.add(
  'sendWelcomeEmail',
  { userId: 'user-123', email: 'user@example.com', name: 'Jane Doe' },
  {
    attempts: 5,
    backoff: {
      type: 'exponential',
      delay: 3000, // 1st retry: 3s, 2nd: 6s, 3rd: 12s, 4th: 24s
      jitter: 0.5, // Apply random variance in the 50%–100% range of delay to prevent Thundering Herd
    },
    removeOnComplete: 100, // Keep at most 100 completed jobs
    removeOnFail: 500,     // Keep at most 500 failed jobs
  }
);

// src/queue/email.worker.ts
import { Worker, Job, MetricsTime } from 'bullmq';
import { connection } from './config';
import sgMail from '@sendgrid/mail';
 
sgMail.setApiKey(process.env.SENDGRID_API_KEY!);
 
interface EmailJobData {
  userId: string;
  email: string;
  name: string;
}
 
export const emailWorker = new Worker<EmailJobData>(
  'email',
  async (job: Job<EmailJobData>) => {
    const { email, name } = job.data;
 
    await sgMail.send({
      to: email,
      from: 'noreply@example.com',
      subject: 'Welcome!',
      text: `Hello, ${name}. Welcome aboard.`,
    });
 
    return { sentAt: new Date().toISOString() };
  },
  {
    connection,
    concurrency: 10, // For I/O-bound jobs, roughly vCPU × 2
    metrics: {
      maxDataPoints: MetricsTime.ONE_WEEK, // Enable time-series data for exportPrometheusMetrics()
    },
  }
);
 
emailWorker.on('completed', (job) => {
  console.log(`Email sent: jobId=${job.id}`);
});
 
emailWorker.on('failed', (job, err) => {
  console.error(`Email failed: jobId=${job?.id}, error=${err.message}`);
});

Scenario 2: Implementing a Dead Letter Queue

BullMQ does not provide a DLQ as a built-in feature. You implement it yourself by subscribing to the failed event from QueueEvents and moving only jobs that have exhausted all their retry attempts to the DLQ.

Since the failed event fires even when retries are still remaining, checking the condition job.attemptsMade >= job.opts.attempts is the key.

flowchart TD
    A[Worker Processes Job] --> B{Success?}
    B -->|Yes| C[completed]
    B -->|No| D[failed event fires]
    D --> E{Retries Exhausted?}
    E -->|No| F[Wait for Backoff, Then Retry]
    F --> A
    E -->|Yes| G[Move to DLQ]
    G --> H[Store Original Job Data and Failure Reason]
    H --> I[Awaiting Manual Review]

// src/queue/dlq.ts
import { Queue, QueueEvents } from 'bullmq';
import { connection } from './config';
 
const SOURCE_QUEUE_NAME = 'email';
 
// Create once at module scope and reuse
// Re-creating on every event causes Redis connections to spike under heavy load
const sourceQueue = new Queue(SOURCE_QUEUE_NAME, { connection });
const dlqQueue = new Queue(`${SOURCE_QUEUE_NAME}-dlq`, { connection });
const queueEvents = new QueueEvents(SOURCE_QUEUE_NAME, { connection });
 
queueEvents.on('failed', async ({ jobId, failedReason }) => {
  const job = await sourceQueue.getJob(jobId);
 
  if (!job) return;
 
  // Do not move to DLQ if retries are still remaining
  const maxAttempts = job.opts.attempts ?? 1;
  if (job.attemptsMade < maxAttempts) return;
 
  // Move to DLQ after all retries are exhausted
  await dlqQueue.add(
    'dead-letter',
    {
      originalJobId: job.id,
      originalQueue: SOURCE_QUEUE_NAME,
      jobName: job.name,
      data: job.data,
      failedReason,
      failedAt: new Date().toISOString(),
      attemptsMade: job.attemptsMade,
    },
    {
      removeOnComplete: false, // Keep DLQ jobs until manually reviewed
      removeOnFail: false,
    }
  );
 
  console.warn(`Moved to DLQ: originalJobId=${job.id}, reason=${failedReason}`);
});

Jobs accumulated in the DLQ can be inspected via the BullBoard UI or reprocessed with a separate management script.

// src/queue/dlq-reprocess.ts
import { Queue } from 'bullmq';
import { connection } from './config';
 
async function reprocessDLQ(limit = 10) {
  const dlqQueue = new Queue('email-dlq', { connection });
  const emailQueue = new Queue('email', { connection });
 
  const waitingJobs = await dlqQueue.getJobs(['waiting'], 0, limit);
 
  for (const dlqJob of waitingJobs) {
    const { data, jobName } = dlqJob.data;
 
    await emailQueue.add(jobName, data, {
      attempts: 5,
      backoff: { type: 'exponential', delay: 3000, jitter: 0.5 },
    });
 
    await dlqJob.remove();
    console.log(`Re-queued: originalJobId=${dlqJob.data.originalJobId}`);
  }
 
  await dlqQueue.close();
  await emailQueue.close();
}
 
reprocessDLQ();

Scenario 3: Image Processing — Using a Sandboxed Processor

Image resizing is CPU-intensive, and running it directly in the main Worker process will block the event loop. BullMQ's Sandboxed Processor lets you run jobs in an isolated child process to protect the main process.

Because the Sandboxed Processor runs as a separate child process, the file the Worker references must be compiled JS. For this reason, even though the processor file is TypeScript, you must use CommonJS module.exports rather than ESM syntax.

// src/queue/image.worker.ts
import { Worker } from 'bullmq';
import path from 'path';
import { connection } from './config';
 
export const imageWorker = new Worker(
  'image-processing',
  path.resolve(__dirname, './processors/image.processor.js'), // Reference the compiled JS file
  {
    connection,
    concurrency: 2, // Recommended to keep at 1–2 for CPU-bound jobs
  }
);

// src/queue/processors/image.processor.ts
import { SandboxedJob } from 'bullmq';
import sharp from 'sharp';
 
// CommonJS module.exports is required because this runs as a child process
module.exports = async (job: SandboxedJob) => {
  const { inputPath, outputPath, width, height } = job.data;
 
  await sharp(inputPath)
    .resize(width, height, { fit: 'cover' })
    .jpeg({ quality: 85 })
    .toFile(outputPath);
 
  await job.updateProgress(100);
  return { outputPath };
};

Scenario 4: Prometheus Metrics Integration

For queue.exportPrometheusMetrics() to export time-series metrics (completed count, failed count, etc.), you must first enable metrics collection on the Worker. If you only set up the /metrics endpoint without this, the time-series metrics will be returned as empty values. Adding metrics: { maxDataPoints: MetricsTime.ONE_WEEK } to the Worker is a prerequisite — this was already applied in the emailWorker code in Scenario 1.

Once the Worker is configured, add a /metrics endpoint to Express.

// src/metrics/prometheus.ts
import express from 'express';
import { Queue } from 'bullmq';
import { Registry, Gauge, collectDefaultMetrics } from 'prom-client';
import { connection } from '../queue/config';
 
const app = express();
const register = new Registry();
collectDefaultMetrics({ register });
 
const emailQueue = new Queue('email', { connection });
const imageQueue = new Queue('image-processing', { connection });
const queues = [emailQueue, imageQueue];
 
const queueDepthGauge = new Gauge({
  name: 'bullmq_queue_depth',
  help: 'Number of jobs waiting in the queue',
  labelNames: ['queue', 'status'],
  registers: [register],
});
 
app.get('/metrics', async (_req, res) => {
  for (const queue of queues) {
    const counts = await queue.getJobCounts(
      'waiting', 'active', 'completed', 'failed', 'delayed'
    );
 
    for (const [status, count] of Object.entries(counts)) {
      queueDepthGauge.set({ queue: queue.name, status }, count);
    }
  }
 
  const bullMetrics = await Promise.all(
    queues.map((q) => q.exportPrometheusMetrics())
  );
 
  const customMetrics = await register.metrics();
 
  res.set('Content-Type', 'text/plain');
  res.send([customMetrics, ...bullMetrics].join('\n'));
});
 
app.listen(9100, () => {
  console.log('Prometheus metrics server started: http://localhost:9100/metrics');
});

With this setup, you can monitor queue depth, throughput, and failure rate in real time on a Grafana dashboard. If you'd prefer a standalone exporter, open-source tools like bullmq-prometheus or bullmq-exporter are ready to use as-is.

Graceful Shutdown

If a Worker receives SIGTERM and the process exits without calling worker.close(), any jobs being processed at that moment are left in Redis in the active state. On the next restart, those jobs are treated as stalled and re-executed — for jobs without idempotency, this can lead to duplicate emails or duplicate payment charges.

// src/queue/graceful-shutdown.ts
import { emailWorker } from './email.worker';
import { imageWorker } from './image.worker';
 
const workers = [emailWorker, imageWorker];
 
async function shutdown() {
  console.log('Shutdown signal received. Waiting for in-progress jobs to finish...');
 
  // close() waits for any currently running jobs to complete before shutting down the Worker
  await Promise.all(workers.map((w) => w.close()));
 
  console.log('Workers shut down successfully');
  process.exit(0);
}
 
process.on('SIGTERM', shutdown);
process.on('SIGINT', shutdown);

In containerized environments, SIGTERM is sent by default, so this handler must always be registered. If you're using Kubernetes, also consider setting terminationGracePeriodSeconds to a value longer than the maximum execution time of any job a Worker may be processing.

Pros and Cons

BullMQ vs. Alternative Tools

Throughput figures can vary by several times depending on hardware, payload size, and configuration, so run your own benchmarks against your actual workload. The table below is a relative comparison of characteristics.

BullMQ Pros and Cons

Common Mistakes in Practice

1. Missing maxRetriesPerRequest: null The most common configuration mistake. BullMQ blocks startup with an explicit error, so always verify this on first connection.

2. Skipping the DLQ move condition check QueueEvents.failed fires for jobs that are still retrying. If you don't check job.attemptsMade < maxAttempts, every failure will pile up in the DLQ.

3. Creating Queue instances repeatedly inside the DLQ handler Calling new Queue() every time inside the failed event callback causes Redis connections to spike under heavy load. Create it once at module scope and reuse it.

4. High concurrency for CPU-bound jobs For I/O-bound jobs (email, HTTP calls), vCPU × 2 is a reasonable target; for CPU-bound jobs (image processing, encryption), 1–2 is recommended. Setting high concurrency for CPU-bound jobs blocks the event loop.

5. Ignoring the Redis maxmemory-policy setting If this is not set to noeviction, Redis can arbitrarily evict job data when memory is under pressure. This is a must-check item in production.

6. No idempotency design for stalled jobs If a Worker crashes abnormally, active jobs will be re-processed. Without idempotency in the job processing logic, this leads to issues like duplicate emails or duplicate payment charges.

7. Missing the Prometheus metrics prerequisite Calling exportPrometheusMetrics() without metrics: { maxDataPoints: MetricsTime.ONE_WEEK } on the Worker will return empty time-series metrics. The Worker configuration must come first.

Closing Thoughts

For teams that already have Redis in their infrastructure, BullMQ 5 is the most proven combination for async job processing in Node.js. Going beyond a simple fire-and-forget queue — absorbing failures in external dependencies with retry strategies (Exponential Backoff + Jitter), preserving failed jobs without loss via a DLQ, and monitoring queue health in real time with Prometheus + Grafana — is what a production-grade job queue looks like.

Three steps you can take right now:

1. Spin up a local Redis and connect a basic Queue + Worker — Run the redis:7-alpine image with Docker and paste in the installation code from this article. Don't forget the maxRetriesPerRequest: null option.

2. Simulate a failure scenario — Intentionally throw an error in the Worker's processing function and set attempts: 3, backoff: { type: 'exponential', delay: 1000, jitter: 0.5 } to watch the retry logs. The behavior will click immediately.

3. Attach a DLQ event handler — Confirm that jobs are moved to the DLQ queue after retries are exhausted, then attach @bull-board/express for the BullBoard UI. Being able to visualize queue state in real time makes operational response far clearer.

If you'd rather start without Redis, pg-boss is a solid choice. But if you're targeting hundreds of jobs per second or more, or need advanced features like Flow/DAG or Sandboxed Processors, BullMQ + Redis remains the most reliable option.

References

• BullMQ Official Website
• BullMQ Official Docs — Full Guide
• BullMQ Official Docs — Retrying Failing Jobs
• BullMQ Official Docs — Prometheus Metrics
• BullMQ Official Docs — Going to Production
• BullMQ Official Docs — Sandboxed Processors
• BullMQ Official Docs — Concurrency
• BullMQ v5 Migration Notes
• BullMQ GitHub Releases
• How to Build a Job Queue in Node.js with BullMQ and Redis
• How to Implement Dead Letter Queues in BullMQ
• How to Implement Job Retries with Exponential Backoff in BullMQ
• How to Export BullMQ Metrics to Prometheus
• BullMQ 5 Background Jobs in Node.js 2026 Guide
• BullMQ Production Architecture: Patterns That Hold at 500 Jobs/Second
• BullMQ Architecture for High Traffic: Queue Isolation and Redis Clustering
• Node.js Job Queues in Production: BullMQ, Bull, and Worker Threads
• Background Job Processing in Node.js: BullMQ, Queues, and Worker Patterns 2026
• BullMQ vs Other Queue Systems
• BullMQ vs Bee-Queue vs pg-boss 2026
• BullMQ Demystified: Queue Names, Job Names, and DLQs
• bullmq-prometheus — GitHub
• bullmq-exporter — GitHub
• How I Scaled a Node.js Worker System to Handle 2M Background Jobs per Day
• Level Up Your NestJS App with BullMQ Queues, DLQs & Bull Board