# Choosing the Queue Driver: Database vs Redis vs SQS for AcelleMail

> AcelleMail ships with `QUEUE_CONNECTION=database` as the default — and that's the right choice for installations up to ~500k emails per day. Past that threshold, database-queue contention starts to dominate worker time, and switching to Redis or SQS produces a step-change in throughput. This guide walks the trade-offs, the threshold where each driver wins, the migration steps, the AUTOMATION_QUEUE_CONNECTION separation pattern, and the operational gotchas you only learn after the first 3am queue incident.

- Canonical URL: https://acellesend.com/kb/articles/choosing-the-queue-driver-database-vs-redis-vs-sqs-for-acellemail
- Author: Lena Hart, Email Deliverability Editor
- Category: Email Infrastructure — Mail servers, MTA configuration, relay setup, IP management, monitoring
- Content type: guide · Difficulty: advanced
- Tags: Acellemail
- Published: 2026-07-04 · Last updated: 2026-08-15
- Reading time: 12 min
- Publisher: AcelleMail Knowledge Base, https://acellesend.com/kb

---

## What this is for

AcelleMail ships with `QUEUE_CONNECTION=database` as the default in `.env.example` line 40. The database driver puts all queue jobs in a MySQL/MariaDB table and uses row-level locks to coordinate between worker processes. It works out of the box, requires no extra services, and is **right for installations up to ~500k emails per day**.

Past that threshold, database-queue contention starts to dominate worker time. Workers spend more time waiting for locks than processing jobs. The symptoms are subtle at first — a small drop in throughput — then sudden when a campaign batch pushes contention past a critical point.

This guide walks the **three queue driver options** AcelleMail supports — database, Redis, AWS SQS — the **threshold where each wins**, the **migration path** between them, the **AUTOMATION_QUEUE_CONNECTION separation pattern** for high-volume installs, and the **operational gotchas** you only learn after the first 3am queue incident.

## How AcelleMail uses queues

Before picking a driver, understand what the queues actually do. From `resources/documents/supervisor_{master,worker}_config.tmpl`:

**Master pool** (2 processes by default):
```
queue:work --queue=import,default --tries=1 --max-time=180
```

The master pool handles:
- `import` — CSV imports of subscribers (chunky long-running work)
- `default` — miscellaneous Laravel default queue

**Worker pool** (15 processes by default):
```
queue:work --queue=high,batch,single,automation,automation-dispatch --tries=1 --max-time=180
```

The worker pool handles email sending:
- `high` — admin-priority sends (welcome to new admin user, password resets, etc.)
- `batch` — campaign sends (the high-volume firehose)
- `single` — individual transactional sends
- `automation` — automation-step emails (drip sequences, welcome series, etc.)
- `automation-dispatch` — the meta job that scans for due automations every 5 minutes

So a single AcelleMail install runs 17 worker processes (2 master + 15 worker), each polling the queue table or service for work.

**The implication for queue drivers:** with 17 workers all polling the same queue backend, the polling pattern itself matters a lot. A database queue with 17 SELECT FOR UPDATE statements firing every few seconds is a different load profile than 17 BLPOP'ing Redis clients.

## The three driver options

### Database (default)

**How it works.** A `jobs` table in your application database. Workers do `SELECT FROM jobs WHERE reserved_at IS NULL ORDER BY id LIMIT 1 FOR UPDATE SKIP LOCKED`, pick up a row, set `reserved_at = NOW()`, process, then DELETE. Failed jobs go to a `failed_jobs` table.

**Pros.**
- Zero infrastructure overhead — already have MySQL.
- Easy to inspect (`SELECT * FROM jobs` shows current backlog).
- Easy to clear (TRUNCATE in a panic).
- Transaction-safe with the rest of your data (campaign create + queue dispatch in one transaction).
- Backups capture queue state alongside application state.

**Cons.**
- **Polling load on MySQL.** Each worker polls every few seconds (configurable). 17 workers × every 3 seconds = ~5-6 SELECT queries per second baseline. Manageable but not free.
- **Row-lock contention** at high concurrency. SKIP LOCKED helps but isn't free.
- **No native blocking semantics.** Workers must poll; they can't subscribe-and-block like Redis BLPOP.
- **Queue table grows.** Completed jobs DELETE but unprocessed/retried jobs accumulate. Backlog visibility costs grow with table size.

**Right when:** <500k emails/day, single-server install, no separate cache layer planned, ops team is database-only-comfortable.

### Redis

**How it works.** Redis lists per queue name. Workers BLPOP-block on the list (`BLPOP queue:batch 5` blocks up to 5 seconds for work). When a campaign dispatches, jobs RPUSH to the list. Failed jobs go to a separate `failed_jobs` list (Redis or DB).

**Pros.**
- **Blocking semantics.** Workers sleep until work arrives — no polling overhead.
- **Sub-millisecond enqueue/dequeue.** 100k jobs/sec on commodity Redis.
- **Lower MySQL load.** Frees your application DB from queue traffic — bigger wins than people expect.
- **Native pub/sub for monitoring.** Tools like Horizon (Laravel's official queue dashboard) plug in cleanly.
- **Memory-only is fast; persistence is configurable** (RDB snapshots or AOF for durability).

**Cons.**
- **Another moving part.** Redis is a service to run, monitor, back up, secure.
- **Memory cost.** Backlogged queues live in RAM. A 1M-job backlog at ~2KB each = ~2GB Redis memory.
- **Less inspection ergonomics.** Hard to "SELECT * FROM queue" — must use Redis CLI or a tool.
- **Failover complexity.** Single-Redis is a single point of failure; HA Redis (Sentinel, Cluster) is non-trivial.

**Right when:** 500k-5M emails/day, single Redis instance acceptable, ops team has Redis experience, throughput sensitivity matters.

### AWS SQS

**How it works.** Managed queue service from AWS. Workers long-poll SQS for messages. Each message has a visibility timeout — if worker doesn't ACK within the timeout, the message reappears.

**Pros.**
- **Fully managed.** AWS handles HA, durability, scaling.
- **Massive scale.** No practical upper limit on messages/sec.
- **Pay-per-use.** $0.40 per million messages. At 10M emails/month, that's $4/month — trivial.
- **Cross-region durability.** Critical for multi-region setups.
- **Native DLQ (dead-letter queue) support.**

**Cons.**
- **AWS lock-in.** Or at least AWS-tax — outbound traffic from non-AWS to SQS costs more.
- **Per-message ordering not guaranteed** on standard SQS. FIFO SQS exists but has lower throughput.
- **30s default visibility timeout** can cause duplicate processing of slow jobs. Tune carefully.
- **Network round-trip per poll.** Slightly higher per-job latency than Redis (5-10ms vs sub-1ms).
- **Per-message size limit 256KB.** Rarely hit but does affect large payload jobs.

**Right when:** Multi-region setup, AWS-native ops team, >5M emails/day, durability requirements demand managed service.

## The threshold question

Rough heuristics:

| Daily email volume | Recommended driver | Why |
|---|---|---|
| < 500k emails/day | **Database** | Default; no operational complexity benefit yet |
| 500k - 5M emails/day | **Redis** | Sweet spot for self-hosted scale |
| > 5M emails/day | **Redis (HA) or SQS** | Database is a bottleneck; managed service worth considering |
| Multi-region | **SQS** | Cross-region durability is hard with self-hosted Redis |

These are guidelines, not laws. Other factors:

- **Existing ops competency.** If your team already runs Redis for caching, the Redis option is cheap to add. If not, the learning curve matters.
- **Database server load.** If MySQL is already at 60%+ CPU during peak campaigns, moving queue off MySQL is a quick win even at lower email volumes.
- **Monitoring requirements.** Database queues are easy to inspect with familiar SQL tools. Redis/SQS need separate dashboards.
- **Failure-mode tolerance.** Database queue is "queue down = whole app down." Redis can be down for queue while app stays up (campaigns queue locally, drain when Redis recovers — though Laravel's behavior here depends on driver config).

## Switching from database to Redis

The migration is straightforward in concept; the operational details matter.

### 1. Install + secure Redis

```bash
# Ubuntu/Debian
sudo apt install -y redis-server
sudo systemctl enable --now redis-server

# Verify
redis-cli ping  # → PONG

# Secure: edit /etc/redis/redis.conf
sudo sed -i 's/^# requirepass .*/requirepass YOUR_LONG_RANDOM_PASSWORD/' /etc/redis/redis.conf
sudo systemctl restart redis-server
```

Set `bind 127.0.0.1` if Redis lives on the AcelleMail VPS. For separate Redis, bind to the internal-network IP and firewall port 6379 to AcelleMail's IP only.

### 2. Update `.env`

```bash
QUEUE_CONNECTION=redis
REDIS_CLIENT=phpredis  # or 'predis'
REDIS_HOST=127.0.0.1
REDIS_PASSWORD=YOUR_LONG_RANDOM_PASSWORD
REDIS_PORT=6379
REDIS_DB=0
REDIS_CACHE_DB=1       # if also using Redis as cache (recommended)
```

If running Redis on a separate server, set `REDIS_HOST` to its private IP and ensure connectivity (firewall, security groups).

### 3. Drain the database queue first

You don't want jobs split across two backends during the cutover. Stop new sends, wait for the database `jobs` table to empty:

```sql
SELECT COUNT(*) FROM jobs;        -- watch until 0
SELECT COUNT(*) FROM failed_jobs; -- check; handle separately
```

Or pause active campaigns and let the queue drain naturally. For high-volume installs where you can't pause, dual-run is possible (set `QUEUE_CONNECTION=redis` but keep one worker on `--connection=database` until it drains).

### 4. Restart workers

```bash
sudo supervisorctl restart acellemail:*
```

Workers pick up the new `QUEUE_CONNECTION` from `.env`.

### 5. Verify

```bash
# Inside Redis
redis-cli
> KEYS *
# Should see keys like "queues:batch", "queues:default", etc. after first send

# In application logs
tail -f storage/logs/laravel.log
# Watch for queue-related warnings
```

Send a test campaign to a small list. Verify the job processes successfully end-to-end.

## The AUTOMATION_QUEUE_CONNECTION separation pattern

For installs with heavy automation use, AcelleMail's `.env` supports an **optional** separate queue connection for automation jobs:

```bash
QUEUE_CONNECTION=redis              # main campaign sends
AUTOMATION_QUEUE_CONNECTION=database # automations + automation-dispatch
```

**Why separate?**

The `batch` queue (campaign sends) and the `automation` queue (drip + welcome + sequence sends) have very different load profiles:

- **Campaign sends** are bursty. A big newsletter dispatch creates 100k+ jobs in seconds, then quiets down.
- **Automation sends** are steady. A welcome-series email per new subscriber, a birthday email per matching profile per day. Continuous low-rate.

If both share the same Redis instance, a campaign burst can backlog the queue and delay automation sends. By splitting them onto separate connections, you isolate the load.

**When to use:**
- 5M+ emails/month
- Time-sensitive automations (welcome series, abandoned cart) where delay is unacceptable
- Heavy automation volume relative to campaign volume

Most installs don't need this. Keep both on the same driver unless you have specific reason to split.

## Operational gotchas

### 1. Backpressure on database driver

When the `jobs` table grows past ~100k rows, MySQL query plans on `reserved_at IS NULL ORDER BY id LIMIT 1` start to degrade. The index helps but doesn't eliminate the problem.

**Symptom.** Workers report "no jobs" repeatedly while there are clearly jobs in the table. Investigation shows the worker queried for `WHERE queue = 'batch' AND reserved_at IS NULL LIMIT 1` and the database optimizer scanned thousands of rows to find one matching.

**Fix.** Add a composite index if missing:

```sql
CREATE INDEX idx_jobs_queue_reserved ON jobs (queue, reserved_at, id);
```

Or migrate to Redis, where this whole class of problem doesn't exist.

### 2. Redis memory blowup during bounce-handler runs

The bounce-handler command (`handler:run` every 30 min per `routes/console.php`) processes bounces in chunks. If you have a sudden bounce-rate spike (a list import gone wrong), Redis can balloon to GBs in minutes.

**Symptom.** Redis memory usage spikes, then crashes with OOM.

**Fix.** Set `maxmemory` + `maxmemory-policy noeviction` in `/etc/redis/redis.conf`. With `noeviction`, Redis refuses writes when full instead of evicting queue jobs (which would lose work). Pair with monitoring alerts.

### 3. SQS visibility timeout vs slow campaign jobs

A campaign batch dispatch can take 30-60 seconds for very large lists. If SQS visibility timeout is 30s default, SQS re-queues the message at 30s, and you process it twice.

**Symptom.** Some recipients get duplicate emails after a campaign send.

**Fix.** Increase SQS visibility timeout on the queue: `aws sqs set-queue-attributes --queue-url ... --attributes VisibilityTimeout=300`. Or set per-message via `--message-attributes`.

### 4. Stale workers after deployment

After deploying new AcelleMail code, workers keep running with the old code in memory. They process new jobs with the stale logic.

**Symptom.** Jobs that should use the new feature use the old behavior; logs show old class definitions.

**Fix.** Restart workers after every deployment:

```bash
sudo supervisorctl restart acellemail:*
```

Or use Laravel's queue:restart command for graceful restart:

```bash
sudo -u www-data php artisan queue:restart
```

Workers will finish their current job, then restart with new code. Add this to your deploy script.

### 5. Failed jobs invisible without inspection habits

Failed jobs go to `failed_jobs` table (database) or a separate Redis list. Both are easy to forget.

**Symptom.** Days later, you notice 500 jobs in failed_jobs from a misconfigured template last Tuesday. Subscribers never got those emails.

**Fix.** Cron-scheduled check on `failed_jobs` count + alerting:

```bash
# In a monitoring script
COUNT=$(php artisan tinker --execute='echo \DB::table("failed_jobs")->count();')
if [ "$COUNT" -gt 0 ]; then
  # alert ops
fi
```

For Redis-backed queues, Laravel Horizon dashboard shows failed jobs visually. Worth the setup time for any install with Redis already running.

### 6. Long-running batch jobs hitting max-time

The supervisor config has `--max-time=180` (3 minutes). Workers self-terminate after 180s and supervisor restarts them. This is normal and intentional — workers can leak memory; periodic restarts protect against that.

**Symptom.** Long campaign batches that take >3 min seem to silently restart mid-stream.

**Fix.** Acelle's batch-dispatch design accounts for this — the campaign's `dispatchWithBatchMonitor` creates a Job Batch that survives worker restarts. Each batch sub-job is short (a few subscribers). If you see longer-running jobs, that's usually a sign of bad batching logic or a custom worker that's not following Acelle's patterns.

## Inspecting queue state

### Database queue

```sql
-- Current queue depth
SELECT queue, COUNT(*) FROM jobs GROUP BY queue;

-- Stuck/long-running jobs
SELECT id, queue, payload->>'$.displayName', attempts, reserved_at
FROM jobs
WHERE reserved_at IS NOT NULL
  AND reserved_at < NOW() - INTERVAL 5 MINUTE;

-- Failed jobs
SELECT id, queue, exception, failed_at
FROM failed_jobs
ORDER BY failed_at DESC LIMIT 20;
```

### Redis queue

```bash
redis-cli

> LRANGE queues:batch 0 10  # peek at first 10 jobs in batch queue
> LLEN queues:batch          # depth of batch queue
> KEYS queues:*              # all active queue keys
> SMEMBERS failed_jobs       # failed-job IDs
```

For pretty visualization, use **Laravel Horizon** (Composer install + small config). It's overkill for tiny installs but excellent for any operator running Redis-backed queues seriously.

### SQS queue

```bash
aws sqs get-queue-attributes \
  --queue-url https://sqs.region.amazonaws.com/account/your-queue \
  --attribute-names All

# ApproximateNumberOfMessages = current backlog
# ApproximateNumberOfMessagesNotVisible = currently being processed
```

## When to think about queue at all

You don't need to think about queue drivers until you hit one of:

- Your `jobs` table exceeds 100k rows during peak
- MySQL CPU hits >70% during campaign sends
- Sub-second delivery latency matters (transactional sends, etc.)
- You're planning >2M emails/day routinely
- An incident makes you realize you couldn't restore the queue from a backup easily

If none of those apply, the database default is doing its job. Spend the ops cycles elsewhere.

## Related reading

- [Setting Up Queue Workers and Cron Jobs](https://acellesend.com/kb/articles/setting-up-queue-workers-and-cron-jobs) — foundation; covers supervisor config
- [Email Queue Architecture in AcelleMail](https://acellesend.com/kb/articles/email-queue-architecture-in-acellemail) — how dispatch + batch monitoring works internally
- [Redis for Queue Processing](https://acellesend.com/kb/articles/redis-for-queue-processing) — Redis-specific deep dive
- [Scaling for 100k Emails per Day](https://acellesend.com/kb/articles/scaling-for-100k-emails-per-day) — sharded-DB pattern for the next scale tier
- [Why Is My AcelleMail Campaign Stuck in "Sending" Status](https://acellesend.com/kb/articles/why-is-my-acellemail-campaign-stuck-in-sending-status) — queue worker troubleshooting
- [Why Is My AcelleMail Automation Not Triggering](https://acellesend.com/kb/articles/why-is-my-acellemail-automation-not-triggering) — automation-queue specific
- [Post-Install Hardening Checklist](https://acellesend.com/kb/articles/post-install-hardening-checklist) — including Redis password + firewall
- [Automated Database Backups](https://acellesend.com/kb/articles/automated-database-backups) — protect both database and Redis state

---

## Related articles

- [How Email Delivery Actually Works (What Happens After You Click Send)](https://acellesend.com/kb/articles/how-email-delivery-actually-works): When you press Send Campaign in AcelleMail, your message travels through five stages before it lands in someone's inbox. Knowing each stage tells you exactly where to look when a campaign goes sideways — and why some recipients see the email and others don't.
- [Dedicated vs Shared Sending IP: Which to Pick](https://acellesend.com/kb/articles/dedicated-vs-shared-ip-address): AcelleMail sends through whatever sending server you connect, so the dedicated-vs-shared IP choice is really a choice about your sending provider. Here's how to decide, and where it lives in AcelleMail.
- [Email Queue Architecture in AcelleMail (Why Sends Are Steady, Not Sudden)](https://acellesend.com/kb/articles/email-queue-architecture-in-acellemail): AcelleMail uses a job queue to pace out large sends — instead of trying to push 50,000 emails at once, workers pick recipients off a queue one at a time. This explainer covers why that matters for deliverability and how to read the campaign status flow.
- [BIMI with VMC — Cost, Process, and Whether It's Worth It](https://acellesend.com/kb/articles/bimi-with-vmc-cost-walkthrough): VMC (Verified Mark Certificate) costs $1,500/yr per domain and takes 4-8 weeks to obtain. This guide walks the cost math, the trademark requirements, the alternative paths, and a decision framework for whether VMC is right for your sending program.
- [Building Your Email Marketing SaaS](https://acellesend.com/kb/articles/building-your-email-marketing-saas): The technology side of an AcelleMail SaaS is solved. The hard part is the business: who you sell to, how you price, how you reach them, and how you keep them. This guide is the GTM + pricing playbook — opinionated, market-tested patterns for launching an email-marketing SaaS on the AcelleMail stack.