---
title: 'Queues for AI workloads: Celery vs ARQ (and when neither)'
excerpt: 'A 90-second LLM call pins a Celery process and can get billed twice. We run five isolated Celery queues in one system and ARQ in another. How to pick.'
date: '2026-06-12'
lastModified: '2026-07-14'
author: 'Teo Deleanu'
authorAvatar: '/team/teo.jpg'
tags: ['Production AI', 'AI Engineering', 'Task Queues', 'Infrastructure']
keywords: ['Celery vs ARQ', 'LLM task queue', 'Celery visibility timeout', 'asyncio task queue']
options: ['Celery', 'arq']
verdict: 'One asyncio service''s worth of IO-bound LLM calls points at ARQ, provided you build the supervision, dead-lettering, and cleanup yourself. Celery earns its configuration surface when you need queue isolation across tenants, worker fleets, scheduled jobs, and cost attribution. When the unit of work is a workflow that moves money, pick neither and move up to a durable workflow engine.'
faqs:
  - q: 'Why do long LLM calls get billed twice on Celery?'
    a: 'With late acks on a Redis broker, an unacknowledged task gets redelivered once the visibility timeout passes. Set that timeout below your longest LLM call and the broker decides your perfectly healthy worker is dead, hands the task to a second worker, and you pay for the same tokens twice.'
  - q: 'What Celery configuration works for LLM workloads?'
    a: 'Late acks, a visibility timeout comfortably above your slowest call, a prefetch multiplier of one on long-task queues, and idempotent tasks for the redeliveries that happen anyway. Celery''s defaults — early acks and a prefetch multiplier of four — were tuned for short tasks, not 90-second network calls.'
  - q: 'When is ARQ the better choice than Celery?'
    a: 'For a single asyncio service with one queue''s worth of IO-bound work. A 90-second LLM call in a prefork Celery worker parks an OS process; the same call in ARQ suspends a coroutine while the event loop serves other jobs. The trade: one broker option, one process model, and the production furniture — heartbeats, restarts, dead-letter paths, cleanup crons — is yours to build.'
featured: false
tldr: 'Most LLM systems need a task queue before they need an orchestrator. Celery earns its configuration surface at volume: a studio-scale ETL we operate runs five isolated queues with RedBeat scheduling, so one tenant''s heavy jobs cannot starve others and per-tenant token rollups make cost attribution work. ARQ fits a single asyncio service: the live demo on this site runs it with heartbeat-supervised workers and a dead-letter path for poison documents. The trap in between is a 90-second LLM call meeting Celery''s defaults, where prefetch and visibility timeouts can bill the same tokens twice. Once the unit of work is a workflow that moves money, neither queue is enough.'
keyTakeaways:
  - 'Celery''s defaults (early acks, a prefetch multiplier of four) were tuned for short tasks. A 90-second LLM call needs late acks and a visibility timeout above your slowest call, or you pay for the same tokens twice.'
  - 'Queue isolation is Celery''s best argument for multi-tenant AI work: five isolated queues keep one tenant''s backfill from starving everyone else, and per-tenant token rollups make the bill traceable.'
  - 'ARQ''s asyncio model fits IO-bound LLM calls: a 90-second call suspends a coroutine instead of parking an OS process. The production furniture, from supervision to dead-lettering, is yours to build.'
  - 'A task queue''s contract stops at single tasks. When a lost step costs money or the work must survive weeks, move up to a durable workflow engine.'
---

Two Redis-backed task queues carry LLM traffic in systems I operate. Celery runs five isolated queues on the studio-scale ETL from [the orchestration comparison](/compare/airflow-vs-celery-vs-temporal). ARQ runs the single-service RAG pipeline behind the [live demo on this site](/demo). Both push the same awkward shape of work through Redis: network calls that take 10 to 90 seconds and cost money on every attempt, whether or not anyone reads the answer.

That orchestration comparison covered spines one layer up and ended with a hedge: for a small asyncio service, the honest answer is none of them. This post is the hedge, expanded. Most LLM systems do not need an orchestrator on day one. They need a task queue, something that accepts a job, runs it on a worker, retries it on failure, and tells you when it gave up. The interesting questions are which queue, and where the ceiling is.

![Two-lane anatomy of Celery and ARQ for LLM workloads: broker and state, concurrency model, retry and ack semantics, and failure behavior, with a footer note on when durable workflow engines apply.](/blog/celery-vs-arq-anatomy.svg)

## What a long LLM call does to Celery

Celery's mental model is a broker in the middle. Producers put task messages on Redis or RabbitMQ, workers pull them off, and the default prefork pool dedicates one OS process to each running task. Two defaults matter more than the rest for LLM work: each worker process prefetches four tasks ahead of need, and tasks are acknowledged early, on delivery rather than on completion.

Walk a 90-second LLM call through those defaults. The call is pure network waiting, but it pins a whole OS process for a minute and a half, and the prefetch multiplier means up to three more tasks sit reserved behind it, invisible to idle workers elsewhere. Early acks add a second problem: the broker considers the task handled the moment the worker picks it up, so a worker that dies 80 seconds into the call takes the task with it. Silently.

The standard fix, `acks_late`, introduces a third character: the visibility timeout. With late acks on a Redis broker, an unacknowledged task gets redelivered once that timeout passes. Set it below your longest LLM call and the broker decides your perfectly healthy worker is dead, hands the task to a second worker, and you pay for the same tokens twice. The configuration that behaves for LLM work is specific: late acks, a visibility timeout comfortably above the slowest call, a prefetch multiplier of one on long-task queues, and idempotency for the redeliveries that happen anyway. Celery also ships greenlet-based pools that relieve the parked-process cost for IO-heavy work, though the ack and timeout semantics travel with you.

## What Celery buys when the shape is right

None of that disqualifies Celery. The studio-scale ETL we operate runs five isolated queues: embeddings, AI relevancy, reports, ad hoc, default. Isolation is the design, and multi-tenancy is the reason. One tenant kicking off a heavy embedding backfill cannot starve the report queue other tenants are waiting on, because the queues have separate workers. Scheduling runs on RedBeat, which keeps the beat schedule in Redis behind a lock instead of on one node's disk. And every task persists per-tenant token counts, so when usage spikes, the rollups say which tenant did the spiking and the bill lands on the right desk.

That is the workload where Celery earns its configuration surface: many queues, many machines, workloads that must not interfere, and costs that must be attributed. Its default failure mode is silence, a queue quietly deepening while every worker stays busy, so queue depth and worker liveness get monitored from outside. Accept that, and Celery is cheap horizontal scale you route by queue name.

<!-- CTA -->

## ARQ, sized to one service

The demo needed none of the above. It is one service with one queue's worth of document-processing work, already written on asyncio.

ARQ is what a task queue looks like when asyncio does the heavy lifting. Redis is the only broker it supports, jobs are coroutines, and a single worker process runs them concurrently on the event loop. For LLM calls that model is close to ideal. A 90-second call in a prefork Celery worker parks an OS process; the same call in ARQ suspends a coroutine, a few kilobytes of state, while the loop serves other jobs. IO-bound concurrency is the exact case asyncio exists for, and an LLM call is about as IO-bound as work gets. The same smallness caps ARQ's ambitions: one broker option, one process model, no routing across a fleet, and that is the trade you are accepting.

The catch is that the production furniture is on you. The demo's workers run under heartbeat supervision: a worker whose heartbeat stops exits non-zero so the platform restarts it. Failures that retrying cannot fix raise `TerminalStageError`, a dedicated exception type that dead-letters the document instead of burning attempts on a poison input. Expired uploads get swept by a cleanup cron built on a `select_expired_doc_ids` helper that has its own tests, because documents accumulate until something deletes them. [The week-one incident log](/blog/it-runs-itself-first-week) files worker supervision under the housekeeping nobody mentions: workers die, and something has to be watching. All of that was written by hand. In Celery, half of it comes in the box; in ARQ, the box is small on purpose.

## When neither is the answer

A task queue's contract is narrow: one task, retried until it succeeds or gets dead-lettered. Nothing in that contract remembers a workflow. The moment step four needs the output of step one after a deploy, or a lost step means lost money or a stuck approval, you have left task-queue territory. Bolting workflow state onto Celery with chords and database flags is how teams end up rebuilding a workflow engine with worse guarantees. [The orchestration comparison](/compare/airflow-vs-celery-vs-temporal) covers that boundary with a receipt, a production migration off Celery onto Temporal after Celery hit a real ceiling, so I will not re-argue it here. The compressed rule: workflows that move money or must survive for weeks belong on a durable workflow engine.

## Picking, compressed

The decision compresses well. One asyncio service's worth of IO-bound LLM calls points at ARQ or a peer, provided you write the heartbeats, the restarts, the dead-letter path, and the cleanup cron yourself. Celery earns its keep when you need queue isolation across tenants, worker fleets across machines, scheduled jobs, and per-tenant cost attribution; set the visibility timeout above your slowest call before production sets it for you. And when the unit of work is a workflow rather than a task, pick neither; the comparison one layer up covers the engines.

Both queues in this article are running today. The Celery deployment is summarized with the rest of the client systems in [the operator brief](/about), and the ARQ pipeline is answering questions at the [live demo](/demo), heartbeats, dead letters, cleanup cron, and all.