Download the PHP package webpatser/torque without Composer

Torque

The queue that keeps spinning. Coroutine-based queue worker for Laravel.

Torque replaces Horizon's 1-job-per-process model with N-jobs-per-process using PHP 8.5 Fibers. When a job waits on I/O, the scheduler switches to another job, so a handful of processes deliver the throughput Horizon needs dozens of processes for.

[!NOTE] Numbers from the fair benchmark. On long-running async I/O (HTTP fan-out, slow external APIs) Torque delivers up to 15x throughput at 95% lower memory footprint. On pure CPU work it is comparable or slightly slower than Horizon, by design.

[!TIP] Live job progress, built in. Every job records a per-job event timeline (queued / started / exception / completed) to a Redis Stream. Tail it from the CLI with torque:tail --job=<uuid>, read it programmatically, or stream it to the dashboard / your own UI. Custom progress events are a one-line $this->emit('...', progress: 0.42) away. No log scraping, no separate progress table, no extra Redis keys to manage.

When to use Torque

• HTTP fan-out: calling N external APIs per job
• Slow external services (>50 ms latency per call)
• Webhook delivery to many endpoints
• Bulk operations against rate-limited APIs
• Search index updates, cache warming, notification fan-out
• Any I/O-bound workload where you currently scale by adding more Horizon processes

When to use Horizon instead

• CPU-bound jobs (image processing, PDF generation, encoding, ML inference)
• Jobs using sync-blocking calls (curl_exec, usleep, PDO without an async wrapper)
• Mature ops tooling around Horizon (we are catching up, not there yet)
• Workloads where memory footprint per worker is not a concern

[!IMPORTANT] Torque only wins when your jobs spend time waiting. If they spend time computing, the Fiber scheduler has nothing to switch to and you pay overhead for nothing. Use the right tool for the workload.

Requirements

• PHP 8.5+
• Laravel 12+
• Redis 7+ or Valkey (Redis Streams support)
• Revolt event loop (installed automatically)

Installation

Publish the config:

Add the queue connection to config/queue.php:

Set it as default in .env:

Usage

Starting the worker

Options:

Dispatching jobs

Standard Laravel dispatching works unchanged:

Async jobs with TorqueJob

Regular Laravel jobs work fine; they run synchronously within their coroutine slot. For full async I/O, extend TorqueJob and type-hint the pools you need:

Working with databases (avoid the Eloquent trap)

[!WARNING] Eloquent uses PDO, which is sync-blocking. One User::find($id) in a handler stalls the entire worker (and every other Fiber on it) until the round-trip completes. On a 25-coroutine worker that's effectively concurrency 1 for the duration of that call, which destroys the fanout advantage Torque is built around.

The fix is to either keep Eloquent out of the handler, or use the async MysqlPool for the queries that matter:

When sync Eloquent is fine:

• The handler does at most one or two queries
• The queries are fast (<5 ms) and the worker isn't fanout-heavy
• You're running a CPU or low-concurrency workload where Fiber concurrency wasn't the goal anyway

When sync Eloquent is a footgun:

• HTTP fan-out jobs (the fanout workload from BENCHMARKS.md)
• Long-running queries (slow joins, large scans)
• Anywhere you'd otherwise be looking at "why am I not getting the throughput Torque promised"

Same pattern applies to other sync clients: curl_exec, Guzzle without a non-blocking handler, usleep, blocking file I/O. Replace with HttpPool, Fledge\Async\delay(), or pre-compute outside the handler.

Per-Fiber state isolation

Use CoroutineContext when you need per-job isolated state (e.g., request-scoped data):

State is automatically cleaned up when the Fiber completes (backed by WeakMap).

Live job progress (built in)

Every job automatically records a lifecycle timeline to a per-job Redis Stream: queued, started, exception, completed, plus any custom events you emit. Watch it live from the CLI, the dashboard, or your own UI without instrumenting each job by hand.

Custom progress events

Add the Streamable trait to emit progress from inside your job:

Reading streams programmatically

Streams auto-expire after 5 minutes (configurable via job_streams.ttl).

Redis Cluster Support

Torque supports Redis Cluster out of the box. Enable it in your .env:

When cluster mode is enabled, all Redis keys for a given queue are wrapped in hash tags ({queue-name}) so they land on the same cluster slot. This ensures Lua scripts and multi-key operations work correctly across the stream, delayed set, and notification keys.

If your queue names already contain hash tags (e.g., {myqueue}), Torque will not double-wrap them.

CLI Commands

Configuration

All options are in config/torque.php. Key settings:

Autoscaling

Connection pools

Circuit breaker

When a stream's dependency dies (an API returning 500s, expired credentials), every job on it fails permanently and the whole backlog burns straight into the dead-letter stream. The breaker stops that: once the permanent-failure ratio of a stream's recent outcomes crosses threshold, that stream is no longer polled for cooldown seconds. Other streams keep running.

Only permanent failures (jobs that reached the dead-letter stream) count as failures; completions count as successes and a retry is neutral. After the cooldown the breaker goes half-open: up to half_open_max jobs are let through, all of them failing re-opens it for another cooldown, and the first success closes it and resets the window.

Override per stream, or opt a stream out entirely:

State is shared across the whole fleet through Redis (every key carries an expiry) and an open breaker is expressed as a paused queue, exactly like php artisan queue:pause <name>. Listen for QueueCircuitOpened / QueueCircuitClosed to alert on it, check torque:status for a line per tripped stream, and force one closed with php artisan torque:pause continue or php artisan queue:resume torque:<queue> once the upstream problem is fixed.

Dashboard

Torque includes a self-contained dashboard at /torque (configurable). It is built with Livewire 4 and plain Blade + Tailwind, served Horizon/Pulse-style: a set of full-page Livewire components share one Blade layout, and the compiled dist/torque.css is inlined into it. Livewire (a hard dependency) ships its own runtime and bundled Alpine through your app, so there is no React, no Flux, no paid UI library, no host Tailwind or Vite build, and no npm step in your application. Enable it in config:

Features:

• Real-time metrics (throughput, latency, concurrent jobs, memory) with a damped throughput gauge: the needle shows a smoothed five-minute jobs/min figure plus the exact count for the last hour, so a burst of 1500 jobs every five minutes reads as 300/min instead of 0 and then 90K
• Long-term history: minute, hour and day rollups (cluster-wide and per stream) power the 1h / 24h / 7d / 90d throughput chart and the processed / failed today columns on the Queues page
• Worker table with coroutine slot usage bars
• Jobs page: throughput, average and peak runtime, and fail rate per job class over 1h / 24h / 7d / 90d
• Stream/queue overview with pending and delayed counts
• Failed jobs list with retry and delete actions (cursor-paginated)
• Per-job inspector with a timeline of lifecycle events and exception details
• Live feed with a rolling stream tail and status filters
• Kibana-style configurable poll interval (1s to 30s, or paused)

Assets

The dashboard ships pre-built. The compiled dist/torque.css lives in the package and is inlined into the layout at request time, so there is nothing to publish and no build step in your application. The reactivity comes from Livewire's own runtime, which your app already loads; the chrome (refresh popover, theme and sidebar toggles, copy button, row links) is a small nonce'd vanilla script in the layout. If you are working on Torque itself, rebuild the stylesheet with:

Content-Security-Policy

The dashboard works under a strict CSP with a nonce and needs no 'unsafe-eval'. Torque stamps its inline <style> and <script> tags with whatever nonce you set, either through Vite::useCspNonce() (picked up automatically, same convention Livewire follows) or explicitly:

The dashboard chrome uses no Alpine expressions on purpose: Alpine compiles every directive expression (x-data, x-show, @click, :class, $store) with new Function, which a script-src without 'unsafe-eval' blocks, and the expressions then fail silently. Everything is delegated data-torque-* handlers in that one nonce'd script instead, and chrome actions that reach the component (the refresh interval) go through Livewire.find(id).call(...), plain JS with nothing to interpret. If your policy omits 'unsafe-eval', also set 'csp_safe' => true in config/livewire.php so Livewire loads its CSP-safe bundle for the wire: directives. The dashboard checks this on every response: a policy without 'unsafe-eval' while csp_safe is off shows a warning banner in the chrome and logs one line per hour, so the mismatch never fails silently.

Authorization

The gate viewTorque is checked on every dashboard route (overview, workers, queues, feed, inspector, dead-letter), so no screen or Livewire action is reachable by users who would fail the gate. Define it in your AuthServiceProvider:

If you don't define a gate, Torque falls back to app()->environment('local'); the dashboard shows up in development but stays locked in production until you define the gate explicitly.

Retries from the failed-jobs page only accept targets that exist in config('torque.streams'), so a compromised session cannot inject jobs into arbitrary Redis streams.

Dashboard middleware

Default: ['web', 'auth']. Override in config:

Failed jobs

Jobs that exhaust all retries are moved to a dead-letter Redis Stream. You can:

• View them in the dashboard
• Retry or delete via dashboard or programmatically
• Listen for the JobPermanentlyFailed event for custom notifications

Dead-letter retention

The stream is bounded in two ways, so a failure storm can never fill Redis:

max_entries is enforced by the write itself (XADD ... MAXLEN ~), which is what keeps the stream bounded even during a storm; trimming is approximate, so the real length settles slightly above the cap in exchange for an O(1) write. Size it against your payloads: at ~6 KB per entry (payload plus truncated trace) the default is roughly 600 MB worst case.

The master applies the TTL, the cap, and a sweep of consumer names left behind by exited workers on its first tick after start and every prune_interval seconds after that, so nothing has to be scheduled. Run the same pass by hand whenever you want:

Architecture

How it works

1. Master spawns N worker processes via pcntl_exec() (php artisan torque:worker)
2. Each worker runs a Revolt event loop with M Fiber slots
3. Each Fiber polls for messages with non-blocking XREADGROUP (no BLOCK). When no work is available, the Fiber yields to the event loop with a configurable delay (block_for / 1000 seconds). This ensures timers (delayed job migration, metrics, pause checks) always fire reliably
4. Fiber startup is staggered across the poll interval so polling is evenly distributed
5. Work-stealing: idle Fibers claim stale messages from dead consumers via XAUTOCLAIM (per-queue retry_after as idle threshold)
6. On completion: XACK + XDEL. On failure: retry with exponential backoff or dead-letter
7. A shared pause flag (updated by a timer) replaces per-Fiber Redis checks, reducing overhead from 50 EXISTS calls per cycle to 1

Queue backend: Redis Streams

Redis Streams (not LISTs like Horizon) provide:

• Consumer groups: multiple workers, no duplicate processing
• Acknowledgment: XACK after success, unacked jobs auto-reclaimed via XAUTOCLAIM
• Non-blocking reads: XREADGROUP without BLOCK returns immediately, letting Fibers yield cleanly
• Pending Entries List: Redis tracks assigned-but-unacked jobs natively
• Cluster support: hash-tagged keys keep related data on the same slot

Compatibility

Production deployment

Generate a Supervisor config:

This creates storage/torque-supervisor.conf. Copy it to your Supervisor config directory:

Operations

A few settings on the Redis instance and the app side that keep a failure storm from turning into an outage:

• Give Torque's Redis a maxmemory and set maxmemory-policy noeviction. With eviction on, a runaway writer silently evicts other keys; with noeviction the writer gets OOM command not allowed instead, which is loud, contained, and recoverable. Torque's own keys are all bounded (dead-letter cap, metrics TTLs, breaker expiries).
• Prune failed_jobs too. The dead-letter cap bounds Redis, not your database: the framework's failed-jobs table grows one row per permanent failure. Schedule Schedule::command('queue:prune-failed --hours=168')->daily();.
• Optionally schedule torque:prune hourly as belt and braces. Since the master runs the same housekeeping every dead_letter.prune_interval seconds it is no longer required, but it costs nothing and covers a fleet that is down while the scheduler still runs.

Upgrading from an older Torque

Version-specific notes live in UPGRADE.md.

Leftovers from a previous version (per-job event streams that never got their terminal expiry, index members pointing at streams that are gone, legacy metric keys) are cleaned up automatically on the first torque:start after the deploy: the master runs the sweep once per version and records it in {prefix}version, logging a count per category.

To preview it before restarting the fleet, run php artisan torque:prune --deep --dry-run and then the same command without --dry-run. torque:status shows the recorded data version next to the installed one.

Zero-downtime reload

torque:stop followed by torque:start works for cold deploys, but it leaves a queue-processing gap (jobs queue up in Redis until the new master is back). torque:reload swaps the master in one step, with no manual chaining of pause + wait + stop. Pick the mode by who owns the respawn:

Under a process supervisor (supervisord, systemd) — use --drain. The supervisor owns respawning, so the reload only signals:

A spawned replacement would escape supervision permanently, which is why the default mode warns when the master looks supervised.

Unsupervised hosts — the default takeover handshake:

The reload spawns torque:start --takeover=<oldPid>, which boots its fleet in its own session, waits for its own workers' first metrics heartbeat (takeover_ready_timeout, default 30 s), and only then claims the PID file and signals the old master to drain. A replacement whose fleet never becomes healthy aborts the takeover with the old master untouched, so a broken deploy stays a failed reload instead of an outage; the replacement's output is surfaced by the reload command on failure.

During the swap the old master's drain pause is scoped to its own workers (the pause key carries the master PID and a TTL), so the new fleet keeps consuming throughout. In-flight jobs finish naturally on the old master; the Redis queue handles claim-once semantics across both fleets. Keep TORQUE_DRAIN_GRACE (default 10) below the stream retry_after so the brief overlap cannot double-claim jobs. One accepted risk on unsupervised hosts: a takeover master SIGKILLed after claiming the PID file leaves nothing to respawn it, which is inherent to running without a supervisor.

Two clocks, one ceiling

--timeout is how long torque:reload itself keeps watching; drain_grace_seconds is what the master and its workers hold themselves to. They are not the same number, and the worst case of a drain is twice the grace: the master waits for its fleet to report idle, SIGTERMs it, and then every worker gets the window again for the job it still holds.

Left unset, --timeout is derived from that worst case, so the default can never be shorter than the drain it waits for. Passing it explicitly is for deploy tools with a run timeout of their own:

A short explicit timeout means "stop watching", not "cut the drain short". Past it the command reports that the master is still draining and returns successfully, without a signal: pickup has been paused since the SIGUSR2, and the master exits on its own once the fleet is idle or the grace runs out. The SIGTERM escalation is reserved for a master still alive past its own ceiling, which is a wedged one. torque:stop derives its pre-SIGKILL window the same way and takes the same --timeout override.

A drain pause belongs to the master that wrote it, so a master starting into a drain:<pid> pause whose PID is no longer running deletes the key and logs it, instead of honouring the rest of its TTL (drain_grace_seconds + 60, hours on installations with a long grace) after a reload was killed mid-drain. A deliberate torque:pause is never cleared automatically, and torque:status names which of the two a pause is.

Mutual exclusion between masters is a lifetime flock on storage/torque.lock (released by the kernel on any exit, including SIGKILL), and the PID file self-heals every second: the owning master rewrites it if missing, reclaims it from a stale PID, and self-demotes into a drain if another live master owns it. Concurrent reloads fail fast on storage/torque.reload.lock.

Containerized deployment

When storage/ is a bind mount or persistent volume, storage/torque.pid outlives the container. torque:start verifies the recorded PID actually belongs to a running Torque master — via /proc/<pid>/cmdline on Linux, ps elsewhere — so a stale PID file left by a previous container reads as "not running", even when its number has since been recycled by an unrelated process. Readers never delete the file (that would race a takeover's atomic rename); the next master simply overwrites it at boot. No manual rm storage/torque.pid is needed between restarts.

Performance

Fair comparison vs Laravel queue:work / Horizon

Same hardware, same Redis, same number of OS processes (2 each), 1000 jobs per run, median of 3 measured runs after a 100-job warmup. Each job emits one XADD result-event so measurement overhead is symmetric on both sides. Full reproduction recipe in BENCHMARKS.md.

[!TIP] The pattern is consistent: Torque wins when handlers yield to I/O, loses when handlers occupy the OS thread. The fanout row is the workload Torque was built for. Pure CPU is not.

Memory at equivalent throughput (the production framing):

For a queue dominated by external API calls and webhooks, that translates directly to fewer servers, less memory pressure, and headroom to absorb traffic spikes without provisioning ahead of time.

Benchmarking your own workload

Torque ships with a torque:bench command that produces reproducible numbers (jobs/sec, p50/p95/p99 latency) on your actual hardware. Run it before tuning anything: serializer choice, worker count, coroutines per worker. Optimization without numbers is guesswork.

Workload profiles:

Flags: --workers, --coroutines, --jobs, --warmup, --serializer, --json, --force. See php artisan torque:bench --help for the full list.

[!NOTE] The v1 bench command requires --use-running-master. Start a torque worker fleet first (php artisan torque:start), then run the bench against it. Self-spawning workers from inside the bench command lands in a follow-up release.

For deeper profiling, use XHProf or Excimer on a running worker. The bench output tells you whether to bother.

igbinary: ~2x faster payload encoding

Torque can encode its Redis Streams envelope with igbinary instead of JSON. Roughly 2x faster on encode and decode, smaller on the wire. Recommended once you have a baseline benchmark to compare against.

Install (PECL):

Or via your distro: apt install php8.5-igbinary on Debian/Ubuntu, brew install [email protected] style packages on macOS.

Enable in your .env:

Verify with the bench command:

torque:start prints Serializer: igbinary on boot when active, and a one-line install hint when the extension is missing.

[!TIP] Safe to flip while running. Torque sniffs the first byte of every payload ({/[ for JSON, \x00\x00\x00\x02 for igbinary), so in-flight messages decoded with the old format keep working. New messages come out as igbinary. Both coexist until the stream organically drains.

[!WARNING] Igbinary payloads are binary, not human-readable. redis-cli XRANGE torque:default - + returns gibberish for the payload field once you flip the switch. Stick with --serializer=json (the default) during debugging sessions.

[!TIP] Setting igbinary.compact_strings = On in php.ini also speeds up Laravel's session and cache serialize() calls globally, even without flipping the torque serializer. Free win across your whole app.

Dependencies

Required (installed automatically):

• revolt/event-loop: Fiber scheduler
• webpatser/fledge-fiber: async/await primitives, non-blocking Redis, sync primitives

Optional (install when needed):

• webpatser/fledge-fiber-database: Async MySQL for MysqlPool
• webpatser/fledge-fiber-http: Async HTTP for HttpPool
• ext-igbinary: ~2x faster payload encoding when TORQUE_SERIALIZER=igbinary is set. See Performance.

License

MIT