Download the PHP package laikait/laika-queue without Composer

Laika Queue

Background job processing for the Laika PHP MVC Framework — database, Redis, and JSON drivers behind one interface, with delayed jobs, retry backoff, failed-job tracking, and a signal-aware worker process.

This README is the full reference. If you just want the short version, the framework handbook has a Queue chapter.

Contents

• Install
• Quick start — the five-step loop
• Configuration
• Defining jobs
• Dispatching jobs
• CLI reference
• Drivers
• The worker
• Failed jobs
• Schema & migrations
• Production
• Concurrency caveat
• Security: trusted job classes
• Known gaps
• Standalone use
• Requirements

Install

Inside a Laika app you normally get it already — laikait/laika-core depends on it.

Installing generates a single worker executable in your project root — the same file on every platform — so the entry point always matches the version in vendor/:

It is a thin proxy into vendor/laikait/laika-queue/bin/worker. It's rewritten only when its contents actually change, and regenerates if you delete it.

Windows: run it as php worker ..., not a bare worker .... There is deliberately no worker.bat shim — cmd and PowerShell resolve commands through PATHEXT and will never execute an extensionless file. A worker.bat left over from an earlier version is deleted on the next composer install.

Generation is driven by Laika\Queue\ScriptHandler::generate. Composer only runs scripts declared by the root project, never by a dependency, so a project not created from the framework skeleton needs to wire it itself:

The framework skeleton already has this. ScriptHandler is a safe no-op anywhere lf-boot/app.php isn't present, so a global install or the package's own CI won't trip over it.

This package used to ship as a Composer plugin, which forced an allow-plugins entry into every consuming project. It's a plain library now — you can drop "laikait/laika-queue": true from your config.allow-plugins.

Global install

Want one worker command across every project?

Put Composer's global vendor/bin on your PATH (see the Composer docs), then run worker from inside any Laika project:

The global binary finds the project by walking up from your working directory until it hits lf-boot/app.php — no php prefix needed, and it works from a sub-directory.

Quick start

The whole loop, end to end.

1. Create the job

That writes lf-app/Job/SendWelcomeEmail.php.

2. Fill in handle()

3. Pick a backend in lf-config/queue.php — 'json' needs no setup and is the default:

Choosing 'database' (or leaving failed_driver on 'database') needs its tables created once — see Schema & migrations.

4. Dispatch it from a controller, service, anywhere:

5. Run the worker

That's it. The job runs, retries on failure with backoff, and lands in the failed-job log once it exhausts maxTries — inspect it with php laika queue:failed.

Configuration

Everything lives in lf-config/queue.php:

Why failed_driver is separate: there is no Redis-backed failed-job provider. A redis queue driver still has to log its failures somewhere, so it falls back to json unless you say 'database'.

Beyond reserve_timeout / max_tries above, the redis and json drivers have no connection config of their own:

• redis connects with lf-config/redis.php as-is, via Laika\Core\Storage\Connection\RedisConnection — the same factory RedisStorage uses, so it picks up host, port, password, username (Redis 6 ACL), database, timeout and read_timeout. Keys are namespaced under that file's prefix + :queue, so queue keys can't collide with cache or session keys sharing the prefix.
• json always uses lf-storage/queues/jobs.json (and lf-storage/queues/failed.json). Every queue shares one file, filtered by a queue field per record.

This same resolution logic lives in two places, kept deliberately in sync: bin/worker (for the worker process) and Laika\Core\Worker\Queue (for the queue:* commands and your own dispatch code).

Defining jobs

Extend Laika\Queue\Abstracts\Job and implement handle(). Use php laika job:make to scaffold one.

Properties

Backoff

backoff() returns the seconds to wait before the next attempt:

With $jitter = true (the default) the result is spread by ±20% so a batch of jobs failing together doesn't retry in lockstep. Set $jitter = false for deterministic timing.

Two traps worth knowing

failed() fires on every failed attempt, not just the last one. Worker::process() calls $job->failed($e) before it decides whether to retry or give up. If you only want the final-failure behaviour, check the counter yourself:

$queue and $delay on the job object are not what routes it. Every driver's push() signature is push(Job $job, string $queue = 'default', int $delay = 0), and it assigns $job->queue = $queue from that argument. So a job declaring public string $queue = 'emails'; — which is exactly what job:make --queue=emails generates — still gets pushed to default unless you pass the queue explicitly:

$job->delay is never read by any driver; delay is the third push() argument only.

Dispatching jobs

Inside the framework (recommended)

Laika\Core\Worker\Queue builds whichever driver lf-config/queue.php names, so your dispatch code doesn't hardcode a backend and doesn't drift from the worker's choice:

Queue ships in laikait/laika-core, so it's always available in a Laika app — web requests included, not just CLI.

Laika\Cli\QueueResolver is the former home of this API and still works, delegating to Queue with the CLI's historical database default. Prefer Queue in new code.

Constructing a driver directly

The escape hatch when you need a specific backend regardless of config:

DatabaseDriver never opens PDO itself — it refers to a connection by name from laika-model's registry. Register the connection first, and make sure the tables exist (Schema & migrations) — the driver no longer creates them.

The driver interface

All three drivers implement Laika\Queue\Interfaces\QueueDriverInterface:

size() counts every unreserved job including ones delayed into the future — consistent across all three drivers.

CLI reference

Eight commands ship with laikait/laika-cli for this package.

Jobs

job:make validates that the name and queue are alphabetic (/^[a-z_]+$/i) and that --max is a positive integer.

Queue

Notes:

• queue:work is a thin proxy — it shells out to the root worker executable (falling back to vendor/laikait/laika-queue/bin/worker) so it can never drift from the real worker's driver selection, signal handling, or memory logic. php worker default and php laika queue:work default do the same thing.
• queue:retry resets $job->tries = 0 before re-pushing, so a payload carrying an exhausted attempt count can't fail permanently on its very next run. Under RedisDriver this is now belt-and-braces — attempts live server-side in the :attempts hash and push() clears that field, so a re-push already starts from a clean count. It also registers trusted classes first, same as the worker.
• queue:flush asks for confirmation before deleting.

Drivers

RedisDriver

The throughput choice, and safe for concurrent workers: pop() is a Lua script, so claiming a job is atomic. It keeps six keys per queue — :pending (list), :delayed (sorted set), :reserved (sorted set), :jobs (hash of payloads), :attempts (hash of per-job delivery counts) and :dead (sorted set) — and every pop() first migrates due delayed jobs into pending, then sweeps reserved jobs older than reserve_timeout back into pending. That sweep is what makes it self-healing when a worker dies mid-job.

The sweep doesn't requeue forever. Each reclaim checks the job's :attempts count, and once it reaches max_tries the job is moved to :dead instead of back onto pending — so a job that reliably kills its worker stops cycling. Its payload stays in :jobs for inspection; nothing prunes :dead automatically.

Build it with RedisDriver::fromConfig() rather than handing it an open Redis instance if it'll run inside a forking worker — see ReconnectableDriver. Requires ext-redis.

DatabaseDriver

Queries through Laika\Queue\Model\QueueModel using laika-model's portable query builder — no raw SQL, no per-driver branching, so it works identically on every backend laika-model supports (mysql, mariadb, pgsql, sqlite, sqlsrv, oci, firebird). The tradeoff is the missing row lock; see Concurrency caveat.

JsonDriver

One JSON array of records in a single file. Every read-modify-write goes through Laika\Core\Storage\JsonStorage::mutate(), which holds a single flock(LOCK_EX) across the whole read → mutate → write, so two workers can't claim the same job. Fine for development, not intended for production. Note it depends on APP_PATH and on JsonStorage itself, so unlike the other two it cannot be used outside a Laika app.

The worker

The queue name is optional and defaults to default. One worker process handles exactly one queue name — run it multiple times for more than one queue. There's no validation against your config: a typo'd name doesn't error, it just runs forever popping nothing.

Worker::work() takes four arguments:

Signals

Requires ext-pcntl. Without it the worker still runs, it just can't be signalled.

Per-job timeout

With pcntl available, each job runs in a forked child. If the child exceeds timeout, the parent SIGKILLs it and treats it as a failed attempt — released with backoff, or logged to the failed-job provider once maxTries is exhausted. So a job that hangs repeatedly is subject to the same retry ceiling as one that throws.

Without pcntl/posix (i.e. Windows) the worker degrades to running jobs inline in the parent process: still correct, but no timeout enforcement and no signal handling.

Reconnecting after fork

pcntl_fork() duplicates the parent's open sockets into the child without reopening them, so both processes share one connection — and killing the child mid-write can desync the parent's socket too. Drivers that hold a stateful connection implement Laika\Queue\Interfaces\ReconnectableDriver, and the worker calls reconnect() in the freshly forked child before running the job.

RedisDriver implements it (which is why fromConfig() matters — a driver built from a bare Redis instance has no way to reopen). DatabaseDriver and DatabaseFailedJobProvider don't: laika-model's Connection registry owns the PDO lifecycle, not the driver.

Memory

The worker exits gracefully when memory crosses a soft threshold, letting supervisor or systemd restart it with a clean heap — this is normal operation, not a crash.

bin/worker first applies the framework's CLI_MEMORY_LIMIT (from lf-inc/const.php) as the process's real memory_limit via Laika\Core\System\MemoryManager. Then Worker::work() with memoryLimit: null reads PHP's current memory_limit and takes ~90% of it as the restart threshold, so the worker bows out before genuinely risking a hard OOM mid-job. Pass an explicit int (MB) to override.

Falls back to a flat 128MB when laikait/laika-core isn't installed, when memory_limit can't be parsed, or when it's unlimited (-1).

Failed jobs

A job that exhausts maxTries — by throwing or by timing out — is handed to a FailedJobProviderInterface and removed from the queue, rather than silently dropped or retried forever.

Two implementations: DatabaseFailedJobProvider and JsonFailedJobProvider. Chosen by failed_driver; there's deliberately no Redis one.

Typical workflow:

Retrying is subject to the same trusted-class allow-list as the worker, since it has to unserialize the stored payload.

Schema & migrations

Two tables, created only when you use a database driver.

laika_queue_jobs — id (char 36, PK), queue (string 100), payload (longtext), attempts (uint, default 0), reserved_at (uint, nullable), available_at (uint), created_at (uint), plus a composite index on (queue, reserved_at, available_at).

laika_failed_jobs — id (char 36, PK), queue (string 100), payload (longtext), exception (text), failed_at (uint), indexed on queue.

The tables are not created for you. Earlier versions had the driver and the failed-job provider create their own tables on construction via ensureSchema(). That method is gone — run the migration once before the first worker start, or push() will fail against a missing table.

Create them once, on the connection the driver uses:

With no argument, both follow queue.connection from lf-config/queue.php; outside a Laika app they fall back to 'default', or pass a connection name. Both use createIfNotExists(), so running them again is safe.

php laika app:migrate does not create these tables. Since v1.1.1 the package no longer declares its models and schemas under extra.laika.resources, so the framework's resource loader does not see them.

DatabaseDriver::install() and DatabaseFailedJobProvider::install() also create the tables, but drop them first, deleting every pending or failed job. Use them for a fresh install or a test, never on a live queue.

Both schema classes extend Laika\Model\Contract\SchemaAbstract and use Laika\Model\Schema\Schema / Blueprint, so schema creation needs laikait/laika-model, not laika-core. They're plain PSR-4 files, lazily autoloaded only when something references them — and since nothing in the drivers touches them any more, that means the explicit calls above, nothing else.

The models themselves (QueueModel, FailedJobModel) are ordinary Laika\Model\Model subclasses with the usual $table / $id / $connection convention — query them directly if you want your own dashboard.

Production

The worker is a long-running process, not a cron job, and it self-exits on the memory threshold. Keep it alive with a supervisor.

supervisor

systemd

Set stopwaitsecs / TimeoutStopSec comfortably above your longest job timeout so a graceful stop can finish the job in flight instead of being killed.

For several queues, add one program block per queue name rather than raising numprocs — and read the caveat below before raising numprocs at all.

Which user?

www-data above is not created by Laika, and it isn't universal — it's the account the Debian/Ubuntu packages give Apache, nginx and PHP-FPM. Elsewhere it's called something else:

Find yours rather than guessing:

or from inside the app: posix_getpwuid(posix_geteuid())['name'].

Run the worker as whatever user owns the project files. This is not cosmetic. The worker writes lf-storage/ — the JSON driver's jobs.json and failed.json especially, which the web process reads and writes too. Get the user wrong and you end up with files the web server can't touch, or vice versa: jobs silently failing to enqueue, or a Permission denied the first time a request tries to push.

Shared hosting

On most shared hosting you have no root, so neither supervisor nor systemd is available and the User= line is moot — everything already runs as your account. Check what your panel offers first; many expose a "background worker", "daemon", or long-running process feature that does the same job.

Failing that, a cron watchdog restarts the worker whenever it isn't running. The worker exits on its own memory threshold, so something has to bring it back:

Note this is a watchdog, not a per-minute run — bin/worker takes only a queue name and loops until it's signalled or hits the memory ceiling; there's no --once or max-jobs flag to make it exit after a batch. Match the pgrep pattern to the queue name so one watchdog line per queue doesn't restart the wrong worker.

If your host kills long-running processes outright — some do — the queue isn't a good fit there; dispatch to an external worker instead.

Concurrency caveat

DatabaseDriver::pop() has no row lock.

DatabaseDriver claims a job with a plain SELECT (filtered to unreserved, due rows) followed by an UPDATE, both inside a transaction, entirely through laika-model's portable query builder. There's no FOR UPDATE / SKIP LOCKED or equivalent, because those aren't portable across the seven backends laika-model targets.

The consequence: two worker processes calling pop() on the same queue at close to the same moment can both read the same unreserved row before either commits its UPDATE, and both will process that job.

This matters only if you run more than one worker on the same queue. Your options:

• Run a single worker process per queue. Always safe. This is the default and the right answer for most apps.
• Make your jobs idempotent, so double-processing is harmless.
• Use RedisDriver, whose Lua-scripted pop() is atomic.
• Add a locking clause to pop() yourself for your specific database.

RedisDriver and JsonDriver are unaffected — the former claims jobs in a Lua script, the latter under a single flock(LOCK_EX) spanning its whole read-modify-write. The flock guarantee holds on a local filesystem; it is unreliable on NFS and some other network filesystems, so don't put jobs.json on a share and expect it to arbitrate between hosts.

Security: trusted job classes

Job payloads are restored with PHP's unserialize(). To prevent PHP Object Injection — an attacker-controlled payload instantiating arbitrary classes to build a gadget chain — Job::unserializePayload() only instantiates classes explicitly registered as trusted. The default list is empty, so it throws rather than silently unserializing something unexpected.

In a Laika app this is handled for you. bin/worker and queue:retry both register every Job subclass discovered under lf-app/Job on startup, via Laika\Service\Infra::getQueueJobsClasses() — the same lookup php laika job:list uses. Discovery only admits classes that genuinely extend Job, so this stays much narrower than trusting the codebase at large. No config needed.

Register manually for job classes living outside lf-app/Job, or when using this package standalone:

Do it once at bootstrap, before any pop(). Calls are additive and de-duplicated.

Known gaps

• No stalled-job reaper for DatabaseDriver/JsonDriver. A worker killed mid-job leaves reserved_at set forever, and that job is never picked up again. RedisDriver::pop() self-heals via its reserved sweep (reserve_timeout, default 90s); a cron-callable reapStalled() for the other two is the obvious next addition.
• DatabaseDriver::pop() has no locking clause — see Concurrency caveat.
• No QueueRelay / QueueServiceProvider. There's no facade in Laika\Service yet; use Laika\Core\Worker\Queue as shown in Dispatching jobs.
• Job::$delay and Job::$queue are not honoured by push() — pass both as arguments instead. See Two traps.
• No batching, chaining, or unique-job support.

Standalone use (outside the framework)

The package works without the Laika framework, with limits:

A minimal standalone loop:

Note laikait/laika-model is not declared in this package's composer.json, since RedisDriver doesn't need it — install it yourself if you're using the database backend.

Requirements

• PHP 8.1+
• ext-pdo
• laikait/laika-model — for DatabaseDriver, DatabaseFailedJobProvider, and the schema classes. Not a declared dependency; comes in via laikait/laika-core in a framework app.
• ext-redis — for RedisDriver
• ext-pcntl + ext-posix — for signal handling and per-job timeouts (Linux/macOS). The worker degrades to inline execution without them.
• laikait/laika-core — only for CLI_MEMORY_LIMIT handling via MemoryManager; optional everywhere else.

License

MIT