Download the PHP package symbiote/silverstripe-sqs-jobqueue without Composer

SilverStripe SQS job queue

A module for sending and consuming SQS tasks. Can be configured to work as the trigger for queuejobs.

When used as the queuedjobs handler, there's a few slight changes to how queuedjobs are run - aside from not needing Cron jobs anymore.

• On calling QueuedJobService->queueJob, a message is sent to SQS
• A consumer picks up that message, then processes that job ID
• All "type 1" aka immediate jobs are subsequently processed in that execution thread
• If the added job is meant to be run in the future, the handler sets the job type to 'Scheduled'.
• A separate SQS task runs that looks for any jobs of type "Scheduled" and executes those
• That task re-queues itself for to run again in 30 seconds time
• The "Scheduled" task runner will also look for any job currently sitting in the "wait" status; this is how paused jobs get picked up for further execution without needing to trigger another SQS message

Configuration for use as the queuejobs handler

That expects a queue to exist with the name 'jobqueue' - if the queue name is different,

Writing a task

Define a class with a method in it. This is the task runner; no need for any specific implementations.

eg

Add configuration to your project that binds the method name to the SqsService

Triggering tasks

To trigger a task, the call

$sqsService->sendSqsMessage(['args' => ['param1', 'param2']], 'taskName');

for convenience, SqsService implements a __call() method that remaps a call like

$sqsService->taskName($arg1, $arg2)

into

$sqsService->sendSqsMessage(['args' => [$arg1, $arg2']], 'taskName');

sendSqsMessage in turn converts this into a message structure such as

So, to manually trigger the messages, create the $sqsMessage structure from your own code, and send using $this->client->sendMessage($sqsMessage);

Running

Development environments

If you don't have SQS available, you can run a file-based queue system by swapping out the AWS queue for the file based SQS queue.

By default, this will create serialised data in sqs-jobqueue/code/service/.queueus (configurable on the FileBasedSqsQueue class).

Run sqs-worker as before.

Docker

If you're using https://github.com/symbiote/docker-runtime, this will spin up an sqsrunner container for you that runs the sqs-worker automatically.

Troubleshooting

So for a project, your config may look something like this where it defaults to the file-based queue system only for test environments:

In the production environment you will still need to provide the credentials using local config.

To get the file-based queue system working, your local config will need something like this:

The important part is making sure /var/www/html/mysite/fake-sqs-queues is writable since that's where your queued jobs will be written to.

Docker Testing

The simplest way to test this once all your configs are in place is to:

• Queue up a job (e.g. using the https://github.com/symbiote/silverstripe-queuedjobs/ module)
• Ensure a file is written to /var/www/html/mysite/fake-sqs-queues
• Run docker logs sqsrunner to see whether it picked up the job
• View the queued jobs admin to ensure the job was completed