Download the PHP package 10up/wp-gears without Composer

WP Minions

Provides a framework for using job queues with WordPress for asynchronous task running. Provides an integration with Gearman and RabbitMQ out of the box.

[!CAUTION] As of 12 April 2024, this project is archived and no longer being actively maintained.

Background & Purpose

As WordPress becomes a more popular publishing platform for increasingly large publishers, with complex workflows, the need for increasingly complex and resource-intensive tasks has only increased. Things like generating reports, expensive API calls, syncing users to mail providers, or even ingesting content from feeds all take a lot of time or a lot of memory (or both), and commonly can't finish within common limitations of web servers, because things like timeouts and memory limits get in the way.

WP Minions provides a few helper functions that allow you to add tasks to a queue, and specify an action that should be called to trigger the task, just hook a callback into the action using

During configuration, a number of minions (workers) are specified. As minions are free, they will take the next task from the queue, call the action, and any callbacks hooked into the action will be run.

In the situation of needing more ram or higher timeouts, a separate server to process the tasks is ideal - Just set up WordPress on that server like the standard web servers, and up the resources. Make sure not to send any production traffic to the server, and it will exclusively handle tasks from the queue.

Installation

1. Install the plugin in WordPress. If desired, you can download a zip and install via the WordPress plugin installer.

2. Create a symlink at the site root (the same directory as ) that points to the file in the plugin (or copy the file, but a symlink will ensure it is updated if the plugin is updated)

3. Define a unique salt in wp-config.php so that multiple installs don't conflict.

Note: If you are using multisite, you'll also have to add the following to your file, after the block with the multisite definitions. This is due to the fact that multisite relies on to detect the site/blog it is running under. You'll also want to make sure you are actually defining in the multisite configuration.

4. Next, you'll need to choose your job queue system. Gearman and RabbitMQ are supported out of the box.

Gearman

There are a few parts to get this all running. First, the Gearman backend needs to be setup - this part will vary depending on your OS. Once that is complete, we can install the WordPress plugin, and set the configuration options for WordPress.

Backend Setup - CentOS 6.x

1. You'll need the EPEL repo for gearman, and the REMI repo for some of the php packages. Make sure to enable the appropriate remi repo for the version of php you are using.
2. Make sure that remi is enabled, as well as any specific version of php you may want in
6. If everything is running on one server, I would recommend limiting connections to localhost only. If not, you'll want to set up firewall rules to only allow certain clients to connect on the Gearman port (Default 4730)
   • edit - set

Backend Setup - CentOS 7.x

1. You'll need the EPEL repo for gearman, and the REMI repo for some of the php packages.
2. . For example, if you are using php 7.2 your command would look like this
3. Optionally, install supervisord if you prefer it
4. If everything is running on one server, I would recommend limiting connections to localhost only. If not, you'll want to set up firewall rules to only allow certain clients to connect on the Gearman port (Default 4730)
   • edit - set

Backend Setup - Ubuntu

As you go through this, you may need to install additional packages, if you do not have them already, such as php-pear or a php*-dev package

4. Once pecl install is complete, it will tell you to place something like into your php.ini file - Do this.
6. If everything is running on one server, I would recommend limiting connections to localhost only. If not, you'll want to set up firewall rules to only allow certain clients to connect on the Gearman port (Default 4730)
   • edit - set

Supervisor Configuration

Filling in values in as required, add the following config to either (CentOS) or (Ubuntu)

• path_to_wordpress: Absolute path to the root of your WordPress install, ex:
• number_of_minions: How many minions should be spawned (How many jobs can be running at once).
• path_to_temp_directory: probably should just be the same as path_to_wordpress.
• user: The system user to run the processes under, probably apache (CentOS), nginx (CentOS), or www-data (Ubuntu).
• You can optionally change the "my_wp_minions_workers" text to something more descriptive, if you'd like.

After updating the supervisor configuration, restart the service (CentOS or Ubuntu)

Systemd Configuration (CentOS 7.x)

Filling in values in as required, add the following to /etc/systemd/system/[email protected]

• path_to_wordpress: Absolute path to the root of your WordPress install, ex:
• user: The system user to run the processes under, probably apache (CentOS), nginx (CentOS), or www-data (Ubuntu).

Reload systemd:

Enable and start as many runners as you'd like to have running:

Where 'n' is the number of processes you want.

WordPress Configuration

Define the WP_MINIONS_BACKEND constant in your . Valid values are gearman or rabbitmq. If left blank, it will default to a cron client.

If your job queue service not running locally or uses a non-standard port, you'll need define your servers in

/etc/sysconfig/gearmandsh OPTIONS="-q MySQL --mysql-host=localhost --mysql-port=3306 --mysql-user= --mysql-password= --mysql-db=gearman --mysql-table=gearman_queue" /etc/default/gearman-job-serversh PARAMS="-q MySQL --mysql-host=localhost --mysql-port=3306 --mysql-user= --mysql-password= --mysql-db=gearman --mysql-table=gearman_queue" /etc/default/gearman-job-serversh -q MySQL --mysql-host=localhost --mysql-port=3306 --mysql-user= --mysql-password= --mysql-db=gearman --mysql-table=gearman_queue sudo service gearman-job-server restarttest-client.phptest-worker.phpctrl-cwp_async_task_add()wp_schedule_single_event()wp_async_task_add()$hookadd_action( $hook, $callback )$args$priority$args```

Customization

The following constants can be used to customize the behaviour of WP Minions.

1. WP_MINIONS_JOBS_PER_WORKER - The number of jobs to execute per Worker, default is 1. Running multiple jobs per worker will reduce the number workers spawned, and can significantly boost performance. However too large a value will cause issues if you have memory leaks. Use with caution.

2. WP_MINIONS_CLIENT_CLASS - You can also alter the Client class used to send jobs to your job queue. It should match the interface of \WpMinions\Client.

3. WP_MINIONS_WORKER_CLASS - Similarly you can alter the Worker class used to execute jobs. It should match the interface of \WpMinions\Worker.

Issues

If you identify any errors or have an idea for improving the plugin, please open an issue. We're excited to see what the community thinks of this project, and we would love your input!

Support Level

Archived: This project is no longer maintained by 10up. We are no longer responding to Issues or Pull Requests unless they relate to security concerns. We encourage interested developers to fork this project and make it their own!

Like what you see?