Download the PHP package webmozarts/console-parallelization without Composer

On this page you can find all versions of the php package webmozarts/console-parallelization. It is possible to download/install these versions without Composer. Possible dependencies are resolved automatically.

FAQ

After the download, you have to make one include require_once('vendor/autoload.php');. After that you have to import the classes with use statements.

Example:
If you use only one package a project is not needed. But if you use more then one package, without a project it is not possible to import the classes with use statements.

In general, it is recommended to use always a project to download your libraries. In an application normally there is more than one library needed.
Some PHP packages are not free to download and because of that hosted in private repositories. In this case some credentials are needed to access such packages. Please use the auth.json textarea to insert credentials, if a package is coming from a private repository. You can look here for more information.

  • Some hosting areas are not accessible by a terminal or SSH. Then it is not possible to use Composer.
  • To use Composer is sometimes complicated. Especially for beginners.
  • Composer needs much resources. Sometimes they are not available on a simple webspace.
  • If you are using private repositories you don't need to share your credentials. You can set up everything on our site and then you provide a simple download link to your team member.
  • Simplify your Composer build process. Use our own command line tool to download the vendor folder as binary. This makes your build process faster and you don't need to expose your credentials for private repositories.
Please rate this library. Is it a good library?

Informations about the package console-parallelization

Parallelization for the Symfony Console

This library supports the parallelization of Symfony Console commands.

How it works

When you launch a command with multiprocessing enabled, a main process fetches items and distributes them across the given number of child processes over the standard input. Child processes are killed after a fixed number of items (a segment) in order to prevent them from slowing down over time.

Optionally, the work of child processes can be split down into further chunks (batches). You can perform certain work before and after each of these batches (for example flushing changes to the database) in order to optimize the performance of your command.

Installation

Use Composer to install the package:

Usage

Add parallelization capabilities to your project, you can either extend the ParallelCommand class or use the Parallelization trait:

You can run this command like a regular Symfony Console command:

Or, if you want, you can run the command using parallelization:

The API

The ParallelCommand and the Parallelization trait

This library offers a ParallelCommand base class and a Parallelization trait. If you are looking for a basic usage, the ParallelCommand should be simpler to use as it provides the strictly required methods as abstract methods. All other hooks can be configured by overriding the ::configureParallelExecutableFactory() method.

The Parallelization trait on the other hand implements all hooks by default, requiring a bit less manual task. It does require to call ParallelizationInput::configureCommand() to add the parallelization related input arguments and options.

Items

The main process fetches all the items that need to be processed and passes them to the child processes through their Standard Input (STDIN). Hence, items must fulfill two requirements:

Typically, you want to keep items small in order to offload processing from the main process to the child process. Some typical examples for items:

Segments

When you run a command with multiprocessing enabled, the items returned by fetchItems() are split into segments of a fixed size. Each child processes process a single segment and kills itself after that.

By default, the segment size is the same as the batch size (see below), but you can try to tweak the performance of your command by choosing a different segment size (ideally a multiple of the batch size). You can do so by overriding the getSegmentSize() method:

Batches

By default, the batch size and the segment size are the same. If desired, you can however choose a smaller batch size than the segment size and run custom code before or after each batch. You will typically do so in order to flush changes to the database or free resources that you don't need anymore.

To run code before/after each batch, override the hooks runBeforeBatch() and runAfterBatch():

You can customize the default batch size of 50 by overriding the getBatchSize() method:

Configuration

The library offers a wide variety of configuration settings:

Hooks

The library supports several process hooks which can be configured via ::configureParallelExecutableFactory():

Method* Scope Description
runBeforeFirstCommand($input, $output) Main process Run before any child process is spawned
runAfterLastCommand($input, $output) Main process Run after all child processes have completed
runBeforeBatch($input, $output, $items) Child process Run before each batch in the child process (or main if no child process is spawned)
runAfterBatch($input, $output, $items) Child process Run after each batch in the child process (or main if no child process is spawned)

*: When using the Parallelization trait, those hooks can be directly configured by overriding the corresponding method.

Subscribed Services

You should be using subscribed services or proxies. Indeed, you may otherwise end up with the issue that the service initially injected in the command may end up being different from the one used by the container. This is because upon error, the ResetServiceErrorHandler error handler is used which resets the container when an item fails. As a result, if the service is not directly fetched from the container (to get a fresh instance if the container resets), you will end up using an obsolete service.

A common symptom of this issue is to run into a closed entity manager issue.

Differences with Amphp/ReactPHP

If you came across this library and wonder what the differences are with Amphp or ReactPHP or other potential parallelization libraries, this section is to highlight a few differences.

The primary difference is the parallelization mechanism itself. Amphp or ReactPHP work by spawning a pool of workers and distributing the work to those. This library however, spawns a pool of processes. To be more specific, the differences lies in how the spawn processed are used:

The other difference is that this library works with a command as its central point. This offers the following advantages:

Contribute

Contributions to the package are always welcome!

To run the CS fixer and tests you can use the command make. More details available with make help.

Upgrade

See the upgrade guide.

Authors

License

All contents of this package are licensed under the MIT license.


All versions of console-parallelization with dependencies

PHP Build Version
Package Version
Requires php Version ^8.1
fidry/cpu-core-counter Version ^0.5.0 || ^1.0
nikic/iter Version ^2.2
psr/log Version ^1.1 || ^2.0 || ^3.0
symfony/console Version ^6.4
symfony/dependency-injection Version ^6.4
symfony/deprecation-contracts Version ^2.5 || ^3.1
symfony/process Version ^6.4
symfony/service-contracts Version ^3.3
thecodingmachine/safe Version ^1.3.3 || ^2.4
webmozart/assert Version ^1.5
Composer command for our command line client (download client) This client runs in each environment. You don't need a specific PHP version etc. The first 20 API calls are free. Standard composer command

The package webmozarts/console-parallelization contains the following files

Loading the files please wait ....