Download the PHP package soarce/parallel-process-dispatcher without Composer
On this page you can find all versions of the php package soarce/parallel-process-dispatcher. It is possible to download/install these versions without Composer. Possible dependencies are resolved automatically.
Download soarce/parallel-process-dispatcher
More information about soarce/parallel-process-dispatcher
Files in soarce/parallel-process-dispatcher
Package parallel-process-dispatcher
Short Description Tiny PHP library for running jobs in background and/or in parallel
License MIT
FAQ
Example:
In general, it is recommended to use always a project to download your libraries. In an application normally there is more than one library needed.
- Some hosting areas are not accessible by a terminal or SSH. Then it is not possible to use Composer.
- To use Composer is sometimes complicated. Especially for beginners.
- Composer needs much resources. Sometimes they are not available on a simple webspace.
- If you are using private repositories you don't need to share your credentials. You can set up everything on our site and then you provide a simple download link to your team member.
- Simplify your Composer build process. Use our own command line tool to download the vendor folder as binary. This makes your build process faster and you don't need to expose your credentials for private repositories.
Informations about the package parallel-process-dispatcher
soarce/parallel-process-dispatcher 
This micro-library has two classes. One encapsulates a (linux commandline) process into an object and allows asynchronous running without deadlocks. The other is a multi-process-dispatcher which takes an arbitrary number of beforementioned processes and runs them simultaneously (with a maximum number of concurrent processes).
Usage examples:
- Dispatching long running cronjobs which e.g. mostly wait for webservice responses (you can run more processes than max. CPUs)
- Running background workers which listen on a queue (maximum should be number of CPUs)
- Running commandline-tasks inside a web application simultaneously, e.g. PDF-Generation, Image-Processing etc.
Installation:
Add the following to your composer.json:
or just run the following command in your project root directory
Usage
Classes
Process
Dispatcher
Advanced
Using Process and Dispatcher to start multiple processes and later collect the results
Reading Output from multiple jobs while they are running
A possible use case for this would be running multiple crawlers over separate slow(er) filesystems or websites to generate a list of certain download or backup worthy files while the main process keeps track of the list, eliminates duplicates and writes the backup.
The function OutputAggregator::getOutput() returns a Generator which returns the job's name (here job1-3) as key and the output
line as value.
Contrary to arrays, the key will with almost certain probability appear multiple times.
Known Issues
Process
- PHP Internals: Be aware that if the child process produces output, it will write into a buffer until the buffer is
full. If the buffer is full the child pauses until the parent reads from the buffer and makes more room. This is done
in the isFinished() method. The dispatcher calls this method periodically to prevent a deadlock. If you use the process
class standalone, you have multiple possibilities to prevent this:
- call isFinished() yourself in either a loop, using a tick function or otherwise during execution of your script
- instead of writing to stdOut, divert output to a temporary file and use its name as output.
- use the combination of OutputAggregator and ProcessLineOutput and work on output as soon as it arrives. This might consume a lot of RAM for buffers if your jobs generate a lot of output, so set high enough limits.
Dispatcher
- Multiple dispatchers (in different processes) are not aware of each other. So if you have a script that uses a dispatcher to call another script which itself uses a dispatcher to spawn multiple processes, you will end up with more child processes than the maximum, so choose the maximum accordingly or use a queue (e.g. Redis) and make the workers aware of each other by e.g. registering in a redis-stack for running workers.
