Download the PHP package h4cc/guzzle-rolling-batch without Composer
On this page you can find all versions of the php package h4cc/guzzle-rolling-batch. It is possible to download/install these versions without Composer. Possible dependencies are resolved automatically.
FAQ
Example:
In general, it is recommended to use always a project to download your libraries. In an application normally there is more than one library needed.
- Some hosting areas are not accessible by a terminal or SSH. Then it is not possible to use Composer.
- To use Composer is sometimes complicated. Especially for beginners.
- Composer needs much resources. Sometimes they are not available on a simple webspace.
- If you are using private repositories you don't need to share your credentials. You can set up everything on our site and then you provide a simple download link to your team member.
- Simplify your Composer build process. Use our own command line tool to download the vendor folder as binary. This makes your build process faster and you don't need to expose your credentials for private repositories.
Informations about the package guzzle-rolling-batch
GuzzleRollingBatch
This is a parallel Executor for guzzle that can process multiple requests in parallel. Instead of processing a batch of requests, there are two queues for requests and responses for dynamic handling of execution, adding new requests and processing responses.
Introduction
When building a crawler with a batch system, there are multiple problems. Using batches wastes time when waiting for the result of the last request. There is no fully parallel processing of responses, adding and preparing new requests. When starting a batch, the result is more like a tiny DOS for the target site. To avoid all of these problems, RollingBatch was created.
Installation
Simply require the package by its name with composer:
Follow the 'dev-master' branch for latest dev version. But i recommend to use more stable version tags if available.
Usage
The RollingBatch consists of three parts. The RequestQueue, ResponseQueue and the RollingBatch. Lets have a look at each part.
RequestQueue
New requests have to be added to the RequestQueue. From there, the RollingBatch fetches them with the 'next()' method.
In case you need a more flexible approach, implement your own RequestQueue.
ResponseQueue
Received responses will be added to the ResponseQueue. From there they can be fetched and processed if needed.
Also, implementing your own ResponseQueue is an option for further functionality.
RollingBatch
The RollingBatch has some methods for controlling the process. Run 'execute()' to start new requests and execute started ones. Getting the current status can be fetched with 'countActive()' and 'isIdle()'.
A example for processing one or more request at once
Be aware, that countActive() alone might not be enough as a predicate for loops etc.
It is also possible, to limit the current executed parallel requests. This can be done with 'setNumberParallel()'. Setting the value '0' will disable the limit. The number of parallel requests can also be reduced while running.
Events
It is also possible to use the guzzle request events.
For example listening for exceptions.
For more events have a look here: http://guzzlephp.org/http-client/request.html#events-emitted-from-a-request
