Download the PHP package steamulo/elastica-fast-populate-bundle without Composer
On this page you can find all versions of the php package steamulo/elastica-fast-populate-bundle. It is possible to download/install these versions without Composer. Possible dependencies are resolved automatically.
Download steamulo/elastica-fast-populate-bundle
More information about steamulo/elastica-fast-populate-bundle
Files in steamulo/elastica-fast-populate-bundle
Package elastica-fast-populate-bundle
Short Description Improves performance of fos:elastica:populate command from FOSElasticaBundle by distributing the work among consumers.
License MIT
Informations about the package elastica-fast-populate-bundle
Elastica Fast Populate Bundle
Improves performance of fos:elastica:populate
command from FOSElasticaBundle by distributing the work among consumers.
This bundle is based on Enqueue Elastica Bundle, and provides a command to directly manage the "populate" with a set of sub-processes automatically. The performance gain depends on how much consumers you run. For example 10 consumers may give you 5 to 7 times better performance.
Installation
When installing the bundle, Enqueue ElasticaBundle
is also automatically installed.
It must then be configured in order to indicate the way in which the sub-processes will communicate.
Default enqueue.yaml
:
Note: As long as you are on Symfony Flex you are done. If not, you have to do some extra things, like registering the bundle in your AppKernel
class.
Usage
- Run the populate command with some consumers (the more you run the better performance you might get):
Customization
Options for the command
To limit the memory consumption of sub-processes you can use different parameters :
message-limit
- Integer. Consume n messages and exit.time-limit
- Integer. Consume messages during this time.memory-limit
- Integer. Consume messages until process reaches this memory limit in MB.
You can also use the classic populate options : PopulateCommand.php
For example, to limit the memory consumption to 800MB for 8 sub-processes and process 1000 elements per page, you can run the command like this:
Customizing the consumer
The QueuePagerPersister
could be customized via options.
The options could be customized in a listener subscribed on FOS\ElasticaBundle\Persister\Event\PrePersistEvent
event for example.
Here's the list of available options:
max_per_page
- Integer. Tells how many objects should be processed by a single worker at a time.first_page
- Integer. Tells from what page to start rebuilding the index.last_page
- Integer. Tells on what page to stop rebuilding the index.populate_queue
- String. It is a name of a populate queue. Workers should consume messages from it.populate_reply_queue
- String. It is a name of a reply queue. The command should consume replies from it. Persister tries to create a temporary queue if not set.reply_receive_timeout
- Float. A time a consumer waits for a message. In milliseconds.limit_overall_reply_time
- Int. Limits an overtime allowed processing time. Throws an exception if it is exceeded.
License
It is released under the MIT License.
All versions of elastica-fast-populate-bundle with dependencies
enqueue/elastica-bundle Version ^0.10.0 || ^0.11.0 || ^0.12.0 || ^0.13.0
symfony/process Version ^4.4 || ^5.4 || ^6.0