Download the PHP package jessecascio/spider without Composer
On this page you can find all versions of the php package jessecascio/spider. It is possible to download/install these versions without Composer. Possible dependencies are resolved automatically.
Download jessecascio/spider
More information about jessecascio/spider
Files in jessecascio/spider
Package spider
Short Description MySQL Parallel Query Library
License MIT
Homepage https://github.com/jessecascio/spider
FAQ
Example:
In general, it is recommended to use always a project to download your libraries. In an application normally there is more than one library needed.
- Some hosting areas are not accessible by a terminal or SSH. Then it is not possible to use Composer.
- To use Composer is sometimes complicated. Especially for beginners.
- Composer needs much resources. Sometimes they are not available on a simple webspace.
- If you are using private repositories you don't need to share your credentials. You can set up everything on our site and then you provide a simple download link to your team member.
- Simplify your Composer build process. Use our own command line tool to download the vendor folder as binary. This makes your build process faster and you don't need to expose your credentials for private repositories.
Informations about the package spider
Spider
Spider is a PHP library offering asynchronous MySQL querying. While there are other ways to achieve this, such as mysqlnd or pthreads, Spider was built to have no outside dependencies except for a connection to a MySQL database.
Usage
Install the latest version with composer require jessecascio/spider
Concepts
Since PHP does not natively support threading, or parallel execution, Spider takes the queries and processes them in their own process. After the query executes, it is placed into temporary storage until all queries have completed. Once all queries are done, the results are returned to the calling process.
Docs
Configurations
With the use of multiple processes it becomes very important to control both the number and the size of the processes. Also, by defining a log output, errors can be correctly logged during process failure.
Alternatively these can be loaded from an .ini file to allow for different settings in different environments.
Storage
By default Spider will store results in a temporary MySQL table. There are however other Storage objects. For example, if you
would rather have results stored in Memcached, simply instantiate the correct Storage object
Callbacks
Often times it is beneficial to process queries as they complete and not have to wait until all queries are done. This can be done with either a single callback applied to all queries, or specific callbacks mapped to query keys.
Considerations
Special considerations will need to be made for the number of processes to run and that allowed size. Since most web servers are
limited on resources you need to ensure that Spider does not spin up too many processes and that those processes do not get too
large. With can be done with the Component\Config object.
There is additional overhead for running the queries in this manner. Not only the cost of spinning up and monitoring the processes, but also of all the additional reads and writes into the temporary storages. Typically Spider would not be beneficial unless your total query time is over 1s. Some good use cases for Spider are querying a large horizontal star table where each component can be queried individually, to return to PHP for aggregation building, or for querying across multiple shards.
Testing should be done to determine optimal process count/size and typical return times on batch queries.
