Download the PHP package thingston/crawler without Composer
On this page you can find all versions of the php package thingston/crawler. It is possible to download/install these versions without Composer. Possible dependencies are resolved automatically.
Download thingston/crawler
More information about thingston/crawler
Files in thingston/crawler
Package crawler
Short Description Web crawler based on PHP Guzzle HTTP Client with concurrency support for faster operation.
License MIT
Homepage https://thingston.com
Informations about the package crawler
Thingston Crawler
Web crawler based on PHP Guzzle HTTP Client with concurrency support for faster operation. Includes support for any content-type download, link profiler and response observers.
Requirements
Thingston Crawler requires:
- PHP 7.1 or above.
Instalation
Add Thingston Crawler to any PHP project using Composer:
Getting Started
Simply create a new Crawler
instance and invoke start
method with any public URI:
In order to process results from the crawling process you may add as many many Observers.
An Observer is a concrete class implement Thingston/Crawler/Observer/ObserverInterface
.
Reporting Issues
In case you find issues with this code please open a ticket in Github Issues at https://github.com/thingston/crawler/issues.
Contributors
Open Source is made of contribuition. If you want to contribute to Thingston please follow these steps:
- Fork latest version into your own repository.
- Write your changes or additions and commit them.
- Follow PSR-2 coding style standard.
- Make sure you have unit tests with full coverage to your changes.
- Go to Github Pull Requests at https://github.com/thingston/crawler/pulls and create a new request.
Thank you!
Changes and Versioning
All relevant changes on this code are logged in a separated log file.
Version numbers follow recommendations from Semantic Versioning.
License
Thingston code is maintained under The MIT License.
All versions of crawler with dependencies
psr/http-message Version ^1.0
psr/log Version ^1.0
guzzlehttp/guzzle Version ^6.3
t1gor/robots-txt-parser Version ^0.2.4
monolog/monolog Version ^1.23
symfony/dom-crawler Version ^4.1
symfony/css-selector Version ^4.1
jwage/purl Version ^0.0.10
jakubkulhan/chrome-devtools-protocol Version ^1.0
zendframework/zend-feed Version ^2.10
doctrine/dbal Version ^2.8
league/flysystem Version ^1.0