Download the PHP package monkeyphp/php-spider without Composer
On this page you can find all versions of the php package monkeyphp/php-spider. It is possible to download/install these versions without Composer. Possible dependencies are resolved automatically.
Download monkeyphp/php-spider
More information about monkeyphp/php-spider
Files in monkeyphp/php-spider
Package php-spider
Short Description A basic web spider written in PHP.
License BSD-3-Clause
Homepage https://github.com/monkeyphp/php-spider
Informations about the package php-spider
PhpSpider
I needed a really basic spider to crawl a site to warm the cache; this is the result.
Examples
Simple
The simplest way to use PhpSpider is to construct an instance and pass the root
url of the site that you wish to crawl to the Spider::crawl
method.
By design, PhpSpider will only crawl pages that
- Return a
content-type
header oftext\html
- That are on the same domain as the supplied root
Advanced
If you need to override how PhpSpider works you can add event listeners to be notified, that can then affect how PhpSpider operates.
There are 5 events triggered by PhpSpider
- Spider::SPIDER_CRAWL_PRE - Triggered before PhpSpider starts to crawl a site.
- Spider::SPIDER_CRAWL_POST - Triggered one PhpSpider has finished it's crawl.
- Spider::SPIDER_CRAWL_PAGE_PRE - Triggered just before a page is crawled
- Spider::SPIDER_CRAWL_PAGE_POST - Triggered once a page is crawled
- Spider::SPIDER_CRAWL_PAGE_ERROR - Trigged if an error occurs whilst crawling a page
You can find examples in the examples directory included in this repository.
To Run Tests
All versions of php-spider with dependencies
zendframework/zend-uri Version ^2.4
zendframework/zend-http Version ^2.4
zendframework/zend-eventmanager Version ^2.4
zendframework/zend-dom Version ^2.4