Download the PHP package vantoozz/proxy-scraper without Composer
On this page you can find all versions of the php package vantoozz/proxy-scraper. It is possible to download/install these versions without Composer. Possible dependencies are resolved automatically.
Informations about the package proxy-scraper
Proxy Scraper
Library for scraping free proxies lists written in PHP
Quick start
Older versions
This is version 3 of the library. For version 2 please check v2 branch; for version 1 please check v1 branch.
Upgrade
How to upgrade
Setup
The library requires a PSR-18 compatible HTTP client. To use the library you have to install any of them, e.g.:
All available clients are listed on Packagist: https://packagist.org/providers/psr/http-client-implementation.
Then install proxy-scraper library itself:
Usage
Auto-configuration
The simplest way to start using the library is to use proxyScraper()
function which instantiates and configures all
the scrapers.
Please note, auto-configuration function in addition to guzzlehttp/guzzle:~7
and guzzlehttp/psr7
requires hanneskod/classtools
dependency.
HTTP Client
In not using auto-configuration you will need an HTTP client.
The library provides guzzleHttpClient()
function creating and configuring the client.
You can create own HTTP client by implementing HttpClientInterface
:
Of course, you may manually configure the scraper and underlying HTTP client:
Single scraper
Composite scraper
You can easily get data from many scrapers at once:
Error handling
Sometimes things go wrong. This example shows how to handle errors while getting data from many scrapers:
Will output
In the same manner you may configure exceptions handling for the scraper created with proxyScraper()
function as it
returns an instance of CompositeScraper
:
Validating proxies
Validation steps may be added:
Will output
Metrics
A Proxy object may have metrics (metadata) associated with.
By default, Proxy object has source metric:
Will output
Note. Examples use Guzzle as HTTP client.
Testing
Unit tests
Integration tests
System tests
Upgrade from version 2
The biggest difference from version 2 is the HTTP client configuration.
Instead of
the client should be instantiated like
All versions of proxy-scraper with dependencies
psr/http-client Version ~1
psr/http-factory Version ~1
psr/http-client-implementation Version ~1
psr/http-message-implementation Version ~1
symfony/dom-crawler Version >=5.1.4
symfony/css-selector Version ~5