Libraries tagged by scrawl
vdb/php-spider
173363 Downloads
A configurable and extensible PHP web spider
terminal42/escargot
1210558 Downloads
A web crawler or spider library based on Symfony components
spekulatius/phpscraper
122881 Downloads
PHPScraper, built with simplicity in mind. See tests/ for more examples.
spatie/laravel-site-search
116365 Downloads
A site search engine
duzun/hquery
115280 Downloads
An extremely fast web scraper that parses megabytes of HTML in a blink of an eye. No dependencies. PHP5+
stil/curl-easy
202506 Downloads
cURL wrapper for PHP. Supports parallel and non-blocking requests. For high speed crawling, see stil/curl-robot.
crwlr/crawler
10520 Downloads
Web crawling and scraping library.
crawlbase/crawlbase
35110 Downloads
A lightweight, dependency free PHP class that acts as wrapper for Crawlbase API
vipnytt/robotstxtparser
705113 Downloads
Robots.txt parsing library, with full support for every directive and specification.
spatie/http-status-check
47869 Downloads
CLI tool to crawl a website and check HTTP status code
proxycrawl/proxycrawl
87937 Downloads
A lightweight, dependency free PHP class that acts as wrapper for ProxyCrawl API
baba/sitemap-crawler
Downloads
smochin/instagram-php-crawler
8588 Downloads
A simple PHP Crawler for Instagram
nmure/crawler-detect-bundle
272485 Downloads
A Symfony bundle for the Crawler-Detect library (detects bots/crawlers/spiders via the user agent)
nadar/crawler
22431 Downloads
A highly extendible, dependency free Crawler for HTML, PDFS or any other type of Documents.