Libraries tagged by web crawling
duzun/hquery
105384 Downloads
An extremely fast web scraper that parses megabytes of HTML in a blink of an eye. No dependencies. PHP5+
crwlr/crawler
8805 Downloads
Web crawling and scraping library.
crwlr/robots-txt
14012 Downloads
Robots Exclusion Standard/Protocol Parser for Web Crawling/Scraping
cleantalk/anti-ddos-lite
1960 Downloads
A small PHP app to protect your site against DDoS attack or crawling web site by bots.
crawlzone/crawlzone
5169 Downloads
Crawlzone is a fast asynchronous internet crawling framework aiming to provide open source web search and testing solution. It can be used for a wide range of purposes, from extracting and indexing structured data to monitoring and automated testing.
proclnas/curl-rox
18 Downloads
Just another curl wrapper for webCrawling purposes
laurentvw/scrapher
2527 Downloads
A web scraper for PHP to easily extract data from web pages
laurentvw/lavacrawler
95 Downloads
A web scraper for PHP to easily extract data from web pages
reactphp-x/limiter
67 Downloads
A generic rate limiter for the web and reactphp. Useful for API clients, web crawling, or other tasks that need to be throttled
mehrabx/web-crawler
8 Downloads
A web crawler package
matheusfaustino/phrawl
3 Downloads
A web crawling framework in PHP
gavinggordon/sorcerer
17 Downloads
An easy-to-use PHP class for scraping webpages' source code.
biberltd/crawler-bundle
137 Downloads
Crawles web pages
proclnas/curl-rox-volatile
4 Downloads
Just another curl wrapper for webCrawling purposes (pthreads modified)
gyaaniguy/pcrawl
8 Downloads
PHP web scraping and crawling library. With support for multiple clients, fast parsing, debugging and on the fly changes to various options