Libraries tagged by web crawling
crwlr/crawler
6275 Downloads
Web crawling and scraping library.
duzun/hquery
97918 Downloads
An extremely fast web scraper that parses megabytes of HTML in a blink of an eye. No dependencies. PHP5+
crwlr/robots-txt
10359 Downloads
Robots Exclusion Standard/Protocol Parser for Web Crawling/Scraping
cleantalk/anti-ddos-lite
1265 Downloads
A small PHP app to protect your site against DDoS attack or crawling web site by bots.
crawlzone/crawlzone
5082 Downloads
Crawlzone is a fast asynchronous internet crawling framework aiming to provide open source web search and testing solution. It can be used for a wide range of purposes, from extracting and indexing structured data to monitoring and automated testing.
reactphp-x/limiter
58 Downloads
A generic rate limiter for the web and reactphp. Useful for API clients, web crawling, or other tasks that need to be throttled
mehrabx/web-crawler
8 Downloads
A web crawler package
proclnas/curl-rox
18 Downloads
Just another curl wrapper for webCrawling purposes
rafaelglikis/sinama
15 Downloads
Web scraping library
laurentvw/scrapher
2524 Downloads
A web scraper for PHP to easily extract data from web pages
laurentvw/lavacrawler
95 Downloads
A web scraper for PHP to easily extract data from web pages
matheusfaustino/phrawl
3 Downloads
A web crawling framework in PHP
gavinggordon/sorcerer
17 Downloads
An easy-to-use PHP class for scraping webpages' source code.
biberltd/crawler-bundle
137 Downloads
Crawles web pages
proclnas/curl-rox-volatile
4 Downloads
Just another curl wrapper for webCrawling purposes (pthreads modified)