Libraries tagged by crawlers
proxycrawl/proxycrawl
68513 Downloads
A lightweight, dependency free PHP class that acts as wrapper for ProxyCrawl API
jyggen/curl
143097 Downloads
A simple and lightweight cURL library with support for asynchronous requests.
schliesser/sitecrawler
21957 Downloads
TYPO3 sitemap crawler
shel/crawler
1423 Downloads
Allows crawling of sitemaps and node-trees
nguyenanhung/my-crawler
1101 Downloads
Crawler helper, library - Basic, Simple and Lightweight
nadar/crawler
19653 Downloads
A highly extendible, dependency free Crawler for HTML, PDFS or any other type of Documents.
friends-of-hyva/magento2-crawler-session
1135 Downloads
Prevent crawlers from creating a session
alikdex/yii2-crawler-detect
18860 Downloads
Yii2 extension for CrawlerDetect library
hedii/php-crawler
1081 Downloads
A crawler application written with php and Laravel that finds email addresses on the internets.
rebelinblue/fluent-web-crawler
3324 Downloads
A web crawler with a fluent interface
katsana/dusk-crawler
11709 Downloads
Web Crawler using Laravel Dusk
hacoidev/ophim-crawler
3601 Downloads
ophim crawler
lobotomised/laravel-autocrawler
20123 Downloads
A tool to crawl your own laravel installation checking your HTTP status codes
crawlzone/crawlzone
5067 Downloads
Crawlzone is a fast asynchronous internet crawling framework aiming to provide open source web search and testing solution. It can be used for a wide range of purposes, from extracting and indexing structured data to monitoring and automated testing.
godbout/htmlpagedom
22954 Downloads
jQuery-inspired DOM manipulation extension for Symfony's Crawler