Libraries tagged by crawler
proxycrawl/proxycrawl
68390 Downloads
A lightweight, dependency free PHP class that acts as wrapper for ProxyCrawl API
jyggen/curl
143090 Downloads
A simple and lightweight cURL library with support for asynchronous requests.
schliesser/sitecrawler
21941 Downloads
TYPO3 sitemap crawler
shel/crawler
1421 Downloads
Allows crawling of sitemaps and node-trees
nguyenanhung/my-crawler
1100 Downloads
Crawler helper, library - Basic, Simple and Lightweight
nadar/crawler
19651 Downloads
A highly extendible, dependency free Crawler for HTML, PDFS or any other type of Documents.
friends-of-hyva/magento2-crawler-session
1128 Downloads
Prevent crawlers from creating a session
alikdex/yii2-crawler-detect
18859 Downloads
Yii2 extension for CrawlerDetect library
hedii/php-crawler
1081 Downloads
A crawler application written with php and Laravel that finds email addresses on the internets.
rebelinblue/fluent-web-crawler
3324 Downloads
A web crawler with a fluent interface
katsana/dusk-crawler
11704 Downloads
Web Crawler using Laravel Dusk
hacoidev/ophim-crawler
3596 Downloads
ophim crawler
lobotomised/laravel-autocrawler
20074 Downloads
A tool to crawl your own laravel installation checking your HTTP status codes
crawlzone/crawlzone
5063 Downloads
Crawlzone is a fast asynchronous internet crawling framework aiming to provide open source web search and testing solution. It can be used for a wide range of purposes, from extracting and indexing structured data to monitoring and automated testing.
godbout/htmlpagedom
22922 Downloads
jQuery-inspired DOM manipulation extension for Symfony's Crawler