Libraries tagged by crawler
mauricerenck/darkvisitors
260 Downloads
Kirby robots.txt plugin for blocking AI Crawlers and Bots
marcvanh/laravel-bot-block
1049 Downloads
A custom middleware package for Laravel. Temporarily blocks crawlers scanning for vulnerabilities.
bilginpro/agency-iha
580 Downloads
Crawler for IHA News Agency in Turkey.
bilginpro/agency-aa
731 Downloads
Crawler for AA News Agency in Turkey.
alapi/querylist
616 Downloads
Simple, elegant, extensible PHP Web Scraper (crawler/spider),Use the css3 dom selector,Based on phpQuery! 简洁、优雅、可扩展的PHP采集工具(爬虫)，基于phpQuery。
zhiyang/php-crawler
14 Downloads
Lightweight web crawler for php
zanysoft/laravel-crawler-detect
361 Downloads
A Laravel package to detect web crawlers via the user agent
webcoast/versatile-crawler
6610 Downloads
A versatile and extendable crawler for TYPO3 CMS
w3zone/crawler
115 Downloads
thingston/crawler
10 Downloads
Web crawler based on PHP Guzzle HTTP Client with concurrency support for faster operation.
teamtnt/crawler
1 Downloads
Distributed Crawler Architecture
repat/spatie-crawler-redis
9739 Downloads
Redis CrawlQueue for spatie/crawler
peimengc/crawler
38 Downloads
爬虫
nggiahao/crawler
8 Downloads
mediamonks/crawler-bundle
43 Downloads
Integrates the mediamonks/crawler package into Symfony