Libraries tagged by crawlora
kalmargabor/crawler-check
4346 Downloads
Verify Googlebot (or others) by their IP addresses
dachcom-digital/dynamic-search-data-provider-crawler
31383 Downloads
bitandblack/document-crawler
616 Downloads
Extract different parts of an HTML or XML document.
alikdex/yii2-crawler-detect
22637 Downloads
Yii2 extension for CrawlerDetect library
zfr/zfr-prerender
8443 Downloads
Integration with prerender.io service
revolution/salvager
10321 Downloads
Tiny WebCrawler for Laravel using Playwright
pushword/page-scanner
3442 Downloads
Find dead links, 404, 301 and more (command line or admin).
lobotomised/laravel-autocrawler
43828 Downloads
A tool to crawl your own laravel installation checking your HTTP status codes
kiwa/sitemap
5976 Downloads
Integrates the Bit&Black Sitemap into a Kiwa website to create sitemap files seamlessly.
kiddyu/beanbun
4118 Downloads
Beanbun 是用 PHP 编写的多进程网络爬虫框架，具有良好的开放性、高可扩展性
fkrzski/robots-txt
2226 Downloads
A modern, fluent PHP package for managing robots.txt rules with type safety and great developer experience
felipemateus/metatags
1217 Downloads
A Laravel package to fetch webpage metadata ( Open Graph | Twitter | Facebook | Article ) ect...
blogdaren/phpcreeper
949 Downloads
A new generation of multi-process async event-driven spider engine based on Workerman
bitandblack/sitemap
5489 Downloads
Creates a sitemap.xml by parsing the whole website.
godbout/htmlpagedom
30321 Downloads
jQuery-inspired DOM manipulation extension for Symfony's Crawler