Libraries tagged by bdcrawler
scrawler/router
1731 Downloads
An Fully Automatic RESTful PHP Router.
zenstruck/dom
1725 Downloads
DOM crawler with advanced selector API and assertions.
octopoda/octopus
4279 Downloads
PHP Sitemap crawler
marcvanh/laravel-bot-block
1009 Downloads
A custom middleware package for Laravel. Temporarily blocks crawlers scanning for vulnerabilities.
flancer32/mage2_ext_bot_sess
10540 Downloads
Magento2: prevent session creation for bots & crawlers.
crwlr/utils
10460 Downloads
Utilities that are needed in multiple crawler packages.
bugbuster/contao-botdetection-bundle
37619 Downloads
Contao bundle helper class to detect search engines, bots, spiders, crawlers ...
yurunsoft/crawler
80 Downloads
宇润爬虫框架(Yurun Crawler) 是一个低代码、高性能、分布式爬虫采集框架，这可能是最一把梭的爬虫框架。
yuan1994/z-crawler
52 Downloads
【正方教务】爬虫，支持成绩查询、考试查询、课表查询、四六级成绩查询、四六级报名、选课查询、修改密码、获取用户菜单等功能，并且解析数据成易读格式，符合 psr 规范，拿来即用
webprofil/crawler
385 Downloads
Crawls Website and can generate backstop json
umpirsky/centipede-crawler
450 Downloads
Crawls all unique links.
tnm/crawler-devbox-sitepackage
2688 Downloads
survos/crawler-bundle
3281 Downloads
Provides a way to create tests that crawl a site
shel/crawler
1825 Downloads
Allows crawling of sitemaps and node-trees
repat/spatie-crawler-redis
9738 Downloads
Redis CrawlQueue for spatie/crawler