Libraries tagged by robo
consolidation/robo
68480657 Downloads
Modern task runner
nette/robot-loader
56101659 Downloads
🍀 Nette RobotLoader: high performance and comfortable autoloader that will search and autoload classes within your application.
spatie/robots-txt
19833693 Downloads
Determine if a page may be crawled from robots.txt and robots meta tags
consolidation/filter-via-dot-access-data
51783732 Downloads
This project uses dflydev/dot-access-data to provide simple output filtering for applications built with annotated-command / Robo.
codegyre/robo
1015327 Downloads
Modern task runner
spatie/laravel-robots-middleware
2334573 Downloads
Add an `all` or `none` robots header to your requests via a middleware in Laravel
vipnytt/sitemapparser
2422570 Downloads
XML Sitemap parser class compliant with the Sitemaps.org protocol.
webignition/robots-txt-file
1457768 Downloads
Models a robots.txt file
codeception/robo-paracept
3728989 Downloads
Codeception Parallel Execution Tasks via Robo Task Runner
bopoda/robots-txt-parser
303486 Downloads
PHP Class for parsing robots.txt files according to Google, Yandex specifications.
monperrus/crawler-user-agents
6671 Downloads
This repository contains a list of of HTTP user-agents used by robots, crawlers, and spiders as in single JSON file.
vipnytt/robotstxtparser
809851 Downloads
Robots.txt parsing library, with full support for every directive and specification.
verschuur/laravel-robotstxt
440594 Downloads
Set the robots.txt content dynamically based on the Laravel app environment.
t1gor/robots-txt-parser
504696 Downloads
PHP class to parse robots.txt rules according to Google, Yandex, W3C and The Web Robots Pages specifications.
mage-os/module-meta-robots-tag
24908 Downloads
NoIndex NoFollow NoArchive Tag extension