Libraries tagged by 64Robots
monsieurbiz/sylius-robots-txt-plugin
16208 Downloads
Manage your robot.txt from your Sylius admin pannel
frosh/robots-txt
37492 Downloads
Generate robots.txt
fkrzski/robots-txt
2707 Downloads
A modern, fluent PHP package for managing robots.txt rules with type safety and great developer experience
crwlr/robots-txt
37682 Downloads
Robots Exclusion Standard/Protocol Parser for Web Crawling/Scraping
bnomei/kirby3-robots-txt
50673 Downloads
Manage a virtual robots.txt from the Kirby config file
mguinea/laravel-robots
66846 Downloads
Laravel package to manage robots in an easy way
ellisthedev/laravel-5-robots
214576 Downloads
Robots.txt Generator for Laravel 5
vipnytt/useragentparser
1018018 Downloads
User-Agent parser for robot rule sets
eliashaeussler/typo3-sitemap-locator
344702 Downloads
Sitemap Locator - Looks up XML sitemaps as part of a configured site. Supports various sitemap providers, e.g. by configured page type or robots.txt, and allows to implement custom providers.
cuab/phpcrawl
40837 Downloads
PHPCrawl is a webcrawler/webspider-library written in PHP. It supports filters, limiters, cookie-handling, robots.txt-handling, multiprocessing and much more.
wizardcompass/robots-txt-parser
2811 Downloads
A PHP library to parse and analyze robots.txt files with validation and URL fetching capabilities
merkandi/laravel-5-robots
37743 Downloads
Robots.txt Generator for Laravel 5
mage-os/meta-robots-tag
924 Downloads
NoIndex NoFollow NoArchive Tag extension
innoweb/silverstripe-robots
18464 Downloads
Adds a Robots.txt file that is configurable from /admin/settings/.
fuelviews/laravel-robots-txt
6790 Downloads
Laravel robots txt package