Libraries tagged by robots-txt
cuab/phpcrawl
35668 Downloads
PHPCrawl is a webcrawler/webspider-library written in PHP. It supports filters, limiters, cookie-handling, robots.txt-handling, multiprocessing and much more.
monsieurbiz/sylius-robots-txt-plugin
4218 Downloads
Manage your robot.txt from your Sylius admin pannel
fkrzski/robots-txt
638 Downloads
A modern, fluent PHP package for managing robots.txt rules with type safety and great developer experience
execut/yii2-robots-txt
12246 Downloads
Module for generate robots.txt file by url rules
evheniy/robots-txt-bundle
18874 Downloads
The RobotsTxtBundle adds the ability to generate robots.txt file to your application.
merkandi/laravel-5-robots
15077 Downloads
Robots.txt Generator for Laravel 5
glicer/link-checker
13663 Downloads
Check broken links in html files, sitemap.xml and robots.txt
fifty-deg/sylius-robots-plugin
6952 Downloads
Channel defined robots.txt plugin for Sylius.
fuelviews/laravel-robots-txt
617 Downloads
Laravel robots txt package
becklyn/robots-txt
7980 Downloads
A library to build robots.txt
jazzman/php-robots
86 Downloads
Generator robots.txt
innoweb/silverstripe-robots
12214 Downloads
Adds a Robots.txt file that is configurable from /admin/settings/.
hettiger/seo-aggregator
371 Downloads
Generate sitemap.xml and robots.txt files in Laravel or any PHP Application with ease.
gverschuur/laravel-robotstxt
51 Downloads
Set the robots.txt content dynamically based on the Laravel app environment.
webvpf/wn-robots-plugin
867 Downloads
Robots - plugin for Winter CMS. Adds robots.txt for your site.