Libraries tagged by robots.txt
presprog/kirby-seo-plugin
287 Downloads
Opinionated SEO plugin for Kirby 4. With meta description, open graph, sitemap and robots.txt
mtconsultingroup/laravel-robots
4544 Downloads
Robots.txt generator service.
mmerian/phpcrawl
7119 Downloads
PHPCrawl is a webcrawler/webspider-library written in PHP. It supports filters, limiters, cookie-handling, robots.txt-handling, multiprocessing and much more.
mauricerenck/darkvisitors
262 Downloads
Kirby robots.txt plugin for blocking AI Crawlers and Bots
mad-web/laravel-robots
126897 Downloads
Robots.txt generator service.
fifty-deg/sylius-robots-plugin
7394 Downloads
Channel defined robots.txt plugin for Sylius.
devgeniem/wp-noindex-testing-staging-robots
13000 Downloads
Uses robots.txt to not index development/testing environments of this site with search-engines
anomaly/robots-extension
40851 Downloads
A simple robots.txt generator.
ellisthedev/laravel-5-robots
214357 Downloads
Robots.txt Generator for Laravel 5
owenmelbz/laravel-robots-txt
1105 Downloads
Laravel 5.x helpers for a semi-automated robots.txt and meta tags
webignition/robots-txt-parser
16814 Downloads
Parser for robots.txt files
bee4/robots.txt
9333 Downloads
Robots.txt parser and matcher
text-media/robotstxt-parser
106 Downloads
odorisioe/silverstripe-robots-txt
2608 Downloads
Feature to manage the robots.txt on SilverStripe
chrisguitarguy/wp-robots-txt
376 Downloads
Manage your robots.txt file from the WordPress admin