Libraries tagged by robots-txt
mortola/behat-seo-contexts
5289 Downloads
Behat extension for testing some On-Page SEO factors: meta title/description, canonical, hreflang, meta robots, robots.txt, redirects, sitemap validation, HTML validation, performance...
m6web/roboxt
573 Downloads
lib used for parsing a robots.txt file
tekord/robots-txt-provider
3 Downloads
Provides various framework-agnostic ways to generate the contents of the robots.txt file
spotman/kohana-robots.txt
9 Downloads
Kohana module for providing robots.txt file
sonhb99/robots-txt
4 Downloads
Determine if a page may be crawled from robots.txt and robots meta tags
ricbra/robots-txt-bundle
5487 Downloads
Simple Symfony2 which generates robots.txt based on config setting
ranvis/robots-txt-processor-test
2 Downloads
robots.txt processor feature/behavior test.
ranvis/robots-txt-processor
21 Downloads
robots.txt filter and tester for untrusted source.
miisieq/robots-txt-bundle
2506 Downloads
Robots.txt generator for Symfony 2, 3 & 4
hugsbrugs/php-robots-txt
35 Downloads
PHP Robots.txt Utilities
cybercog/robots-txt
2401 Downloads
Robots.txt Generator
cgoit/contao-robots-txt-bundle
18 Downloads
Gives you the option to use a centralized robots.txt configuration for multiple contao installations.
b1tc0re/robots-txt
38 Downloads
Generator robots.txt
itinerisltd/add-yoast-seo-sitemap-to-robots-txt
17141 Downloads
Add Yoast SEO Sitemap to robots.txt
hofff/contao-robots-txt-editor
123 Downloads
Editor for auto creation and modification of robots.txt.