Libraries tagged by robotstxt

Dynamic robots.txt files for Laravel projects

Go to Download

Non Production Robots Ignore is a simple WordPress plugin that is used with vlucas/phpdotenv (or similar) that sets environment variables to set robots.txt to ignore all on all environments except production.

Go to Download

A route for serving a basic robots.txt in Laravel 5.1+, based on configuration settings.

Go to Download

A PHP 5.3+ framework for Robots.txt

Go to Download

Laravel robots.txt manager

Go to Download

Increase and control index-ability of your site; PHP library for generating sitemaps and controlling all the robots.

Go to Download

Professional SEO/AI indexing + Sitemap (index) + RSS + News/Image/Video sitemaps for CodeIgniter 4

Go to Download

Deploy Wordpress via composer with extra tools to make it easier.

Go to Download

PHPCrawl is a webcrawler/webspider-library written in PHP. It supports filters, limiters, cookie-handling, robots.txt-handling, multiprocessing and much more.

Go to Download

Behat extension for testing some On-Page SEO factors: meta title/description, canonical, hreflang, meta robots, robots.txt, redirects, sitemap validation, HTML validation, performance...

Go to Download

create a file robots.txt

Go to Download

Robots.txt Generator for Laravel 5

Go to Download

Opinionated SEO plugin for Kirby 4. With meta description, open graph, sitemap and robots.txt

Go to Download

Robots.txt Generator for Laravel 5

Go to Download

Yii2 Robots.txt Generator Module

Go to Download