Libraries tagged by robotstxt
beeg99/phpcrawl
0 Favers
3 Downloads
3 Downloads
PHPCrawl is a webcrawler/webspider-library written in PHP. It supports filters, limiters, cookie-handling, robots.txt-handling, multiprocessing and much more.
bakiro/laravel-5-robots
0 Favers
7292 Downloads
7292 Downloads
Robots.txt Generator for Laravel 5
anatoliy700/yii2-robots
0 Favers
2 Downloads
2 Downloads
Implements a dynamic robots.txt file
nox-it/yii2-nox-robots
1 Favers
372 Downloads
372 Downloads
Yii2 Robots.txt Generator Module
thepoddi/kirby3-seokit
5 Favers
10 Downloads
10 Downloads
SEO Kit for Kirby 3 including sitemap.xml and robotx.txt
melisplatform/melis-cms-site-robot
1 Favers
1987 Downloads
1987 Downloads
This module adds the dynamic generation of the robot.txt file to a tool and per site