PHPCrawl is a webcrawler/webspider-library written in PHP. It supports filters, limiters, cookie-handling, robots.txt-handling, multiprocessing and much more.

Go to Download

monsieurbiz/sylius-robots-txt-plugin

3 Favers
5921 Downloads

Manage your robot.txt from your Sylius admin pannel

Go to Download

merkandi/laravel-5-robots

0 Favers
19457 Downloads

Robots.txt Generator for Laravel 5

Go to Download

innoweb/silverstripe-robots

0 Favers
13585 Downloads

Adds a Robots.txt file that is configurable from /admin/settings/.

Go to Download

execut/yii2-robots-txt

7 Favers
12540 Downloads

Module for generate robots.txt file by url rules

Go to Download

evheniy/robots-txt-bundle

5 Favers
19828 Downloads

The RobotsTxtBundle adds the ability to generate robots.txt file to your application.

Go to Download

eliashaeussler/typo3-sitemap-robots

1 Favers
6310 Downloads

Extension for TYPO3 CMS to enhance robots.txt with sitemap configurations

Go to Download

bnomei/kirby3-htmlhead

23 Favers
3560 Downloads

Kirby Plugin for a best-practice HTML Head Element extendable with snippets.

Go to Download

mmerian/phpcrawl

59 Favers
7143 Downloads

PHPCrawl is a webcrawler/webspider-library written in PHP. It supports filters, limiters, cookie-handling, robots.txt-handling, multiprocessing and much more.

Go to Download

razorcreations/nova-robots-editor

0 Favers
13220 Downloads

A Laravel Nova tool.

Go to Download

pixeo/laravel-robots

1 Favers
3169 Downloads

Set the robots.txt content dynamically based on the Laravel app environment.

Go to Download

<< Previous Next >>