PHPCrawl is a webcrawler/webspider-library written in PHP. It supports filters, limiters, cookie-handling, robots.txt-handling, multiprocessing and much more.

Go to Download

razorcreations/nova-robots-editor

0 Favers
12905 Downloads

A Laravel Nova tool.

Go to Download

monsieurbiz/sylius-robots-txt-plugin

3 Favers
5146 Downloads

Manage your robot.txt from your Sylius admin pannel

Go to Download

merkandi/laravel-5-robots

0 Favers
17481 Downloads

Robots.txt Generator for Laravel 5

Go to Download

innoweb/silverstripe-robots

0 Favers
12941 Downloads

Adds a Robots.txt file that is configurable from /admin/settings/.

Go to Download

fifty-deg/sylius-robots-plugin

0 Favers
7708 Downloads

Channel defined robots.txt plugin for Sylius.

Go to Download

execut/yii2-robots-txt

7 Favers
12432 Downloads

Module for generate robots.txt file by url rules

Go to Download

evheniy/robots-txt-bundle

5 Favers
19479 Downloads

The RobotsTxtBundle adds the ability to generate robots.txt file to your application.

Go to Download

eliashaeussler/typo3-sitemap-robots

1 Favers
5614 Downloads

Extension for TYPO3 CMS to enhance robots.txt with sitemap configurations

Go to Download

creativestyle/magesuite-seo-meta-robots

0 Favers
11658 Downloads

Extension allows control over robots meta tag

Go to Download

becklyn/robots-txt

2 Favers
8321 Downloads

A library to build robots.txt

Go to Download

<< Previous Next >>