Libraries tagged by 64Robots

nawarian/php-bot
11 Favers
49 Downloads

A PHP library for creating automation robots

Go to Download

m6web/roboxt
19 Favers
574 Downloads

lib used for parsing a robots.txt file

Go to Download

brittainmedia/phpcrawl
11 Favers
3491 Downloads

PHPCrawl is a webcrawler/webspider-library written in PHP. It supports filters, limiters, cookie-handling, robots.txt-handling, multiprocessing and much more.

Go to Download

wdmg/yii2-robots
0 Favers
407 Downloads

Module for automatically generating and edit the robots.txt file

Go to Download

visma/module-seo-meta-robots
0 Favers
5954 Downloads

Extension allows control over robots meta tag

Go to Download

vipnytt/robotstagparser
3 Favers
368 Downloads

X-Robots-Tag HTTP header parser class

Go to Download

vanlinhha/robots-counter
0 Favers
11 Downloads

Count and report how many times a bot visit your website

Go to Download

tourze/robots-txt-bundle
0 Favers
0 Downloads

robots.txt管理

Go to Download

text-media/robotstxt-parser
0 Favers
107 Downloads

Go to Download

test-magenx/module-robots
0 Favers
0 Downloads

N/A

Go to Download

tekord/robots-txt-provider
0 Favers
4 Downloads

Provides various framework-agnostic ways to generate the contents of the robots.txt file

Go to Download

svetamor/robots_component
0 Favers
2 Downloads

create a file robots.txt

Go to Download

steein/robots
2 Favers
6329 Downloads

Generator robots.txt

Go to Download

spotman/kohana-robots.txt
0 Favers
10 Downloads

Kohana module for providing robots.txt file

Go to Download

sonhb99/robots-txt
0 Favers
4 Downloads

Determine if a page may be crawled from robots.txt and robots meta tags

Go to Download

<< Previous Next >>