Libraries tagged by 64Robots

Check broken links in html files, sitemap.xml and robots.txt

Go to Download

Crawler for any type of site using robots.txt and sitemap.xml as the source of URL. Useful for cache regenerating.

Go to Download

Simple SEO class for WordPress to create page metatags: title, description, robots, keywords, Open Graph.

Go to Download

Sitemap and robots.txt generated from your site's own route table, plus schema.org helper builders.

Go to Download

A lightweight plugin that protects email addresses from email-harvesting robots by encoding them into decimal and hexadecimal entities.

Go to Download

This module blocks (well-behaved) search engine robots from crawling, indexing, or archiving your site by setting a "X-Robots-Tag: noindex,nofollow,noarchive" HTTP header.

Go to Download

A PHP library for creating automation robots

Go to Download

lib used for parsing a robots.txt file

Go to Download

PHPCrawl is a webcrawler/webspider-library written in PHP. It supports filters, limiters, cookie-handling, robots.txt-handling, multiprocessing and much more.

Go to Download

Module for automatically generating and edit the robots.txt file

Go to Download

Extension allows control over robots meta tag

Go to Download

Count and report how many times a bot visit your website

Go to Download

robots.txt管理

Go to Download

Go to Download

N/A

Go to Download