Libraries tagged by tx
healey/robots
194579 Downloads
Robots.txt Generator for Laravel 4 and 5
tomverran/robots-txt-checker
53531 Downloads
Given a robots.txt file, user agent and URL path will tell you whether you're allowed to access a page
frosh/robots-txt
23988 Downloads
Generate robots.txt
crwlr/robots-txt
16981 Downloads
Robots Exclusion Standard/Protocol Parser for Web Crawling/Scraping
bnomei/kirby3-robots-txt
44933 Downloads
Manage a virtual robots.txt from the Kirby config file
kornrunner/ethereum-token
11358 Downloads
PHP Ethereum Token Utils
vipnytt/useragentparser
871517 Downloads
User-Agent parser for robot rule sets
serendipity_hq/component-text-matrix
152355 Downloads
A PHP library to render an array representing a matrix as a plain text table.
wpreadme2markdown/wpreadme2markdown
59733 Downloads
Convert WordPress Plugin readme.txt to Markdown
cuab/phpcrawl
36596 Downloads
PHPCrawl is a webcrawler/webspider-library written in PHP. It supports filters, limiters, cookie-handling, robots.txt-handling, multiprocessing and much more.
stolt/llms-txt-php
453 Downloads
A library for writing, reading, and validating llms.txt Markdown files.
spaze/security-txt
5073 Downloads
security.txt (RFC 9116) generator, parser, validator
pronamic/changelog-md-to-wordpress-plugin-readme-txt
5127 Downloads
Automates syncing of changelog entries from CHANGELOG.md to WordPress plugin readme.txt.
monsieurbiz/sylius-robots-txt-plugin
5368 Downloads
Manage your robot.txt from your Sylius admin pannel
execut/yii2-robots-txt
12460 Downloads
Module for generate robots.txt file by url rules