Libraries tagged by txt

User-Agent parser for robot rule sets

Go to Download

Manage your site's robots.txt file from the Filament admin panel, stored in the database and served dynamically.

Go to Download

Robots.txt Generator for Laravel 5

Go to Download

Use php-cs-fixer to format bats,blade.php,Dockerfile,env,json,md,mdx,sh,sql,tex,text,toml,txt,xml,yaml...files. - 使用 php-cs-fixer 去格式化 bats、blade.php、Dockerfile、env、json、md、mdx、sh、sql、tex、text、toml、txt、xml、yaml...文件。

Go to Download

Sitemap Locator - Looks up XML sitemaps as part of a configured site. Supports various sitemap providers, e.g. by configured page type or robots.txt, and allows to implement custom providers.

Go to Download

PHPCrawl is a webcrawler/webspider-library written in PHP. It supports filters, limiters, cookie-handling, robots.txt-handling, multiprocessing and much more.

Go to Download

security.txt (RFC 9116) generator, parser, validator

Go to Download

Build, dump and serve an llms.txt file from your Symfony application, the PrestaSitemapBundle way.

Go to Download

Expose WordPress content through /llms.txt endpoints

Go to Download

Panth LLMs.txt — AI Indexing Engine for Magento 2. Serves structured /llms.txt, /llms-full.txt and /llms.json with a dynamic multi-URL sitemap engine, weighted ranking, auto-generated summaries, product-type and use-case grouping, and cron-based cache warm-up. Built for ChatGPT, Claude, Perplexity, Gemini and future LLM crawlers.

Go to Download

Laravel robots txt package

Go to Download

A modern, fluent PHP package for managing robots.txt rules with type safety and great developer experience

Go to Download

Module for generate robots.txt file by url rules

Go to Download

DEPRECATED — use leopoletto/robots-txt-parser. Compatibility shim that delegates to it.

Go to Download

CakePHP 5 plugin: page meta, structured data (JSON-LD), sitemap, robots.txt, and 301/410 redirects from one page index

Go to Download