Libraries tagged by robodt

A simple robots.txt generator.

Go to Download

Parser for robots.txt files

Go to Download

Set the robots.txt content dynamically based on the Laravel app environment.

Go to Download

Headless manager for the well-known files every modern web app owes bots AND agents — robots.txt, .well-known/security.txt, llms.txt, humans.txt, sitemap.xml, and an agents/AI manifest. Define them once in code/config and serve them consistently. Zero runtime deps; ships a default-open robots evaluator so scrapers can honor robots honestly, and makes it hard to leak a protected path. Framework-agnostic core + Laravel adapter.

Go to Download

Symfony bundle for SEO management: meta tags (OpenGraph, Twitter Cards), Schema.org JSON-LD structured data, sitemap XML generation, and robots.txt

Go to Download

Laravel middleware za blokiranje AI crawlera i tražilica. Auto-registracija, konfigurabilan, podržava Laravel 10/11/12/13.

Go to Download

SEO management plugin for Filament v5 - meta tags, global settings, Schema.org, robots.txt

Go to Download

Shared, framework-agnostic building blocks for polite and safe web crawlers: SSRF-aware HTTP throttling, robots.txt parsing, and HTML link discovery.

Go to Download

A professional, production-ready PHP XML sitemap generator with advanced features: concurrency, robots.txt, caching, database storage, change detection, SEO analysis, JavaScript rendering, proxy support, webhooks, and more.

Go to Download

Pre-flight check for indexnowkit: a GET of every URL before it is submitted to IndexNow — noindex, robots.txt, canonical, redirects and origin errors are skipped; check --sample

Go to Download

Symfony 8 library providing a Doctrine ORM trait for SEO meta fields — title, description, keywords, Open Graph image, and robots behaviour

Go to Download

A package containing SEO helpers.

Go to Download

Bad Behaviour is a modern bot detection and blocking for PHP applications

Go to Download

Project Maintenance & Contributing Automation Tool

Go to Download

Laravel integration for BearyChat.

Go to Download