Libraries tagged by ai-crawler

Serve Markdown versions of your Craft CMS pages to AI crawlers and LLMs via .md URLs, content negotiation, and user-agent detection.

Go to Download

Bot and AI-crawler control presets for the Phirewall PHP firewall - block AI scrapers and rate-limit aggressive SEO crawlers

Go to Download

Magento 2 module for AI Engine Optimization (AEO). Injects AI crawler rules (OAI-SearchBot, GPTBot, ChatGPT-User, PerplexityBot, Perplexity-User, Google-Extended, ClaudeBot, anthropic-ai, Claude-User, Applebot, cohere-ai, Amazonbot, Meta-ExternalAgent) into robots.txt — without overwriting your existing configuration. Supports per-bot Allow/Disallow lists, Crawl-delay, Sitemap directives, multi-store, and a public Api\RobotsStatusInterface for cross-module integration with angeo/module-aeo-audit.

Go to Download

Laravel package for smking AEO — auto-inject JSON-LD, FAQ, and AI summary into Laravel responses so AI crawlers (ChatGPT, Perplexity, Google AI) can cite your pages.

Go to Download

Server-side AI crawler & bot traffic tracking for Laravel, reporting to DataFast. Unofficial PHP port of @datafast/ai-crawl.

Go to Download

Laravel middleware za blokiranje AI crawlera i tražilica. Auto-registracija, konfigurabilan, podržava Laravel 10/11/12/13.

Go to Download

Kirby robots.txt plugin for blocking AI Crawlers and Bots

Go to Download

Marked Down serves Markdown to AI crawlers, command-line tools, and APIs while browsers get HTML. Zero-config per-section setup, automatic AI bot detection, .md URL suffix, /llms.txt, YAML front matter, and dedicated Markdown templates — out of the box.

Go to Download

Generate product feed for AI Crawlers

Go to Download

Ban AI crawlers from visiting your Symfony website or application

Go to Download

Generative Engine Optimization (GEO) for Laravel: llms.txt, JSON-LD structured data, and AI crawler tooling.

Go to Download

LLMO (Large Language Model Optimization) for Magento 2. ACP-style product feed, schema validation, AI crawler controls, AI-ready content fields, attribution and conversion tracking, campaign abstraction for AI ads, and an LLMO visibility dashboard.

Go to Download

SEO and GEO for Craft 5 — meta, schema, redirects, llms.txt, AI crawler controls. API-first, with PHPBench-enforced performance budgets.

Go to Download