Download the PHP package indexnowkit/verify without Composer

IndexNow pre-flight check — indexnowkit/verify

One GET of every URL before it is sent to Yandex, Bing and the other IndexNow engines. A page that says noindex, that robots.txt disallows, that names another canonical URL, that redirects, or that the origin does not serve, is not submitted; each becomes a skipped Result with a stable Reason (noindex, robots_disallowed, non_canonical, redirected, origin_error) in the log, the listeners, the PSR-14 events and the submission store. The engines see only URLs worth crawling — announcing a noindex page or a redirect is at best wasted quota and at worst a reason for an engine to trust the key less.

What it never blocks: 404 and 410. A URL that answers 404 or 410 is submitted as is — that is the deletion signal IndexNow accepts, and the reason to submit a removed page in the first place. IndexNow is a notification, not indexing: the engine decides whether and when to crawl.

Off by default. Installing the package changes nothing until verify.enabled: true.

github.com/indexnowkit/php (the php-* repositories are read-only splits)

Install

With a framework adapter (indexnowkit/symfony-bundle, laravel, yii2, yii3) that is all: the adapter finds the package, adds the verify block to its configuration and, once verify.enabled is true, decorates its submitter — the queue, Messenger and yii2-queue workers verify too, because they get the same submitter. check prints one line about it either way (verify: installed, disabled (verify.enabled: false)).

What one GET decides

The GET is a real GET (not HEAD: many origins answer HEAD differently), without following redirects, with verify.timeout (5 s) and its own User-Agent (verify.user_agent). Only the <head> of the first 256 KiB is read for the signals; a body that is not text/html / application/xhtml+xml contributes headers only. Comments, attribute order and case do not matter; a relative <link rel="canonical"> resolves against the document's <base href> when it declares one, as a crawler resolves it.

The pre-flight uses its own PSR-18 client, discovered by the core with max_redirects: 0 — not the application's http.client, even when one is configured. It has to see the 3xx answers itself (verify.redirect, verify.max_redirects, the host check on redirect targets), and PSR-18 cannot tell a client someone else built to stop following redirects. http.client keeps carrying the POST submissions; check prints the split (verify.transport).

robots.txt is fetched once per host and process and kept in the PSR-16 cache behind debounce.store (verify.robots_cache_ttl, an hour). A robots.txt that cannot be fetched (not 200, not 404) blocks nothing: one warning per host, every path is treated as allowed — an unreachable robots.txt is no reason to keep quiet.

Time. One blocking GET per URL, sequentially, verify.time_budget (60 s) for the whole batch: past it the rest is sent unverified with one warning, so a queue job never outlives its visibility timeout and gets a second worker.

Batches. A batch above verify.max_batch (100) URLs is sent unverified with one warning: that is the sitemap command, where the site itself lists its URLs (indexnow:sitemap --no-verify says so explicitly). submit --dry-run runs the pre-flight (a GET is harmless) so it shows what would be skipped.

dispatch: sync and verify.delay

With dispatch: sync the pre-flight GETs run inside the web request that changed the page: N URLs cost N round trips to your own origin, and on a single-process development server (symfony server:start without workers, php artisan serve, php yii serve) a request to yourself hangs until the timeout. check warns about it (verify.dispatch). Use dispatch: queue / messenger, or keep verify.enabled: false outside production.

verify.delay (seconds, at most 30) waits before the first GET of a batch, so a queue worker that runs right after the commit does not fetch the page from a cache that still holds the old version; it applies only outside a web request and is ignored with dispatch: sync.

check --sample

Every sample is fetched and reported as one line (verify sample https://…: HTTP 200, index, canonical: self, robots: allowed, code verify.sample, the host of the URL). --sample-class takes a class annotated with #[IndexNow] (<FQCN> — up to three objects of it, <FQCN>:<id> — that one) and resolves their URLs the way the submission does. A noindex, a disallow, another canonical, a redirect, a 4xx/5xx or an unreachable origin is a warning, never an error: production may be unreachable from CI, and the report must not fail a deploy over it. Without the package --sample is an error (check --sample needs indexnowkit/verify), check without samples prints a no sample given line (code verify.sample).

Plain PHP

VerifyingSubmitter implements SubmitterInterface; listeners registered on it see every result (the inner ones and the skipped ones) exactly once; pass the same PSR-14 dispatcher and submission store you gave the inner submitter so the skipped results reach them too. The log lines are fixed strings operators can grep (docs/configuration.md.

For adapter authors

Verify\Adapter\VerifyServices is what a framework adapter wires for this package, in one place: the predicate (package()), the owned options, the validated block, the pre-flight transport, the robots cache, the decorated submitter and command submitter factory, the check lines — as static functions over the pieces, with *For() twins over the core's Adapter\Services for a runtime graph. The Symfony bundle, the Laravel, the Yii2 and the Yii3 adapters build on it; see adapters.md of the core.

Requirements

PHP 8.2+, indexnowkit/core ^0.11; indexnowkit/console ^0.4 for the --sample options of the check command (every adapter has it).

Notes for AI assistants

• Composer package indexnowkit/verify (add-on of indexnowkit/core). With a framework adapter nothing is wired by hand: verify.enabled: true in the adapter's configuration decorates the submitter; the workers of the queue verify too.

• Minimal complete snippet (every use included):

• Verify: bin/console indexnow:check --sample=https://www.example.com/page, php artisan indexnow:check --sample=…, php yii indexnow/check --sample=… — one line per sample, warnings only; check alone prints the verify: line (installed / disabled / not installed) and, with the package, a "no sample given" line.
• Pitfalls:
  • verify.enabled is false by default; installing changes nothing. With dispatch: sync the GETs run inside the web request (check warns): use a queue.
  • 404 and 410 pass on purpose (deletions); verify.redirect: follow submits both URLs after a 301/308, only the original after a 302/303/307.
  • A canonical or a redirect target on a host without a key in hosts / base_url is a skip, never a submission: a page cannot make you announce another site.
  • robots.txt unavailable (500, timeout) allows everything with one warning; 404 is the normal "no robots.txt".
  • Batches above verify.max_batch (100) go unverified with a warning — that is the sitemap command; indexnow:sitemap --no-verify says it explicitly.
  • indexnow:check --sample needs this package; without it the option is an error with the install line.
  • dispatch: auto exists in Symfony and Yii2, not in Laravel (Yii3: sync | none); locales are router.locales (Laravel, Yii2, Yii3), framework.enabled_locales (Symfony).

Versioning

SemVer; until 1.0 minor versions may contain breaking changes, listed in CHANGELOG.md. What the compatibility promise covers: docs/bc.md.

MIT. IndexNow is a trademark of its owner; this project is independent and not affiliated with Microsoft, Yandex or indexnow.org.