Download the PHP package indexnowkit/core without Composer

IndexNow client for PHP — indexnowkit/core

Tell Yandex, Bing and the other IndexNow engines which URLs changed, from any PHP application. Batching, debounce, throttling, retry policy, key file handling and the #[IndexNow] rule model, on top of PSR-18 / PSR-17 / PSR-3 / PSR-16 only. The framework adapters (Symfony, Doctrine, Laravel, Yii2, Yii3) and the add-on packages build on it; use it directly in plain PHP, a CMS plugin or a custom framework.

github.com/indexnowkit/php (the php-* repositories are read-only splits)

Who gets notified

Yandex, Bing (and DuckDuckGo via Bing), Naver, Seznam, Yep, Internet Archive, Amazon — every engine in the IndexNow registry. One request to the shared endpoint api.indexnow.org reaches all of them; name engines explicitly (engines: [yandex, bing]) only to reach a single one. Internet Archive has no working direct endpoint at the time of writing — it is reached through api.

Google: no. Google does not support IndexNow, its sitemap ping endpoint is gone and the Indexing API is limited to JobPosting / BroadcastEvent. Keep your sitemap for Google; this library will not pretend otherwise.

Notification, not indexing. IndexNow tells an engine that a URL changed; whether and when the page is crawled and indexed is the engine's decision. See the result in Bing Webmaster Tools (IndexNow Insights) and Yandex.Webmaster (Indexing → Reindex pages); a useful metric is the share of submitted URLs in the index after a few days. Deleted pages: answer 410 (gone for good) or 404 (temporarily); for a move answer 301 and submit both URLs; a soft-404 or a redirect to the home page does harm. Bing's URL Submission API and Google's Indexing API are different protocols and not covered here.

Why this over X

Most IndexNow packages are a thin HTTP client: you collect the URLs, you call it, you read the answer. This family does the part that goes wrong in practice:

• Declared on the model (#[IndexNow]) and submitted from the ORM hooks — no controller code to forget.
• After the commit, not on flush: a rolled-back transaction announces nothing.
• Debounce (10 minutes per URL, shared through your cache), batches of up to 10 000 URLs, one key per host from env.
• Answers handled: 202 (key pending), 422, 429 with Retry-After back-off and a retry through your queue, 403 escalation.
• check before the first submission says what is wrong (key file, engines, queue, cache, environment); explain says why a URL was or was not sent.
• One core under the Symfony, Laravel, Yii2, Yii3 and Doctrine adapters with a shared conformance suite: the same behaviour everywhere, documented once.

Install

If you use a framework, prefer its adapter: it wires everything below through your container and hooks into entity changes. The family:

Quick start

submit() never throws for remote problems: every engine × host × batch yields a Result and a log line, and URLs that were not sent (debounced, disabled, dry-run, unknown host) yield a skipped result that says why.

The key file

Search engines verify ownership by fetching https://{host}/{key}.txt, whose body must be exactly the key.

Serve it with 200 OK and text/plain, without redirects; KeyFileResponder::headers() has the right headers. A key file elsewhere on the host is fine with key_location. Check\Checker validates the configuration, fetches every key file and, with liveProbe: true, sends a real probe. 403 always means the key file is wrong; rotation guidance is in docs/operations.md.

What happens to a URL

1. Normalize — relative paths resolved against base_url, scheme and host lower-cased, IDN hosts to punycode, default ports and fragments removed, dot-segments resolved. Anything that is not a public http(s) URL is dropped with a warning.
2. De-duplicate within the call, then debounce: URLs sent successfully in the last debounce.per_url seconds are skipped. A failing store never blocks delivery, it just stops de-duplicating and logs a warning.
3. Group by host and look up the key. Hosts without a key are skipped and never sent under another host's key.
4. Chunk into at most batch.max_urls URLs, throttle one token per HTTP request, and POST one batch per endpoint: {"host", "key", "keyLocation"?, "urlList"} as application/json; charset=utf-8.
5. Interpret the answer into a Result and mark successful URLs in the debounce store.

Results

Reason is the stable identifier for metrics and alerts, Result::$error the human sentence; Reason::translationKey() (indexnowkit.reason.<value>) names the message for a UI. Decide whether to retry from Result::$retryable, not from the reason. Result also carries engine, endpoint, host, urls, httpCode and metricLabels(); Result::retryableUrls($results) collects what is worth retrying.

Log lines go to the PSR-3 logger you pass to IndexNowKit::create(). See docs/operations.md for the levels, the exact messages and a "my URL was not submitted" checklist.

Declaring pages: #[IndexNow]

#[IndexNow] is repeatable: one attribute per family of public URLs the object has. Exactly one source per rule — route, resolver, via, url or urls. Class-wide policy goes to #[IndexNowDefaults], whose when is ANDed with each rule's own when (a draft page is never public, whatever the rule says).

Typed parameter sources, next to the plain accessor string (property, getter, is/has method, dotted.path, self):

Other shapes, all real cases:

Rules are inherited from parent classes and identified by name (derived from the source, or given explicitly): a subclass rule whose name repeats an ancestor's replaces it, a new name adds a page.

Deletion semantics

Visibility (when) is evaluated per rule, before and after a change. true → false submits that rule's URLs as a deletion so engines recrawl the 404; false → true is a creation; no transition is an update filtered by fields. Deleting an object whose rule does not apply submits nothing: the page was never public.

when is often a getter (isPublished) while the ORM change set holds the field (published). The convention isPublished → published/is_published and getStatus → status is applied automatically; when the names are unrelated, name the backing fields with whenFields. A status string or enum is not a boolean: use when: new Equals('status', 'published') (IndexNowKit\Attribute\Param\Equals); rules registered at runtime may pass a closure.

Full model, semantics table and the adapter-facing types (UrlRule, RuleSet, RuleRegistry): docs/attribute-reference.md.

urlsFor(), explain() and submitEntity() go through GuardedUrlResolver, which never throws: an invalid attribute is logged and yields no URLs, so a typo cannot break a flush.

Configuration

Also key_file.enabled, http.user_agent and key_location. Every value is validated at construction, so a bad setup fails at boot, not at the first submission. Full reference, per-host overrides, Config::with(), Config::OPTIONS and unknownOptions(): docs/configuration.md.

Retries, queues and bulk

No retries inside a web request: 429/5xx come back as retryable results. Use RetryingSubmitter in CLI, cron and workers, or re-enqueue Result::retryableUrls($results) after (new RetryPolicy())->delayAfter($results, $attempt) seconds. Collect during a unit of work, deliver once:

See docs/retries-and-queues.md for the worker recipe and bulk/migration guidance.

Re-announcing a bulk change from the site's own URL list is the job of the add-on package in the family table (Install); $kit->transport is the transport such consumers read through.

Adapters prove their wiring with Testing\Conformance\CoreConformanceTestCase: extend it, return the facade your container built and its FakeTransport, and the protocol scenarios of the spec run against it.

Testing

IndexNowKit\Testing is part of the published package: FakeTransport (records POSTs, answers queued responses), ArrayLogger, FrozenClock, RecordingDispatcher.

More recipes in docs/testing.md.

Extension points

Pass any of them to IndexNowKit::create() by name, or assemble the graph by hand: Client → Submitter → Collector + DispatcherInterface → IndexNowKit. The pieces a framework adapter wires from its configuration have factories with one source of error texts — Http\TransportFactory::lazy() (http.client), Debounce\DebounceStoreFactory::fromConfig() (debounce.store), Dispatch\DispatcherFactory::fromConfig() (dispatch), fromConfig() on Collector, TokenBucket, AttributeUrlResolver and KeyFileResponder — and Adapter\ConfigFactory turns a raw framework array into a Config without ever throwing from a hook. A container that assembles at runtime describes the whole graph once with Adapter\ServicesBuilder and gets it lazily from Adapter\Services; ORM hooks share Hook\ObserverHelper, queue jobs Retry\WorkerOutcome, commands the runners and Console\Definitions of indexnowkit/console. Writing an adapter? docs/adapters.md.

Exceptions

All exceptions implement IndexNowKit\Exception\IndexNowException: ConfigurationException (invalid Config, attribute or resolver setup), InvalidUrlException (a URL that cannot be submitted, caught by Submitter and dropped with a warning), InvalidArgumentException (programming errors) and Http\Exception\TransportException (network failure, turned into a retryable Result by Client; consumers reading documents through the transport see it; Checker turns it into an error line). Nothing throws out of a lifecycle hook — see the error contract in docs/adapters.md.

Limitations

• The same URL is not re-sent within debounce.per_url (10 minutes by default): that is what Yandex asks for.
• No retries inside a web request; TokenBucket throttles per process. Multi-process limits belong to your queue.
• Only http(s) URLs on hosts you hold a key for. Sub-domains are separate hosts, each with its own key file.
• Bulk ORM operations bypass entity hooks in every adapter: submit those URLs yourself.
• Google is not reachable through IndexNow.

Requirements

PHP 8.2+, ext-json, ext-filter, a PSR-18 client with PSR-17 factories (symfony/http-client and Guzzle are configured automatically with the timeout and no redirects; other clients are used as is). Optional: ext-intl (IDN via UTS #46, otherwise a pure-PHP punycode encoder).

Versioning

SemVer. Before 1.0, minor versions may contain breaking changes; they are listed in CHANGELOG.md. What is covered by the promise and what is not: docs/bc.md.

Notes for AI assistants

• Composer package indexnowkit/core (PHP ^8.2, PSR-18 client + PSR-17 factories needed). Framework users want an adapter instead: indexnowkit/symfony-bundle, indexnowkit/laravel, indexnowkit/yii2, indexnowkit/yii3, indexnowkit/doctrine.

• Minimal complete snippet (every use included):

• Verify: (new IndexNowKit\Check\Checker($config, $indexNow->keys, $indexNow->transport))->run() is what the adapters' check command runs; $indexNow->explain($object) shows the rule behind every URL; every remote outcome is a Result with status/reason, nothing throws.
• Pitfalls:
  • dispatch: auto exists in Symfony (auto | messenger | sync | none) and Yii2 (auto | queue | sync | none), not in Laravel (queue | sync | none); Yii3 has sync | none only.
  • Locales: router.locales in Laravel, Yii2 and Yii3, framework.enabled_locales in Symfony; locales: 'all' on a rule uses that list.
  • url: names an accessor (method or property) that returns the URL; urls: is a list of literal URLs. Never put a literal in url:.
  • A string in when: is an accessor read as truthy (published, isPublished). A status string needs Equals: when: new Equals('status', 'published') (IndexNowKit\Attribute\Param\Equals).
  • Manual submission is submitEntity() in Symfony, submitModel() in Laravel, submitRecord() in Yii2 and Yii3; the commands are indexnow:submit-entity, indexnow:submit-model, indexnow/submit-record (Yii2), indexnow:submit-record (Yii3). Bulk queries (update(), DB::table(), updateAll()) fire no hooks: submit afterwards with those.
  • Laravel has two classes called IndexNowKit: the facade IndexNowKit\Laravel\Facades\IndexNowKit and the core service IndexNowKit\IndexNowKit (inject by type). Yii2 exposes the core through Yii::$app->indexnow->kit(); Yii3 defines IndexNowKit\IndexNowKit in the container.
  • Outside production a configured key with dry_run unset makes check fail (a staging copy would submit real URLs): set dry_run: true there, or dry_run: false explicitly when it submits on purpose.
  • Unknown configuration keys are warned about at boot (typos such as debounce.per_urls); the key list is Config::OPTIONS plus the adapter's own keys.

Other packages

Design rationale and the cross-language model: docs/spec. Conformance suite: indexnowkit/spec.

MIT. IndexNow is a trademark of its owner; this project is independent and not affiliated with Microsoft, Yandex or indexnow.org.