Download the PHP package lbonnet/link-checker-bundle without Composer

LinkCheckerBundle

A Symfony bundle to crawl a site and detect broken internal and external links.

Designed to run outside the request/response cycle — as a console command, a scheduled cron, or an async Messenger worker — so it fits both CI pipelines and continuous monitoring of a live site.

Requirements

• PHP >= 8.1
• Symfony 6.4, 7.x, or 8.x

Installation

If you don't use Symfony Flex, enable the bundle manually in config/bundles.php:

Configuration

Create config/packages/link_checker.yaml:

Usage

1. Console Command (CLI & CI)

Run an on-demand audit directly from the command line:

[!TIP] CI / Exit Codes: The command returns 0 (Command::SUCCESS) if no broken links are found, and 1 (Command::FAILURE) if any broken links are detected. This makes it ideal for pull request validations and deployment checks.

2. Asynchronous Execution (Messenger)

The bundle provides a CheckLinksMessage and its handler to offload the crawl to an asynchronous worker queue:

3. Automated Monitoring (Symfony Scheduler)

If you use symfony/scheduler (Symfony 6.3+), you can schedule periodic audits in your application's ScheduleProvider:

4. Custom Notifications & Event Handling

When a crawl completes, a CrawlCompletedEvent is dispatched. You can listen to this event to send alerts (Slack, Email, Discord) or perform custom actions:

[!NOTE] If a CrawlCompletedEvent listener throws (e.g., a misconfigured notifier transport), the bundle catches and logs the error instead of letting it propagate — a broken notification integration won't discard an otherwise successful crawl or prevent the JSON report from being saved.

5. Report Storage

By default, every completed crawl automatically saves a detailed JSON snapshot in var/link_checker/:

To disable automatic file storage, set storage_dir: null in your bundle configuration.

Reports are rotated per crawled URL: only the storage_max_reports most recent (30 by default) are kept for a given start URL, so a daily scheduled audit doesn't grow the storage directory forever. Set it to 0 to keep every report.

Notes

External link false positives

Some external sites reject automated HTTP clients outright (Akamai, Cloudflare, Sucuri, Incapsula, DataDome...), independently of what the resource actually contains. The bundle can't reliably bypass this — doing so would mean impersonating a browser to evade bot protection, which this bundle deliberately does not do. UrlChecker still flags this pattern when it recognizes a known bot-mitigation signature on a 403/429/503 response: the link is still reported as broken (the crawler genuinely couldn't fetch it), but each affected entry gets "likelyBlocked": true and a "blockedBy" provider name so you can distinguish "possibly just blocked" from "probably a real dead link" instead of treating every entry the same. The CLI table and summary line surface the same distinction. Domains you've manually verified as false positives can be silenced with exclude_patterns.

SSRF protection

The crawler follows every link it finds on the pages it visits — including links planted by whoever controls the content being audited. If you point it at untrusted or third-party content, a malicious page could contain a link to http://169.254.169.254/... (cloud instance metadata), http://localhost:6379 (an internal service), or any other address on your private network, and the bundle would otherwise dutifully request it from the machine running the crawl.

To prevent this, requests are made through Symfony's NoPrivateNetworkHttpClient, which blocks requests resolving (including via DNS) to private, loopback, or link-local IP ranges. This is on by default and applies to every HTTP request the bundle makes (link checks and page fetches alike).

Set allow_private_network: true only if you intentionally want to audit an internal network (e.g., a staging site reachable solely from behind a VPN) — and only when the content being crawled is fully trusted, since this also re-opens the SSRF exposure described above.

Being a polite crawler

Because check_external: true is the default, an audit routinely sends requests to sites you don't own or control. Two settings help keep that well-behaved:

• request_delay_ms (200 by default) enforces a minimum delay between consecutive requests to a host, across both link checks and page fetches. The host you're crawling is unthrottled against itself by default — it's the one site you actually control and want audited quickly — so this setting only ever slows down requests to other hosts, chiefly the external links it finds. Raise it if a crawl is likely to hammer one particular third-party domain with many links; set it to 0 to disable throttling everywhere, including external hosts, if you're confident that's fine for your use case.
• respect_robots_txt (on by default) fetches the crawled site's robots.txt once per host and stops the crawler from following or checking further internal links under a disallowed path. It doesn't affect the URL you explicitly pass as the crawl's starting point, and it doesn't apply to external links, which only ever get a single status check rather than being recursively crawled. If that same robots.txt publishes a Crawl-delay for our user agent, it overrides request_delay_ms for the audited host specifically — the site owner's explicit request takes precedence over the "unthrottled against itself" default. And like Google, if that robots.txt answers a server error (5xx, 429, or no response at all), the crawler treats the whole site as off-limits: it checks the starting URL and its links, but reads no further internal page, and the report is flagged blockedByRobotsTxt.

Query strings and crawl size

URLs are deduplicated as-is: /page?id=1 and /page?id=2 are treated as two distinct pages, since a query string often identifies genuinely different content (an article ID, a product SKU...) rather than noise — silently collapsing them could make the crawler skip a broken link instead of reporting it. On sites where some parameters are pure noise instead (pagination, sorting, tracking like utm_*), this can inflate the number of URLs crawled; use exclude_patterns to filter those out explicitly, e.g. '#[?&]utm_#' or '#\?page=#'.

Fragments aren't validated

A link's #fragment (e.g. /page#section) is stripped before checking: the bundle verifies that /page itself resolves, not that an element with id="section" (or name="section") actually exists on it. A link to a genuinely missing anchor is therefore reported as OK. Validating fragments would mean parsing the DOM of every linked page looking for a matching id/name, which the bundle doesn't do today.

Security

To report a vulnerability, please don't open a public issue — see SECURITY.md for how to report it privately.

License

MIT — see LICENSE.