Download the PHP package semitexa/laravel-ai-verify without Composer

Laravel AI Verify: stop AI agents from faking "done"

AI code verification for Laravel. Claude Code, Cursor, Codex, Copilot or Gemini says the change is done. php artisan ai:verify checks whether it is.

It runs only the tests, Larastan, Pint and Laravel boot checks the diff actually needs. It fails the change if the agent deleted, skipped or weakened a test to get green. It returns one verdict the agent must act on: pass, fail or incomplete. One more command writes the instructions into AGENTS.md and CLAUDE.md. For Claude Code it also adds a Stop hook, so the agent cannot finish while verification fails. Verified commits get a trailer that CI can check.

Before the agent edits anything, php artisan ai:graph gives it a code graph of the Laravel app. It covers routes → controllers → form requests → models → tables → policies → views → tests, and events → listeners. It also does impact analysis: what a change affects and which tests cover it.

How it works: ai:verify reads what changed (uncommitted work, a git ref, or a file list). It works out the smallest set of checks that can vouch for that change and runs them with timeouts. The verdict comes as streamed NDJSON for agents, a JSON envelope or terminal text.

Note that this run did not execute the whole suite and did not run PHPStan on the whole app. It followed the change. The edited controller handles GET /posts/{post}, and tests/Feature/PostTest.php requests that URI, so that is the test that runs.

A real catch

While the demo app for this README was being written, the RSS view started with <?xml version="1.0"?>. That is valid XML, but Blade compiles it into PHP that doesn't parse. ai:verify flagged the view, the failing FeedTest, and the controller line that rendered it. The obvious "fix" was {!! '<?xml … ?>' !!}. It still failed, because the ?> inside the string closes PHP mode in the compiled template. The run went green only after the real fix ('<'.'?xml … ?'.'>'). Without the check, an agent would most likely have reported that first fix as done.

This package is a Laravel port of ai:verify from Semitexa, a PHP framework built so that AI agents can work on it safely. Semitexa describes its own structure and checks every change. This package brings the "checks every change" half of that to Laravel.

Why

Coding agents say "done" when the code looks right. Giving them a way to check that is harder than it sounds:

• The full test suite is slow, so agents skip it or run one test by hand.
• pest --dirty runs only the test files that changed, not the tests for the code that changed.
• Pest's --tia is precise but needs a coverage driver and a recorded baseline.
• PHPStan and Pint on a whole legacy app bury the agent in errors it didn't cause.
• Many Laravel failures never show up in a unit test. A closure in config/, a broken @forelse, a typo in a migration, or a provider that no longer boots all slip past them.

ai:verify handles all of this with one command and a fixed contract:

Install

Requires PHP 8.3+ and Laravel 12 or 13. Larastan and Pint are used when the project has them. Pest or PHPUnit is detected automatically.

Tell your agents

An agent only uses a tool it knows about. One command puts the instructions where each agent looks:

• Idempotent. Each file gets one <!-- ai-verify:start -->…<!-- ai-verify:end --> block, which is updated in place on re-run. The rest of the file is left alone, including Laravel Boost's <laravel-boost-guidelines> block. Boost rewrites only its own block, so the two coexist.
• More agents. --all also creates the files for agents the project doesn't use yet.
• Undo. --remove takes everything out again, and --dry-run shows the plan first.
• Commit the files. Every teammate's agent then gets the same instructions.

Why the hook. An agent can ignore an instruction, but it can't ignore a Stop hook. With --hook, every time Claude Code is about to finish, ai:verify checks the uncommitted work. If the verdict is fail, Claude is sent back with the violations and fixing commands instead of stopping. To keep it from getting in the way:

• It never blocks twice in a row (Claude Code's stop_hook_active).
• It doesn't re-run on a working tree it has already checked, so answering a question in a dirty repo costs nothing.
• AI_VERIFY_HOOK_SCOPE=minimal keeps it to a few seconds.

With Laravel Boost, you can instead tick semitexa/laravel-ai-verify in boost:install's third-party list, or add it to "packages" in boost.json. Boost then ships the same guideline and the ai-verify skill, and ai:verify:install steps aside to avoid duplicates. --hook still applies.

php artisan about shows whether the instructions and the hook are in place.

Usage

Exit code: 0 for pass/skipped, 1 for fail/incomplete.

Scopes

standard widens to broad on its own when a contract, a service provider, bootstrap/app.php or composer.json changes, or when 15 or more files change. Each widening is reported as an expansion event, with the reason.

What runs for what

Above max_test_targets (25) related test files, the whole suite runs once instead.

Output for agents

When stdout is not a terminal, which is how agents run commands, the output is NDJSON: one event per line, results streamed as each check finishes, and the verdict always last.

Event kinds, in order: summary, file, impact, expansion, warning, target, then result and its violation lines, then restart, next and verdict. A violation carries path, line, rule, message, severity and, when available, a tip with the command that fixes it.

--json prints one envelope (schema semitexa.laravel-ai-verify/v1) with the same data plus files, targets, results, violations, counts and unchecked_files. --human forces the terminal view.

Verdict rules

1. No checks ran → incomplete.
2. Any check failed → fail.
3. A required check could not finish (timeout, crash, tool missing) → incomplete.
4. Every check was skipped → skipped.
5. Otherwise → pass. If no content-reading check reached a result on any changed file (for example, only a README changed), pass is downgraded to incomplete.

Files that no check read are always listed in unchecked_files and in the headline.

Receipts: "verified" you can check

Each ai:verify run writes a receipt. A receipt records:

• the verdict;
• every check with its exit code and a hash of its output;
• a sha256 of every changed file, taken before the checks ran;
• any file that changed while the checks ran;
• the git tree id of the working state, which is exactly the tree a commit gets if this state is committed as is;
• a digest over all of the above.

The commit trailer

With php artisan ai:verify:install --git-hook, a prepare-commit-msg hook stamps every commit that a passing receipt covers exactly:

• Any agent, any person. It works for every agent and every person who commits. Git runs this hook even with --no-verify. core.hooksPath (Husky, lefthook) is respected.

• Exact match only. If the commit isn't exactly what was verified, for example a new file left unstaged or an edit after the run, there is no trailer. The hook says why on stderr:

• Amend after verifying. An amended commit that changed after verification loses its old trailer instead of carrying it.
• Stamp an existing commit. Run php artisan ai:verify --git-ref=HEAD~1, then git commit --amend --no-edit.

In CI

• Receipt check. --range checks every commit's trailer against that commit's own tree, with three outcomes: verified, stale (changed after verification) or missing. --require decides which commits must be verified: ai (commits co-authored by Claude, Codex, Copilot, Cursor, Gemini, …), all, or none.
• Re-verify. ai:verify --github re-runs the checks on the PR diff. Every violation becomes an annotation on the diff line, and the job summary gets the check table.

Message vs diff

A receipt proves the checks passed. It does not prove the change is the one the commit message describes. While we tested the GitHub Action, Claude Code committed Limit contact name to 250 chars in our test app. Its edit replaced max:255 with max:250, but the name limit was 200, so it changed the email rule. Tests passed, the trailer was earned, and the claim was false.

So the hook and --range also read each commit's subject against its diff:

• What it flags. A word in the subject names something in a changed code file, and that line is left alone while a line of the same shape within 12 lines of it changed. That is the wrong-line slip: 'name' => [...] untouched, its twin 'email' => [...] edited. It also flags a number in the subject that no added line contains, such as 200 when the diff writes 250.
• What it ignores. Words the changed code never mentions; prose, JSON, YAML, Twig and HTML files; tests; merges, reverts, and commits touching more than 25 files. PHP files, Blade views included, are read. Across 937 commits from five of our PHP repositories it flagged 23 (2.5%), one of them the commit above.
• Where it shows. When the message comes with git commit -m or -F, as agents commit, the git hook prints it on stderr before the commit exists, so the agent can fix it right away. A message typed in the editor is checked in CI. In CI each mismatch becomes a warning annotation on the changed line and a "Message vs diff" list in the job summary.
• How strict. --claims=warn (default) only reports. --claims=fail fails --range. --claims=off turns it off.

GitHub Action

This repository is also a GitHub Action that runs both checks on every pull request:

Outputs: receipts (ok / failed / skipped) and verdict (pass / fail / incomplete / error / not-run). Each violation becomes an annotation on the PR diff, and the job summary gets one table with both checks.

What a receipt is and isn't. The digest catches an edited receipt, not a forged one: whoever can write storage/ can rewrite both. What a receipt adds is that "verified" names something specific. It can be checked against a commit, inspected, and re-run, and CI re-runs it anyway.

ai:graph: a code graph of your Laravel app, for agents

The same graph that selects tests is available directly:

The graph has two layers:

• Static. A tokenizer reads PHP and Blade. It never runs project code, so it works even when the app is half broken. It records which classes reference each other, extends/implements, views rendered (view('posts.show')), @include/@extends/<x-component>, tables created by migrations, model → factory, and tests → the classes they use.
• Runtime. It reads facts from the booted app: routes → controller@method, FormRequests, middleware; event → listeners; model → table, relations, policy. Tests are linked to the routes they request, by literal URI or route name.

Edges point from the thing that depends to the thing it depends on, so "what does this change affect" is a walk backwards along them.

Safety

• Nothing is written to bootstrap/cache. Config, route and event cache paths are redirected to temp files.
• Child processes don't inherit .env. Laravel copies .env into the process environment, and phpunit.xml <env> entries don't override variables that already exist. Without this step, tests started from inside Artisan would run against your .env database. Variables you set in the shell or in CI with a different value are kept.
• Migrations run only in pretend mode. The SQL is compiled, nothing is executed. If schema introspection needs a live database, the check is skipped, not failed.
• Every check has a 120 s timeout and a 4 MB output cap. Hitting either makes the result incomplete, never pass.
• Agent output wrappers are bypassed. laravel/pao is switched off for child processes. Results are read from JUnit XML and the tools' JSON formats, never from console text.

Configuration

How it relates to other tools

FAQ

How do I stop Claude Code (or Cursor, Codex) from saying "done" when the tests fail?

Rules in CLAUDE.md or AGENTS.md help, but agents don't always follow them. Run php artisan ai:verify:install --hook. The rules go into the instruction files, and Claude Code gets a Stop hook: when the agent tries to finish, ai:verify checks the uncommitted work, and on fail Claude is sent back with the violations. Other agents get the same rules and the same command to run.

How do I stop an AI agent from deleting, skipping or weakening tests to make them pass?

Agents sometimes make a red suite green by editing the tests instead of the code. Researchers call this "reward hacking". Every changed test file is compared with the base ref. The run fails when it loses tests or assertions, or gains skip(), todo(), only() or markTestSkipped. Commenting out an assertion counts as removing it. If a test change is genuinely intended, the agent has to say so explicitly with // verify:accept-test-change <reason>, which shows up in review.

What should go in AGENTS.md / CLAUDE.md for a Laravel project?

At minimum, the agent needs to know how to verify its work and what not to do to pass. See the guideline this package installs. Run php artisan ai:verify:install and it is written into AGENTS.md and CLAUDE.md, and into GEMINI.md, Copilot, Junie, Cursor and Windsurf files if you use them, as an idempotent block next to your own rules and Laravel Boost's.

How do I run only the tests affected by a change in Laravel?

php artisan ai:verify --git-ref=main (or php artisan ai:graph --impact=app/Models/Post.php to just list them). Tests are selected through a project graph: tests that reference a changed class, request a route its controller handles, render a view it touches, or use its factory. It needs no coverage driver and no recorded baseline, unlike coverage-based test impact analysis.

How do I give an AI agent a map of my Laravel codebase?

php artisan ai:graph builds a code graph from your app. It reads the code statically, and it asks the booted app for routes, listeners, model tables and policies. Agents call it with --json (the default when output isn't a terminal):

• ai:graph posts.show: the whole route chain: controller@method, FormRequest, middleware, views, models with their tables and policies, and the tests that hit it.
• ai:graph "App\Models\Post": what the class uses, what uses it, and which tests reach it.
• ai:graph: every route and its action, the most depended-on classes, and code that no test reaches.

It gives the agent the same picture a senior Laravel developer has in their head, so it doesn't have to grep and guess, especially around implicit wiring like route-model binding, event listeners and policies.

What will this change break? (impact analysis for Laravel)

php artisan ai:graph --impact=app/Models/Post.php walks the graph backwards from the file. It reports dependent classes, affected routes and the tests that cover them, with a low/medium/high blast-radius band. ai:verify --impact adds the same report to a verification run, and ai:verify uses the graph anyway to pick which tests to run.

Does it replace AI code review?

No. It checks things a reviewer shouldn't have to: the code parses, the views compile, the app boots, migrations run, static analysis and style are clean on the changed lines, the related tests pass, and the tests weren't tampered with. Review time then goes on design and intent, and "it doesn't even run" is off the table.

How do I catch an AI commit that says one thing and changes another?

Install the git hook (php artisan ai:verify:install --git-hook). When a commit subject names a field or method whose line stayed the same while a line just like it changed, the hook warns the agent before the commit exists, and CI annotates the line. See Message vs diff.

Larastan and Pint fail on my legacy code. Will the agent drown in errors?

No. Only problems on the lines this change touched fail the run. PHPStan errors elsewhere in the touched files, and Pint fixers the file already needed, are reported as pre-existing warnings.

Does it work with Laravel Boost, Pest 4/5, PHPUnit, Octane, Horizon?

Yes. It ships a Boost guideline and skill. It runs Pest or PHPUnit and reads their JUnit output. After a change it tells the agent when to run queue:restart, horizon:terminate or octane:reload.

Where it comes from

Semitexa is a modular PHP framework with a Swoole-first runtime, designed so that an AI coding agent can work on a codebase safely. It has a project graph to ask "what does this change affect" instead of grepping, and ai:verify, which checks every change and records what it checked.

This package ports that verification loop to Laravel: the same change classification, scoped planning, coverage gap, test-integrity guard and verdict contract, mapped onto Laravel's conventions and tooling. If you like this way of working, have a look at how far it goes when a framework is designed around it: semitexa.com.

Contributing

Issues and pull requests are welcome at github.com/semitexa/laravel-ai-verify.

License

MIT. See LICENSE.