Download the PHP package alies-dev/psalm-tester without Composer

Psalm Tester

Regression tests for what Psalm reports, written as .phpt files and run by PHPUnit. Each file holds a PHP snippet and the exact issues Psalm must report for it, type assertions included. It is built for authors of Psalm plugins and stubs, who need to pin inferred types and issues without hand-writing a PHPUnit test and a Psalm invocation per case.

Changes: UPGRADING.md.

Quick start

Requires PHP 8.2+ and PHPUnit 11, 12 or 13; installs Psalm 6.10+ or 7.

A fixture, tests/Psalm/phpt/array_values.phpt:

The empty --EXPECT-- means "Psalm reports no issues", so the test passes exactly when the inferred type matches.

A test case, tests/Psalm/PsalmTest.php:

Every *.phpt file under phptDirectory() (recursively) becomes one data set of testPhpt, named by its path relative to that directory. Only the selected data sets are analyzed, so --filter also makes the Psalm run cheaper:

At the start of the batch, one STDERR line sums it up, e.g. psalm-tester: 3 phpt files (0 skipped), 2 Psalm runs.

Do not pass a directory holding fixtures on the command line (vendor/bin/phpunit tests/Psalm): PHPUnit then also runs every .phpt file as its own PHPT test, executing the code instead of analyzing it. A <directory> in phpunit.xml is safe, since it collects only *Test.php by default.

Had the tag said list<int>, the test would fail with a summary (type, missing and unexpected issues) above PHPUnit's diff:

Writing fixtures

The expectation is one <IssueType> on line <n>: <message> line per issue, sorted by line and column, with line numbers counted from the top of the .phpt file. The default config (src/psalm.xml) is strict (errorLevel="1", findUnusedCode, ...), hence $_list: an unread $list would add an UnusedVariable issue.

Writing type assertions

@psalm-check-type-exact compares types semantically: 1|2 equals 2|1, but list<int> differs from non-empty-list<int>. Unlike a traced type, it does not depend on how a Psalm version prints types.

• Put the tag on its own line right after the statement that sets the variable, at top level or inside a function body. On a class or method docblock it is silently ignored.
• The variable must exist at that point, or Psalm reports InvalidDocblock.
• To find the type, add /** @psalm-trace $x */ temporarily, copy the traced type into the tag, remove the trace.
• Plain @psalm-check-type only checks that the actual type is contained in the given one.

Other expectations

Leave out what should not be pinned, such as a line number that shifts when the fixture is edited:

Skip on an environment condition, pass extra Psalm arguments, and document a known wrong result:

Fixtures with the same arguments share one Psalm run and symbol table, so keep class and function names unique, or give a fixture its own run with --CONFLICTS--.

Supported phpt sections

The format comes from php-src (phpt file layout, writing tests, run-tests.php). psalm-tester supports a subset, with Psalm semantics:

Any other section throws "Unknown section", and a repeated one "Duplicate section"; either errors only that test.

--ARGS-- is split into words like a shell would (quotes and backslashes work, nothing is expanded) and appended to the tester's arguments. A config option in it (--config=x, --config x, -c x) replaces the configured one. The tester passes the files itself, so -f and paths are rejected.

Configuring the tester

Override tester() in the test case (importing AliesDev\PsalmTester\PsalmTester). Every with*() method returns a configured copy:

A plugin's tests/Psalm/psalm.xml needs no <projectFiles>:

Using PsalmTester directly

PsalmPhptTestCase is a thin layer over PsalmTester::run(), which takes an iterable of Phpt and returns a Result per key. runOne() runs a single one:

Outcome is Passed, Failed, Skipped, XFailed, XPassed or Error; $result->reason explains the last four, $result->issues holds each issue's type, line, column and message, and $result->assert() reports the result to PHPUnit. Phpt::fromFile() loads a fixture.

How tests run

• SKIPIF scripts run first, concurrently. The rest is analyzed with one Psalm run per distinct argument set, so an expensive plugin boot is paid once per set; up to withConcurrency() runs go at once. Flag order does not split a set, unless an option takes its value as a separate word (--config x, -c x, --root x, -r x, --printer x).
• Each run gets --no-cache and its own XDG_CACHE_HOME and temp directory, but an explicit <cacheDirectory> in psalm.xml takes precedence: Config::getGlobalCacheDirectory() then returns the same path in every concurrent run, so a plugin writing there must handle concurrent writers.
• A run that crashes, exits with a status other than 0 or 2, prints something other than Psalm's JSON issue list, reports issues in other files, or times out gives Outcome::Error to each of its tests. It never passes.