Download the PHP package rafiquljakir/fynoverify without Composer

fynoVerify: Developer Documentation

Package: rafiquljakir/fynoverify · Namespace: Rafiquljakir\FynoVerify · Repository: https://github.com/Rafiqul-jakir/fynoverify · License: MIT

Status: early release (0.x). Commands, options and report fields may still change before 1.0. Declared support: PHP ^8.2, Laravel 11 and 12.

Contents

1. What fynoVerify is
2. Installation
3. Quick start
4. Core concepts
5. Command reference
6. Configuration reference
7. Report schemas
8. Exit codes and CI
9. Output path policy
10. Security and privacy
11. Architecture
12. Extending the package
13. Testing the package
14. Releasing
15. Troubleshooting and FAQ
16. Limitations and roadmap

1. What fynoVerify is

fynoVerify is a Laravel development and CI tool for API and authorization regression testing. It does three jobs:

1. Inventory the routes your application registers (fyno:scan).
2. Audit declared test evidence: check that each in-scope route is mapped to an existing test file or has a written exemption (fyno:coverage).
3. Suggest tests for each in-scope route, based only on route metadata and your configuration (fyno:plan).

A fourth command, fyno:doctor, diagnoses installation and configuration problems.

What it does not do

2. Installation

Requirements: PHP 8.2+, Composer 2, Laravel 11 or 12.

Verify discovery

The service provider is auto-discovered through extra.laravel.providers. Confirm:

You should see fyno:scan, fyno:coverage, fyno:plan and fyno:doctor. Then run php artisan fyno:doctor.

Publish the config (optional)

This copies config/fynoVerify.php into your app. Without it the package works on defaults, and a partially published config is fine because omitted keys fall back to defaults.

Because the package is a require-dev dependency, the commands are absent after composer install --no-dev.

To work on the package itself, clone the repository and see section 13.

3. Quick start

Example mapping:

The IDs shown are examples. Use the IDs your own application reports. The packaged default config contains no fictional mappings.

4. Core concepts

4.1 Route IDs

Every command identifies a route the same way, and your config keys must match.

Notes:

• METHODS is the full sorted method list Laravel registers, so a GET route is GET|HEAD.
• Unnamed IDs change when the method or URI changes. Naming routes keeps your mappings stable.
• If two routes share an ID, every affected in-scope route becomes configuration-error, and fyno:scan prints a warning. Give each route a unique ->name().

4.2 URI normalization and patterns

URIs are normalized before matching: no leading or trailing slash, and the root route is /.

Patterns are globs. * matches any characters, including /. Everything else is literal and matching is case-sensitive.

A leading slash in a pattern is normalized away (/api/* equals api/*). Patterns must be non-empty strings with no control characters.

4.3 Scope

For fyno:coverage and fyno:plan, a route is in scope when it matches at least one include_uri_patterns entry and no exclude_uri_patterns entry. Include runs first, then exclude removes routes from that set.

Routes outside the scope are not audited, but coverage reports list their IDs under scope.not_in_scope_route_ids, so the effective scope is always visible.

4.4 Coverage statuses

Evaluation order for each in-scope route: ambiguous ID, then declaration problem, then covered, then ignored, then uncovered.

Stale declarations are IDs in covered_routes or ignored_routes that match no registered route. They are reported separately from route statuses (stale_declarations). A declaration for a route that exists but is outside the scope is not stale; it produces a warning and has no effect.

4.5 Declared test-evidence coverage

covered means "you pointed this route at a file that exists". It does not mean the file contains relevant assertions, that tests pass, that the route is secure, or that branches are covered. Every coverage report carries this disclaimer.

5. Command reference

All commands return a Laravel exit code (see section 8). Machine-readable output (json, markdown) written to stdout is never mixed with decorations. Error and diagnostic messages go to stderr.

5.1 fyno:scan

Read-only route inventory.

Examples:

Route record fields

Behavior:

• Sorted by URI, then methods, then domain, then ID, using byte-wise comparison, so output is identical between runs.
• Handles the root route, unnamed routes, optional parameters, constraints, HEAD/OPTIONS, domain routes and routes without middleware.
• An empty result prints "No routes matched" (table) or an empty routes list, and exits 0.
• Table output truncates long values. JSON and Markdown never truncate.

5.2 fyno:coverage

Audits declared test evidence for in-scope routes.

Flow:

1. Validate configuration. Structural errors produce a report with configuration_errors and exit 2.
2. Inspect routes and partition them into in-scope and not-in-scope.
3. Classify each in-scope route, check stale declarations, and compute failure reasons.
4. Render. A report is always produced, even for a failed audit, and --output still writes it.

The audit fails with exit 1 when any of these hold:

• an in-scope route is uncovered, missing-evidence or configuration-error;
• stale declarations exist and fail_on_stale_declarations is true;
• the scope matched no routes and fail_on_empty_scope is true.

Evidence files are only checked for existence, readability and being a regular file inside the project root. They are never opened for their contents and never executed.

5.3 fyno:plan

Generates reviewable test suggestions for in-scope routes.

Invalid configuration prints each error to stderr and exits 2.

Each scenario is always status: "suggested", heuristic: true, requires_human_review: true, and lists what your team must supply (requires_from_team). Nothing is ever labelled passed, failed or vulnerable. Every plan carries: "These are metadata-based test suggestions, not executed tests or a security verdict."

Signals derived from a route:

Scenario catalog (output order is by priority, then this order):

Suggestions are de-duplicated per category|title, filtered by plan.enabled_suggestions, ordered, and capped at plan.max_scenarios_per_route. plan.max_routes limits the number of routes; omitted routes are reported in summary.routes_omitted and warnings.

5.4 fyno:doctor

Read-only diagnostics.

Each check has a status (pass, warning, fail), a message, and an optional recommendation. The command exits 1 only if a check fails. An exception inside a check becomes a failing check rather than a crash. It never prints environment values, credentials or connection strings, and it never modifies files. Evidence existence is checked by fyno:coverage, not by doctor.

6. Configuration reference

File: config/fynoVerify.php. Config key: fynoVerify. Publish tag: fynoVerify-config. Every key is optional and validated at runtime; messages name the exact key.

coverage

plan

Validation levels

Pitfalls

• Dotted route IDs and config()->set(). Setting config()->set('fynoVerify.coverage.covered_routes.api.users.index', ...) creates nested arrays because dots are path separators. Assign the whole map instead, as a PHP array literal in the config file.
• Numeric-looking keys. PHP turns the key '123' into an integer. Integer keys in the maps are reported as errors ("found list entry").
• Merging. The provider uses Laravel's shallow mergeConfigFrom. The validator applies defaults per key, so a partial coverage array does not lose other defaults.

Evidence path rules

• Must be project-relative: no leading / or \, no drive letter such as C:.
• No .. segments. Both / and \ separators are accepted.
• Must resolve to an existing, readable, regular file.
• The real path (after resolving symlinks) must stay inside the project root. A symlink pointing outside is rejected; a symlink pointing inside is accepted.

7. Report schemas

JSON reports use UTF-8, unescaped slashes and a trailing newline. Canonical output has no timestamps, so identical inputs give identical output and diffs are meaningful.

schema_version is "1.0". Adding optional fields keeps the version. Renaming, removing or retyping fields bumps it.

Common fields: tool (fynoVerify), schema_version, report_type (scan, coverage, plan, doctor), disclaimer (all except doctor), warnings (scan, coverage, plan).

7.1 Scan

filter is null when no filter was given.

7.2 Coverage

• declared_as is covered, ignored or covered+ignored.
• evidence is set for covered and missing-evidence; reason for ignored; message explains non-passing statuses.
• With invalid configuration: scope and summary are null, routes is empty, exit_code is 2, passed is false, and configuration_errors lists each problem.

7.3 Plan

7.4 Doctor

7.5 Markdown output

Scan, coverage and plan can be rendered as Markdown. All route metadata is escaped: \ | * _ [ ]are backslash-escaped,< > &` become HTML entities, and control characters (including newlines and ANSI escapes) collapse to spaces. A hostile route name cannot add table rows, inject links or emit raw HTML.

7.6 Parsing example

8. Exit codes and CI

With --format=json and no --output, stdout contains only the JSON document, so pipelines can parse it safely. Errors go to stderr. Passing -v adds a stack trace for internal errors.

GitHub Actions

Run the audit and your tests as separate steps. A passing audit never implies tests ran.

if: always() uploads the report even when the audit fails. Gate merges on the exit code: 1 means fix declarations, 2 means fix configuration.

9. Output path policy

--output is the only way fynoVerify writes a file. Rules enforced by ReportWriter:

Every violation exits 2 with a message naming the problem. A good default location is storage/app/fynoverify/<name>.json. The protected-directory list is a safeguard, not a security boundary.

10. Security and privacy

• No network requests, telemetry, source upload or AI calls. A unit test (NoNetworkOrExecutionTest) scans src/ for network and process-execution primitives, eval, and similar calls.
• No controller execution, HTTP probes, migrations or database access.
• Test files are only checked for existence and readability, never read or executed (a feature test plants a sentinel file to prove this).
• Route names, URIs, actions, middleware labels, config strings and paths are treated as untrusted and escaped for the terminal, Markdown and JSON.
• fyno:doctor prints only versions, counts and project-relative paths. A feature test confirms environment secrets and app.key never appear.
• Generated reports describe your API surface. Keep them internal and out of public artifacts.
• fynoVerify offers no security guarantee. Use wording such as "potential risk signal", "suggested scenario" and "requires application-specific review".

Report suspected vulnerabilities privately through GitHub's Security tab ("Report a vulnerability") rather than a public issue.

11. Architecture

11.1 Layout

11.2 Data flow

Reports are plain arrays built in services. Commands only parse options, call services, render and map results to exit codes.

11.3 Design principles

• Thin commands, testable services. Commands receive services through method injection on handle(). Core logic runs without an HTTP request.
• Public APIs only. RouteInspector uses Router::getRoutes() and the public Route methods. No reflection on framework internals.
• Deterministic output. Byte-wise sorting, no timestamps, JSON_THROW_ON_ERROR, and JSON_INVALID_UTF8_SUBSTITUTE so bad bytes do not abort a report.
• Fail loudly. BaseCommand::guard() converts any Throwable into a stderr message and exit 3. The audit is never reported as passed after an exception.
• Untrusted input. All strings from routes and config pass through Text before reaching a terminal or Markdown.
• Raw writes. Documents are written with OutputInterface::OUTPUT_RAW so Symfony's formatter cannot interpret <tag>-like text inside JSON or Markdown.

11.4 Key classes

12. Extending the package

12.1 Add a plan scenario category

1. Add the category name to TestPlanBuilder::CATEGORIES. The validator reads this constant, so plan.enabled_suggestions accepts it automatically.
2. In TestPlanBuilder::scenariosFor(), add a candidate via $this->scenario(category, title, rationale, priority, priorityRule, basis, requires). Base it only on route metadata or explicit config.
3. Add the category to the default enabled_suggestions list in config/fynoVerify.php.
4. Add unit tests in TestPlanBuilderTest (when it appears, when it does not, its priority) and update the tables in this file.

Rules to keep: never use verdict language ("vulnerable", "passed", "failed"); always set heuristic and requires_human_review; say what the team must supply.

12.2 Add a config key

1. Add a default in config/fynoVerify.php with a comment.
2. Add the key to COVERAGE_KEYS or PLAN_KEYS in ConfigurationValidator (so it is not flagged unknown), validate it with the existing helpers, and add a property to Settings.
3. Pass it where needed and test both valid and invalid values.

12.3 Add an output format

Add a renderer method to ReportRenderer, allow the format in the commands' formatOption([...]) list, and extend the extension rules in ReportWriter::write(). Escape every route-derived string through Text.

12.4 Add a command

Create a class extending BaseCommand with a $signature (fyno:...), a handle() that wraps its body in $this->guard(...), use failWith() for user-correctable errors, and register it in FynoVerifyServiceProvider::boot(). Keep logic in services.

12.5 Future test helpers

The source is organized so a Testing namespace can be added without touching the CLI. Any helper must take the principal, factory, request and expected status explicitly, must not assume an App\Models\User or a role system, and must never produce tests that pass trivially.

13. Testing the package

Layout

• tests/Unit: pure logic (pattern matching, IDs, path safety, report writer, text escaping, validator, auditor, plan builder, renderers, and the no-network guard).
• tests/Feature: real Laravel apps via Orchestra Testbench, running the commands end to end.

Test infrastructure

tests/TestCase.php creates a throwaway project root under the system temp directory for each test and calls $this->app->setBasePath(), so evidence checks and report writes never touch real files. It also sets include_uri_patterns to ['fvt/*'], and the feature tests register their routes under that prefix to stay clear of routes the Testbench skeleton registers.

Under Testbench, stderr and stdout share one buffer, so error text appears in Artisan::output(). In real use they are separate streams.

What the suite covers

• Provider discovery, command registration, config defaults without publishing, and the publish tag.
• Scan: valid JSON, determinism, ordering, filtering, edge-case routes, Markdown escaping, invalid options, safe and unsafe output paths, and read-only behavior.
• Coverage: every status, stale declarations, empty scope, include/exclude order, traversal and out-of-root evidence, invalid config (exit 2 with a report), duplicate IDs, report files on failed audits, wording, and no execution of evidence files.
• Plan: coverage of in-scope routes, category rules and priorities, suggestion-only labels, no verdict words, caps and limits.
• Doctor: healthy and failing environments, secret redaction, and no mutation.

Known gaps

There is no Windows path test and no end-to-end test that installs the package into a fresh Laravel application. The CI workflow (.github/workflows/tests.yml) runs a PHP 8.2 to 8.4 and Laravel 11 to 12 matrix.

14. Releasing

1. Run all checks locally (section 13) and make sure CI is green.
2. Update CHANGELOG.md (only features that exist and are tested).
3. Do not add a version field to composer.json; Packagist reads Git tags.
4. Tag and push: git tag -a v0.1.0 -m "v0.1.0" && git push origin v0.1.0.
5. Packagist picks up the tag automatically through the GitHub webhook. If the webhook is not enabled, open the package page and click Update.

Follow semantic versioning. Changing the report schema_version, command signatures, exit codes, config keys, the route ID scheme, the vendor name or the namespace is a breaking change.

15. Troubleshooting and FAQ

*The `fyno:commands are not listed.** Runcomposer dump-autoload, confirm the package is inrequire-dev, and that you did not install with--no-dev`.

fyno:coverage exits 2. The configuration is invalid. The message names the key, for example coverage.include_uri_patterns: expected a list of strings, got string. fyno:doctor shows the same errors.

A route shows missing-evidence. Read the route's message: the file does not exist, the path is absolute or contains .., or it resolves outside the project. Use a path relative to the project root.

Everything is reported as a stale declaration. Route IDs changed. Re-run fyno:scan and copy the current IDs. Unnamed route IDs change with their method or URI.

Empty scope failure. Your include patterns match nothing. Compare them with the uri column of fyno:scan. URIs have no leading slash, and matching is case-sensitive.

Two routes share an ID. Add a unique ->name() to each. Until then both are configuration-error.

Output file refused. Choose a path such as storage/app/fynoverify/report.json with the right extension, and add --force to replace an existing file.

Does a covered route mean it is tested? No. It means a file you named exists. Run your test suite to learn whether tests pass.

Why does fyno:plan suggest tests for routes that have auth middleware? A middleware label is not proof of correct behavior. The suggestions ask you to verify guest, authorization and ownership behavior explicitly.

Why is a route with no visible middleware not flagged as public? Global and custom middleware are invisible to route metadata, so it gets an access-intent suggestion ("confirm the intended policy") instead of a claim.

Can I commit the JSON reports? You can, but treat them as internal documentation because they describe your API surface.

Which Laravel versions are supported? Laravel 11 and 12 on PHP 8.2 or newer, as declared in composer.json and exercised by the CI matrix.

16. Limitations and roadmap

Current limitations

• Early release: expect changes before 1.0.
• Middleware is shown as declared on the route. Global middleware and unexpanded groups (api, web) are invisible, so authentication signals are heuristics.
• Plan suggestions cannot know your roles, tenants, ownership rules or response contracts, and they list what you need to supply.
• Declared evidence is not proof of meaningful tests.
• Absolute --output paths are not supported.
• No timestamps in canonical output, by design.

Not implemented (ideas only)

• A PHPUnit/Pest helper API for access-control regression tests.
• Human-reviewed test scaffolding with prominent TODOs that never overwrites existing tests.
• Richer static signals for authentication and authorization (still not a verdict).
• Optional, opt-in, clearly labelled LLM-assisted suggestions, disabled by default, with data minimization.
• Other frameworks, and hosted team reporting, if there is demand.

None of these exist in the current code.