Download the PHP package gosuperscript/axiom-lookup without Composer

Axiom Lookup

A high-performance PHP library for querying CSV/TSV files with streaming, dynamic filtering, aggregate functions, and range-based banding — packaged as a first-class Axiom source.

Features

• Memory-Efficient Streaming: O(1) memory complexity with the default reader - processes records one-at-a-time
• Pluggable Reads: a host may inject a LookupSourceReader that answers indexed equality lookups from its own store, see Indexed reads
• Eight Aggregate Functions: first, last, min, max, count, sum, avg, all — enumerable at runtime, see Aggregates
• Explicit Filter API: ValueFilter and RangeFilter for clear, self-documenting code
• Range-Based Banding: Support for scenarios like tax brackets, premium tiers, shipping rates
• Dynamic Filter Resolution: Use nested lookups and symbols as filter values
• Dialect-Native Comparisons: Filter operators are compiled from the same composed Axiom dialect as ordinary infix expressions
• Typed CSV Boundaries: Declare column types when filters need coercion or non-string operations; undeclared columns remain raw strings
• Serialisable descriptions: a LookupSource is pure data — the filesystem lives on the LookupExtension, so a lookup tree can be persisted and loaded later
• Honest Types: numeric aggregates declare Option<Number>, all declares List<Unknown>, and aggregates returning one raw row/cell declare Option<Unknown>
• Early Exit Optimization: first aggregate stops reading after first match
• Flexible Storage: Support for local files, S3, and other storage backends via Flysystem
• PHP 8.4 Compatible: Full compatibility with latest PHP features

Installation

Quick Start

A LookupSource is pure, serialisable data — the file path, the filters, the columns, the aggregate. The filesystem the read needs is injected into a LookupExtension, which you compose onto the dialect; the source itself carries no live collaborator. Compile the source into a Program, then invoke it.

Aggregates

LookupSource::$aggregate is one of the names AggregateKind defines, and that enum is the only place the list lives. Ask it rather than restating the list:

requiresColumn() is the difference between the aggregates that read whole records and those that read one column's values. first, last, count and all count matching records or extract the requested columns from them, so they need no aggregateColumn. sum, avg, min and max need one — there is no sum of a whole record — and refuse without it:

So a caller validating a lookup before running it, or offering a column picker only where a column means something, reads both facts from the kind instead of keeping its own copy in step with this package. Given an aggregate state, $aggregate->kind() gets back to the same answers.

Using Different Storage Backends

The library uses Flysystem for filesystem abstraction, enabling you to read CSV files from various storage backends. The filesystem operator is passed to the LookupExtension, so you choose the right adapter once when you compose the dialect — every LookupSource compiled with it reads through that filesystem.

Local Filesystem

Amazon S3

Reusing a Program with Different Inputs

A filter value is a Source, so it can be a SymbolSource supplied at call time. Declare the symbol's type on the Expression; the compiled Program then admits it at the boundary and you invoke it with per-call bindings:

Indexed reads (the reader seam)

By default every lookup streams its whole file. A host that keeps an indexed copy of a lookup file — a database artifact, for instance — can answer equality lookups from it instead by injecting a LookupSourceReader:

The reader's contract: yield a superset of the records the filter pipeline would have matched in a full scan, in the file's original row order. Every yielded record still passes the full filter pipeline, so a reader may only change where the file is read — never a lookup's answer. A reader may fail eagerly or lazily (mid-iteration); both surface as the lookup's failure Result.

LookupSource accepts an optional index naming the column an indexed reader may seek on — purely an access-path hint, persisted with the source; declaring it never changes results. Each invocation hands the reader its probes: the resolved value of every == filter on a String-typed column, keyed by column. Skipping a probe is always safe; the filter pipeline still applies every filter.

Probes are byte-equality seeks. They are only sound where the dialect's String == String is raw byte equality (the core rule); a host whose dialect overloads that comparison must not serve probes from a byte-built index.

For observability the evaluation annotates scan with the strategy that answered (full-stream, or the indexed reader's own label), at most once, in a stable position before label.

Typed filters and operators

Filters are serialisable descriptions. During compilation, LookupExtension compiles each filter value and binds its operator from the expression's composed dialect. The resulting operation is reused for every row; filters do not contain a resolver and do not reimplement comparisons at runtime.

CSV cells are strings by default. Add a schema entry when a filter should read a cell as another Axiom type. For example, numeric ordering needs a numeric column declaration:

RangeFilter uses the same mechanism for its [minimum, maximum) test, so both bound columns should declare an orderable type:

Extension-owned operators work without lookup-specific integration. If an extension in the dialect owns equals-ignore-case for String × String → Boolean, a ValueFilter(..., 'equals-ignore-case') binds that exact rule. Unknown operators, incompatible operands, and operators that do not return Boolean are compile errors. A cell that cannot be coerced to its declared type is a runtime boundary error rather than a silent string comparison.

An all lookup is a total collection: no matching rows produce [], not absence. This makes a nested collection lookup usable as the right side of in after one explicit element-type bridge:

Other Storage Options

Flysystem supports many adapters including:

• FTP/SFTP
• Azure Blob Storage
• Google Cloud Storage
• In-memory filesystem
• And many more...

See the Flysystem documentation for more options.

Requirements

• PHP 8.4+
• gosuperscript/axiom (the typesafe compile/Program line)
• league/csv ^9.27.0
• league/flysystem ^3.0
• gosuperscript/monads

Testing

Benchmarking

Performance Characteristics

• Memory: with the default full-scan reader, constant usage regardless of file size (single-pass streaming); an injected indexed reader owns its own profile
• Early Exit: first aggregate stops after the first match
• Scalability: Linear time scaling with row count
• Validated: Comprehensive benchmarks with files up to 100k rows

License

Proprietary

Credits

Developed by GoSuperscript