Download the PHP package survos/classifier-bundle without Composer

survos/classifier-bundle

Assign controlled labels to text. One set of vocabularies, several interchangeable ways of assigning from them, and a benchmark that scores those ways against labeled data.

• A vocabulary says which labels exist: a flat list (US metro areas) or a tree (countries by region, IPTC Media Topics).
• A strategy says how labels get assigned: word rules, a vote over labels the text already has, or typed judgments from TypeSafe's Jev model.

Every assignment records the strategy that made it. Assignments made by an editor are never replaced or removed. A strategy that can't tell abstains instead of guessing.

Install

No configuration is needed for the rules strategies. For Jev, see Jev. For IPTC Media Topics, composer require survos/media-topics (or survos/media-topics-bundle, whose vocabulary instance is then shared).

Vocabularies

Add your own by registering a service that implements VocabularyInterface; it is tagged automatically. ArrayVocabulary::fromRows() builds one from plain rows, which is the easy way to expose an app's tag table:

Row keys: code, label, definition, parent, triggerTerms, properNouns, exceptions, regularExpressions.

Strategies

Rules

Terms match whole words. A term containing an uppercase letter is case-sensitive (NE matches "Omaha, NE", not "the northeast"); proper nouns always are.

Exceptions are phrases that contain a trigger without meaning it. They are blanked out of the text before matching, so with the trigger tobacco and the exception Alcohol, Tobacco and Firearms, a story about an ATF gun raid is not labeled tobacco, but an ATF story that also mentions smuggled tobacco is. In triggerTerms, a leading ! marks an exception.

Scores, and rules that settle it

A concept can carry structured rules beside its term lists (rules in ArrayVocabulary::fromRows(), or RuleSet::fromRules()). Matches add to a score; the concept is assigned when the total reaches its threshold (default 1).

Options per rule: score (default 1, may be negative), scope (headline, summary or body; default is what the strategy reads), case (default: case-sensitive iff the term has a capital), word (whole words, default true). meta rules test Document::$metadata, which is whatever the caller already knows about the item (host, source, marking, tag ids ...); a list value matches if it contains the value. An accept rule, in text or metadata, assigns the concept with Assignment::$decisive = true and stops evaluating it; reject withholds it. Decisive rules are checked before any score is added, wherever they appear in the list.

The older term syntax still works: term! is worth 100, term!! accepts outright, !term is an exception.

Jev (optional)

Set TYPESAFE_API_KEY, or point the bundle at a platform you already configured:

Two modes, chosen per vocabulary:

• labels (default for flat vocabularies, and for country): one yes/no question per concept, sent together. Every concept at or above threshold is assigned. A concept's definition is sent as the meaning of "yes", so write one.
• walk (default for trees): finds one label. Starting at the roots, each step asks which child fits, with "none of these" as an option, keeping the best beam paths. It stops at a parent when no child fits, and abstains when the best path is weak or barely ahead of a different branch.

The default threshold of 0.5 assigns far too many labels. Tune it on your own labeled data with the benchmark below; cached responses make rerunning at another threshold free.

Commands

All three take --format=json. The first two are also agent tools when survos/command-bundle is installed.

Benchmark

classifier:benchmark <file> <vocabulary> runs every strategy over labeled documents, one JSON object per line:

--concepts=tags.jsonl loads the vocabulary from a file of rows (the keys above, plus vocabulary) instead of the registry. Other options: --strategy, --limit, --per-concept, --price (US dollars per million input tokens, default 0.042).

Per strategy it reports true and false positives, false negatives, precision, recall and F1 over the documents the strategy answered; abstentions separately, with the gold labels they left unassigned; milliseconds per document; and for Jev, requests, input tokens and cost. For trees it also counts documents where the prediction is on the same branch as a gold label.

In code

Development

DESIGN.md has the design notes and the first measurements against a news archive.