Libraries tagged by tokenize

Another class loader using PHP tokenizer

Go to Download

PHP version of tiktoken. PHP 7.4+ required.

Go to Download

Lazy class discovery with composable filters and listener notification

Go to Download

Tokenizer-agnostic text preprocessor for trimming and optimising content for LLM context windows.

Go to Download

Turn everything into a unique encrypted JWT.

Go to Download

A collection of fast and easy to use attributed string classes for PHP. Attributed strings can have multiple attributes for each character of the string and are for example used for word processors and natural language processing.

Go to Download

Serialize Closure objects, including their context and binding

Go to Download

nikic/php-parser for HTML — an immutable HTML5 AST with positions, trivia, visitors, and a fidelity printer

Go to Download

A very lightweight PHP class that reformats unreadable and computer-generated SQL query statements to human-friendly, readable text.

Go to Download

Library emulating the PHP internal reflection using just the tokenized source code.

Go to Download

Doing interesting things with closures like serialization.

Go to Download

Turn everything into a token: randomized or predictable.

Go to Download

PHP_CodeSniffer tokenizes PHP, JavaScript and CSS files and detects violations of a defined set of coding standards.

Go to Download

Implements Iterator interface over a String for extract data with an array of tokens. The name comes from Mage and tokenizer :)

Go to Download

Fully integrated with Hyvä React Checkout Tokenized vault payments for faster checkout Admin capture, refund, and void support AVS/CVV fraud-prevention checks No Knockout or RequireJS needed Need checkout implementation help? Explore our Magento development services, Magento maintenance services, and Hyva theme performance services.

Go to Download