Libraries tagged by tokenizer
golding/dns-zone-tokenizer
18 Downloads
tokenize dns zone files and that's all, folks.
dhii/xml-tokenizer-interface
8 Downloads
An interface for XML tokenizers
dhii/tokenizer-interface
166 Downloads
Interfaces for tokenizer interoperability.
dhii/tokenizer-base
8 Downloads
Base concrete functionality for tokenizers
dhii/tokenizer-abstract
44 Downloads
Abstract common functionality for tokenizers.
brunocfalcao/laravel-tokenizer
97 Downloads
Laravel tokenizer - Token usage for web apps (create, burn, check, etc)
athron98/ihas-tokenizer
4 Downloads
iHAS API Token Generator
devtronic/super-tokenizer
70 Downloads
A powerful dynamic tokenizer written in PHP
cyberalien/simple-tokenizer
38 Downloads
Simple CSS tokenizer
xtreed/php-token-stream
3559 Downloads
Wrapper around PHP's tokenizer extension.
vchagin/superclosure
65 Downloads
Serialize Closures, including their context
td7650/superclosure
58 Downloads
Serialize Closure objects, including their context and binding
taufik-nurrohman/minify
11 Downloads
Reduce CSS, HTML, JavaScript, JSON, PHP, and XML code size on the server side.
scheldejonas/phpunit-php-token-stream
208 Downloads
Wrapper around PHP's tokenizer extension.
purewater2011/tiktoken-php7
3 Downloads
PHP 7.4+ compatible version of tiktoken - OpenAI's tiktoken tokenizer ported to PHP