Libraries tagged by tokenzie
affinity4/tokenizer
7 Downloads
A zero-dedpendency tokenizer written in PHP. Returns an easily navigatable Stream object of Token objects with public type, value, offset and length properties
ae8/tokenizer
16 Downloads
A simple PHP library for token-based encryption.
mathewberry/tokenizer
770 Downloads
A very lightweight token system for micro-services
gossi/tokenizer
150 Downloads
Tokenizer for PHP code
fiv/tokenizer
254 Downloads
Extreme and flexible library for parsing and modify php code
cyberalien/simple-tokenizer
38 Downloads
Simple CSS tokenizer
yurunsoft/tiktoken
242 Downloads
PHP version of tiktoken
vchagin/superclosure
65 Downloads
Serialize Closures, including their context
td7650/superclosure
58 Downloads
Serialize Closure objects, including their context and binding
scheldejonas/phpunit-php-token-stream
208 Downloads
Wrapper around PHP's tokenizer extension.
purewater2011/tiktoken-php7
3 Downloads
PHP 7.4+ compatible version of tiktoken - OpenAI's tiktoken tokenizer ported to PHP
octava/code-generator
1 Downloads
PHP code generator
nicxon/temp-download-links
1 Downloads
Laravel package to generate temporary download links with expiry time and max download limits.
nexxes/tokenmatcher
9 Downloads
PHP library to match tokenstreams (arrays of tokens) for later parsing.
muhfirdaus19/bahasata
6 Downloads
Text Processing PHP bahasa indonesia