Libraries tagged by tokenize
yurunsoft/tiktoken
239 Downloads
PHP version of tiktoken
xtreed/php-token-stream
3532 Downloads
Wrapper around PHP's tokenizer extension.
vchagin/superclosure
65 Downloads
Serialize Closures, including their context
td7650/superclosure
57 Downloads
Serialize Closure objects, including their context and binding
scheldejonas/phpunit-php-token-stream
206 Downloads
Wrapper around PHP's tokenizer extension.
octava/code-generator
1 Downloads
PHP code generator
nexxes/tokenmatcher
9 Downloads
PHP library to match tokenstreams (arrays of tokens) for later parsing.
muhfirdaus19/bahasata
6 Downloads
Text Processing PHP bahasa indonesia
megao-phpunit/php-token-stream
42 Downloads
Wrapper around PHP's tokenizer extension.
jamesfrost/statistical-classifier
2680 Downloads
A PHP implementation of Complement Naive Bayes and SVM statistical classifiers, including a structure for building other classifier, multiple data sources and multiple caching backends
html5/htmlreader
3299 Downloads
Html5 stream tokenizer/reader (not using libxml)
hike/hike
5 Downloads
A consistent interface to tokenize, analyse and modify any text format. Programming languages, natural languages, markup languages
guanhui07/tiktoken
6 Downloads
PHP version of tiktoken
gorriecoe/silverstripe-sreg
134 Downloads
Simple tokenizer that allows you to use .ss template like variables in a dataobjects string.
fourkitchens/tiktoken
0 Downloads
PHP version of tiktoken