Libraries tagged by tokenize
scheldejonas/phpunit-php-token-stream
208 Downloads
Wrapper around PHP's tokenizer extension.
rayhan-bapari/bkash-payment
6 Downloads
A production-ready Laravel package for bKash Tokenized Checkout Payment Gateway — with token caching, webhook/IPN support, refund, search, and more. Supports Laravel 9–12 and PHP 8.1–8.5.
purewater2011/tiktoken-php7
3 Downloads
PHP 7.4+ compatible version of tiktoken - OpenAI's tiktoken tokenizer ported to PHP
octava/code-generator
1 Downloads
PHP code generator
nicxon/temp-download-links
2 Downloads
Laravel package to generate temporary download links with expiry time and max download limits.
nexxes/tokenmatcher
9 Downloads
PHP library to match tokenstreams (arrays of tokens) for later parsing.
muhfirdaus19/bahasata
6 Downloads
Text Processing PHP bahasa indonesia
megao-phpunit/php-token-stream
45 Downloads
Wrapper around PHP's tokenizer extension.
legacyphpdoctor/yethee_tiktoken-php-7-4
1 Downloads
PHP version of tiktoken
jamesfrost/statistical-classifier
2680 Downloads
A PHP implementation of Complement Naive Bayes and SVM statistical classifiers, including a structure for building other classifier, multiple data sources and multiple caching backends
html5/htmlreader
3306 Downloads
Html5 stream tokenizer/reader (not using libxml)
hike/hike
5 Downloads
A consistent interface to tokenize, analyse and modify any text format. Programming languages, natural languages, markup languages
guanhui07/tiktoken
8 Downloads
PHP version of tiktoken
gorriecoe/silverstripe-sreg
137 Downloads
Simple tokenizer that allows you to use .ss template like variables in a dataobjects string.
fourkitchens/tiktoken
0 Downloads
PHP version of tiktoken