Libraries tagged by tokeniser
devtronic/super-tokenizer
61 Downloads
A powerful dynamic tokenizer written in PHP
crazzy501/gpt3-tokenizer
6 Downloads
PHP package for Byte Pair Encoding (BPE) used by GPT-3.
chinleung/tokenizer
591 Downloads
PHP Tokenizer
brunocfalcao/laravel-tokenizer
97 Downloads
Laravel tokenizer - Token usage for web apps (create, burn, check, etc)
athron98/ihas-tokenizer
4 Downloads
iHAS API Token Generator
amsterdamstandard/tokenizer
17 Downloads
Tokenizer api wrapper for PHP
affinity4/tokenizer
3 Downloads
A zero-dedpendency tokenizer written in PHP. Returns an easily navigatable Stream object of Token objects with public type, value, offset and length properties
ae8/tokenizer
12 Downloads
A simple PHP library for token-based encryption.
mathewberry/tokenizer
768 Downloads
A very lightweight token system for micro-services
gossi/tokenizer
150 Downloads
Tokenizer for PHP code
fiv/tokenizer
244 Downloads
Extreme and flexible library for parsing and modify php code
cyberalien/simple-tokenizer
33 Downloads
Simple CSS tokenizer
yurunsoft/tiktoken
204 Downloads
PHP version of tiktoken
xtreed/php-token-stream
3522 Downloads
Wrapper around PHP's tokenizer extension.
vchagin/superclosure
65 Downloads
Serialize Closures, including their context