Libraries tagged by tokenized
hike/hike
5 Downloads
A consistent interface to tokenize, analyse and modify any text format. Programming languages, natural languages, markup languages
guanhui07/tiktoken
6 Downloads
PHP version of tiktoken
gorriecoe/silverstripe-sreg
134 Downloads
Simple tokenizer that allows you to use .ss template like variables in a dataobjects string.
fourkitchens/tiktoken
0 Downloads
PHP version of tiktoken
djiele/autoloader
11 Downloads
Another class loader using PHP tokenizer
crazzy501/gpt-3-encoder-php
379 Downloads
PHP BPE Text Encoder/Decoder for GPT-2 / GPT-3. Based on coderevolutionplugins/gpt-3-encoder-php
attla/token
3 Downloads
Turn everything into a unique encrypted JWT.
antto1/superclosure
160 Downloads
Serialize Closure objects, including their context and binding
brad-jones/superclosure
236 Downloads
Doing interesting things with closures like serialization.
attla/data-token
21 Downloads
Turn everything into a token: randomized or predictable.
wambacom/php_codesniffer
69 Downloads
PHP_CodeSniffer tokenizes PHP, JavaScript and CSS files and detects violations of a defined set of coding standards.
undercoder/magenizer
4 Downloads
Implements Iterator interface over a String for extract data with an array of tokens. The name comes from Mage and tokenizer :)
tienne/php_codesniffer
19 Downloads
PHP_CodeSniffer tokenizes PHP, JavaScript and CSS files and detects violations of a defined set of coding standards.
thomasbachem/php-short-array-syntax-converter
10785 Downloads
Command-line script to convert PHP's array() syntax to PHP 5.4's short array syntax [] using PHP's built-in tokenizer.
shaggyrec/php_codesniffer
6 Downloads
PHP_CodeSniffer tokenizes PHP, JavaScript and CSS files and detects violations of a defined set of coding standards.