Libraries tagged by tokeniser

vchagin/superclosure

2 Favers
65 Downloads

Serialize Closures, including their context

td7650/superclosure

0 Favers
58 Downloads

Serialize Closure objects, including their context and binding

taufik-nurrohman/minify

3 Favers
11 Downloads

Reduce CSS, HTML, JavaScript, JSON, PHP, and XML code size on the server side.

scheldejonas/phpunit-php-token-stream

0 Favers
208 Downloads

Wrapper around PHP's tokenizer extension.

purewater2011/tiktoken-php7

0 Favers
3 Downloads

PHP 7.4+ compatible version of tiktoken - OpenAI's tiktoken tokenizer ported to PHP

octava/code-generator

0 Favers
1 Downloads

PHP code generator

nexxes/tokenmatcher

0 Favers
9 Downloads

PHP library to match tokenstreams (arrays of tokens) for later parsing.

muhfirdaus19/bahasata

1 Favers
6 Downloads

Text Processing PHP bahasa indonesia

megao-phpunit/php-token-stream

1 Favers
45 Downloads

Wrapper around PHP's tokenizer extension.

legacyphpdoctor/yethee_tiktoken-php-7-4

0 Favers
1 Downloads

PHP version of tiktoken

jamesfrost/statistical-classifier

1 Favers
2680 Downloads

A PHP implementation of Complement Naive Bayes and SVM statistical classifiers, including a structure for building other classifier, multiple data sources and multiple caching backends

html5/htmlreader

1 Favers
3306 Downloads

Html5 stream tokenizer/reader (not using libxml)

hike/hike

1 Favers
5 Downloads

A consistent interface to tokenize, analyse and modify any text format. Programming languages, natural languages, markup languages

guanhui07/tiktoken

1 Favers
8 Downloads

PHP version of tiktoken

gorriecoe/silverstripe-sreg

1 Favers
137 Downloads

Simple tokenizer that allows you to use .ss template like variables in a dataobjects string.

<< Previous Next >>