Libraries tagged by tokenizer
lalbert/tokenizer-engine
17 Downloads
Tokenize string like Elasticsearch
juanchosl/tokenizer
5 Downloads
A small collection of encoder/decoder Token for authentication
interitty/tokenizer
41 Downloads
Use regular expressions to split a given string into tokens.
giftuals/tokenizer
2 Downloads
PHP JWT library that allows you to create trustworthy links from and to your website or webapp
fenom/tokenizer
21 Downloads
Stand-alone Fenom's tokenizer
crazzy501/gpt3-tokenizer
7 Downloads
PHP package for Byte Pair Encoding (BPE) used by GPT-3.
chinleung/tokenizer
591 Downloads
PHP Tokenizer
amsterdamstandard/tokenizer
17 Downloads
Tokenizer api wrapper for PHP
affinity4/tokenizer
4 Downloads
A zero-dedpendency tokenizer written in PHP. Returns an easily navigatable Stream object of Token objects with public type, value, offset and length properties
ae8/tokenizer
14 Downloads
A simple PHP library for token-based encryption.
mathewberry/tokenizer
769 Downloads
A very lightweight token system for micro-services
gossi/tokenizer
150 Downloads
Tokenizer for PHP code
wpscholar/php-tokenizer
2 Downloads
An experiment with using the PHP tokenizer functions.
ueef/auth-tokenizer
901 Downloads
soloproyectos-php/text-tokenizer
891 Downloads
A library to split a string into smaller tokens. This library is part of the SoloProyectos PHP API.