Libraries tagged by tokenize
ze/tokenizer-gpt3
1412 Downloads
PHP package for Byte Pair Encoding (BPE) used by GPT-3.
textualization/ropherta-tokenizer
62 Downloads
GPT3Tokenizer (BPE) with Roberta-base vocabulary.
tanuel/tokenizer
14 Downloads
A tokenizer for PHP
robinthehood/tokenizer
816 Downloads
With Tokenizer you can easily convert a string to a stream or array of tokens.
phpfacile/string-tokenizer
39 Downloads
Library to parse a string (especially to extract text and keywords identified by a begin and end delimiter and to substitute keywords by values)
mathematicator-core/tokenizer
4414 Downloads
Math Tokenizer
irfan-chowdhury/bkash-tokenized-checkout
15 Downloads
This is a bkash tokenized payment gateway solution with laravel. Integrate with the payment solution that fits all business needs and Enrich the customers' online payment experience.
hippophp/tokenizer
1101 Downloads
Wrapper around token_get_all
gollumsf/url-tokenizer-bundle
1476 Downloads
Simple url tokenizer
everstu/gpt3-tokenizer
74 Downloads
PHP package for Byte Pair Encoding (BPE) used by GPT-3.
atefeboluri/laravel-tokenize-login
7 Downloads
sastrawi/tokenizer
946 Downloads
PHP library that allows you to tokenize Bahasa Indonesia.
fiv/tokenizer
248 Downloads
Extreme and flexible library for parsing and modify php code
masroore/stopwords
3002 Downloads
A PHP package to remove common stopwords from an input text, it covers most languages.
vanderlee/comprehend
16 Downloads
Framework for building BNF LR(1) parsers