Libraries tagged by tokenize
irfan-chowdhury/bkash-tokenized-checkout
9 Downloads
This is a bkash tokenized payment gateway solution with laravel. Integrate with the payment solution that fits all business needs and Enrich the customers' online payment experience.
interitty/tokenizer
38 Downloads
Use regular expressions to split a given string into tokens.
hippophp/tokenizer
1101 Downloads
Wrapper around token_get_all
everstu/gpt3-tokenizer
59 Downloads
PHP package for Byte Pair Encoding (BPE) used by GPT-3.
sastrawi/tokenizer
946 Downloads
PHP library that allows you to tokenize Bahasa Indonesia.
masroore/stopwords
2985 Downloads
A PHP package to remove common stopwords from an input text, it covers most languages.
xtreed/php-token-stream
3520 Downloads
Wrapper around PHP's tokenizer extension.
vanderlee/comprehend
14 Downloads
Framework for building BNF LR(1) parsers
turanjanin/serbian-language-tools
1918 Downloads
Set of tools for tokenization, transliteration and diacritic restoration of a text written in Serbian language.
teepluss/wordbreaker
23 Downloads
Word breaker is Thai wording separator for using with fulltext software.
spec-doc/parser-contracts
29 Downloads
A set of interfaces for the parser for the specificator
rudl/logger
2351 Downloads
Rudl UDP cluster logging library (PSR4 compliant)
phpwatch/phptoken-polyfill
17 Downloads
A polyfill for PHP 8.0's PhpToken class
patchwork/php-parser
26 Downloads
Fast and extensible PHP code analysis and transformation
dbeurive/lexer
2274 Downloads
This project implements a simple lexer.