Libraries tagged by tokenize

rajentrivedi/tokenizer-x

66 Favers
43758 Downloads

TokenizerX calculates required tokens for given prompt

nicoswd/php-rule-parser

127 Favers
55733 Downloads

Rule Engine - Rule Parser & Evaluator

jeremeamia/functionparser

50 Favers
153092 Downloads

Function parser for PHP functions, methods, and closures

danny50610/bpe-tokeniser

6 Favers
89743 Downloads

PHP port for openai/tiktoken (most)

coderevolutionplugins/gpt-3-encoder-php

98 Favers
32960 Downloads

PHP BPE Text Encoder/Decoder for GPT-2 / GPT-3

strscan/strscan

13 Favers
112375 Downloads

Simple string tokenizer for lexical scanning operations

semji/gpt-3-tokenizer-php

4 Favers
39242 Downloads

PHP Text Tokenizer for GPT models

ltd-beget/dns-zone-tokenizer

6 Favers
56630 Downloads

tokenize dns zone files and that's all, folks.

karim007/laravel-bkash-tokenize

18 Favers
4087 Downloads

This is bKash tokenize payment gateway for laravel

remorhaz/php-unilex

3 Favers
104300 Downloads

Unilex: lexical analyzer generator with Unicode support written in PHP

phplrt/lexer-contracts

1 Favers
114419 Downloads

A set of phplrt lexer abstractions

phplrt/lexer

3 Favers
103268 Downloads

The phplrt lexer library that can be used for scanners implementation using PCRE

camspiers/statistical-classifier

178 Favers
36953 Downloads

A PHP implementation of Complement Naive Bayes and SVM statistical classifiers, including a structure for building other classifier, multiple data sources and multiple caching backends

xp-framework/tokenize

0 Favers
89778 Downloads

Text tokenizing

muffin/tokenize

12 Favers
14471 Downloads

Security Tokens

<< Previous Next >>