Libraries tagged by tokenize
frolaxhq/laravel-llm-tokenkit
1772 Downloads
Token estimation, cost calculation & context-window utilities for LLM-powered Laravel apps
didix16/php-grammar
9824 Downloads
A simple library for make Lexer and Parsers to build a language
devrkb21/bkash-pgw-laravel
109 Downloads
Production-ready Laravel integration for bKash Tokenized Checkout V2 (auth, payments, webhooks).
dbeurive/lexer
2886 Downloads
This project implements a simple lexer.
cxrxexo/tiktoken
2196 Downloads
PHP version of tiktoken. PHP 7.4+ required.
chatflowphp/parser
64 Downloads
Handler-driven tokenizer and AST builder for PHP: define your grammar as small callables, get tokens, a navigable syntax tree, and compilation.
apicart/fql
10769 Downloads
Filter Query Language
alto/code-highlight
177 Downloads
Server-side syntax highlighting for PHP with semantic scopes, embedded languages, and adaptable themes.
hippophp/hippo
32 Downloads
Hippo is an open-source tool that helps PHP programmers adhere to certain a given convention.
laurynasgadl/travers
463489 Downloads
Travers - PHP array traversing via tokenized paths
devsrealm/tonics-template-system-extension-bundle
3682 Downloads
Custom Bundle Extension (Custom Tokenizers) for Tonics Template System
tsukasa/php-tokenizer
24160 Downloads
Polyfill for php extension Tokenizer
tanuel/tokenizer
15 Downloads
A tokenizer for PHP
robinthehood/tokenizer
968 Downloads
With Tokenizer you can easily convert a string to a stream or array of tokens.
ricwein/tokenizer
1614 Downloads
configurable string tokenizer