Libraries tagged by linktoken

developer-savyour/laravel-tiktoken
0 Favers
66 Downloads

A package to connect tiktoken DB with php laravel

Go to Download

topthink/tiktoken
0 Favers
12 Downloads

Go to Download

zohaibtariq/count-gpt-token
0 Favers
2614 Downloads

A laravel project used to count token for chat gpt 2 & 3 prompt.

Go to Download

webrek/tokenizers
0 Favers
0 Downloads

Native byte-level BPE (tiktoken-compatible: cl100k_base/o200k_base) plus WordPiece and Unigram tokenizers for PHP 8.3+, with a process-shared vocab cache. Loads HuggingFace tokenizer.json models and counts Claude/Gemini tokens via their APIs.

Go to Download

token27/nexus-ai-tokenizer
0 Favers
25 Downloads

Provider-agnostic LLM token counter for PHP 8.3+. Supports Tiktoken BPE (OpenAI, Claude approx.), HuggingFace JSON vocabularies (DeepSeek, LLaMA, Mistral), SentencePiece (Gemini), and extensible custom strategies.

Go to Download

<< Previous