Libraries tagged by cl100k_base

rajentrivedi/tokenizer-x

88 Favers
100417 Downloads

TokenizerX calculates required tokens for given prompt

danny50610/bpe-tokeniser

9 Favers
235391 Downloads

PHP port for openai/tiktoken (most)

your1/tokenizer-x

0 Favers
272 Downloads

TokenizerX calculates required tokens for given prompt