Download the PHP package aysnc/llm-eval without Composer
On this page you can find all versions of the php package aysnc/llm-eval. It is possible to download/install these versions without Composer. Possible dependencies are resolved automatically.
Download aysnc/llm-eval
More information about aysnc/llm-eval
Files in aysnc/llm-eval
Package llm-eval
Short Description Simple LLM evaluation framework for PHP.
License MIT
FAQ
Example:
In general, it is recommended to use always a project to download your libraries. In an application normally there is more than one library needed.
- Some hosting areas are not accessible by a terminal or SSH. Then it is not possible to use Composer.
- To use Composer is sometimes complicated. Especially for beginners.
- Composer needs much resources. Sometimes they are not available on a simple webspace.
- If you are using private repositories you don't need to share your credentials. You can set up everything on our site and then you provide a simple download link to your team member.
- Simplify your Composer build process. Use our own command line tool to download the vendor folder as binary. This makes your build process faster and you don't need to expose your credentials for private repositories.
Informations about the package llm-eval
LLM-Eval
A PHP package for evaluating LLM outputs. Test your prompts, validate responses, and ensure your AI features work correctly.
Installation
Configuration
Create llm-eval.php in your project root:
| Option | Type | Default | Description |
|---|---|---|---|
provider |
ProviderInterface |
— | The LLM provider shared across all eval files |
directory |
string |
'evals' |
Directory containing your eval files |
cache |
bool\|string |
false |
true uses .llm-cache/, or pass a custom path |
cacheTtl |
int |
0 |
Cache lifetime in seconds (0 = forever) |
parallel |
bool |
false |
Run evals in parallel by default |
concurrency |
int |
0 |
Max concurrent requests when parallel (0 = unlimited) |
Quick Start
1. Create an eval file in your evals directory. Each file returns an LlmEval instance:
2. Run it:
The config provides the LLM provider, the eval file defines what to test — no ->provider() or ->runAll() needed in the file.
Core Concepts
An evaluation has three parts: a provider (which LLM to call), a dataset (prompts + expected answers), and assertions (how to check the response).
Datasets
A dataset is a collection of test cases. Each test case has a prompt and an optional expected value.
The expected key can be a single value or multiple named values:
CSV files use column prefixes for multiple values: expected_name, expected_age.
Any keys that aren't prompt or expected become metadata, accessible via $testCase->getData('key').
Assertions
Assertions define what "correct" means for a response. You chain them inside the assertions() callback.
Text
JSON
Custom
There are also assertions for LLM-as-judge — covered in the sections below.
Testing Scenarios
Structured Output
Validate that the LLM returns well-formed JSON with the right content. Combine isJson() with contains() or multiple expected values.
Tool Call Testing
Test that your LLM calls tools with the right parameters — without executing a full conversation loop. This uses LlmEval::create() (not createConversation) since you're only checking the first response.
Available tool call assertions:
Multi-Turn Conversations
Test agentic workflows where the LLM calls tools, receives results, and continues reasoning. Use LlmEval::createConversation() with a tool executor that returns simulated results.
Available conversation assertions:
Multi-Turn Datasets
Use a turns array to test conversations with multiple user messages. Each turn has its own prompt and optional expected values for per-turn assertions. Use getTurn() to access the 1-indexed turn number.
LLM-as-Judge
Use one LLM to evaluate another's response quality. Instead of checking for exact strings, you describe what "good" looks like and a judge model scores the response 0-100%.
Judging Conversations
For multi-turn conversations, you can use judgedBy() inside assertions() to judge per-turn, or use ->judge() on the eval to run a single evaluation after all turns complete. The judge receives the full conversation history — all messages, tool calls, and results.
CLI Runner
Output
With -v, passing tests also show judge scores and tool call details.
Providers
Anthropic Claude
Direct API access. Get your key at console.anthropic.com.
Default model: claude-sonnet-4-20250514
AWS Bedrock
Uses the Converse API — works with Claude, Titan, Llama, Mistral, and other Bedrock models. Requires composer require aws/aws-sdk-php. See AWS Bedrock docs.
Default model: anthropic.claude-3-5-sonnet-20241022-v2:0
Changing the Model
Use ->model() to override the default model for any provider:
This works with both AnthropicProvider (Anthropic model IDs) and BedrockProvider (Bedrock model IDs).
You can also set ->maxTokens(2048) to override the default max tokens (1024).
Programmatic API
If you need to run evals from PHP code — inside a test suite, a CI script, or anywhere you want to work with the results directly — use ->provider() and ->runAll():
Requirements
- PHP 8.3+
guzzlehttp/guzzle^7.10aws/aws-sdk-php^3.0 (optional, for Bedrock)
