Grade the outcome.
Pass or fail comes from the final state of balances, approvals and deployments, not from how convincing the reply sounds.
Break it on purpose.
Inject a timeout, rate limit or expired credential at an exact tool call, before or after the commit.
Block the regression.
Compare versions on the same scenario revision. A failing candidate exits 3 and stops the release in CI.
Keep the evidence.
Every argument, result, policy denial and state change is recorded in an event hash chain you can export and replay.
Meet Verixa
Test what your agent does, not just what it says.
Simulate stateful systems
A stateful world your agent can't break for real.
Orders, refunds, approvals, support tickets and deployments behave like the real thing: policy, idempotency keys and balances included. Nothing ever reaches a payment gateway, ticket system or cluster.
- Ten executable scenarios, from refund success to bounded deployment scale
- Import or edit scenarios as JSON; the server validates every fixture
- The engine has no networking and no subprocess execution

Inject faults
The failures that cause real incidents, placed exactly.
Pick a tool and an occurrence, choose a fault, and run it. A timeout before the commit and a response lost after it look identical to the agent; Verixa tests both.
- timeout, commit_timeout, rate_limit, malformed and expired_credentials
- Faults land on an exact tool invocation, before or after commit
- A step-by-step fault builder in the console creates new scenarios

Verify final state
Did the refund happen once? Verixa checks.
Assertions run against the final simulated state. Approvals must cover the exact order and amount, refund balances prevent over-refunding, and a missing assertion path fails.
- Stateful refund ledger with idempotency checks
- Approval records matched to the exact order and amount
- Tool policy enforced even when the agent asks for an unauthorized tool

Compare versions · release gate
Ship the new model only when it does no worse.
Compare a baseline and a candidate on the same scenario revision, so a changed expectation can't pass for an improvement. Any failing candidate blocks the gate.
- Runs store scenario hashes; comparison requires them to match
- Exit code 3 on a failed outcome or regression
- JUnit reports for any CI system

Evidence and replay
A record of what the agent actually did.
Each rehearsal records tool arguments, results, policy denials, state diffs and an event hash chain. Export it, verify it, or replay the exact action sequence in fresh state.
- Evidence and action-trace export as JSON
- Chain verification detects local edits to the event sequence
- Deterministic replay of a recorded action sequence

Bring any model
Bring the model you already run.
Point Verixa at any OpenAI-compatible chat-completions endpoint with tool calling. The model proposes calls; Verixa executes them against the simulated world.
- Model, replay and scripted reference adapters, or your own Python adapter
- Model network calls are opt-in, from the command line
- Token usage recorded when the endpoint reports it

The rehearsal loop
Every tool call, rehearsed end to end.
Your agent proposes the next action. Verixa checks it against policy, injects the fault you configured, applies the change to simulated state and, when the agent stops, decides the verdict from what changed.
- Propose
- Policy check
- Inject fault
- Apply
- Assert
- Verdict
- One engine for the command line and the console, with the same assertions and store.
- Model adapters can't mutate state directly; every change goes through policy.
- Each step records arguments, result, state diff and an event hash.
- Finished runs are saved as immutable records, ready to compare or export.
Do more with Verixa
Catch the failures a transcript hides.
Why Verixa
Prompt evals grade the reply. Verixa grades the outcome.
An agent can sound right and still change the wrong record, retry a committed payment or cross a permission boundary.
| Typical prompt/LLM eval tool | Verixa | |
|---|---|---|
| What is tested | Prompts and model replies across test cases | The agent's tool calls, executed against stateful simulated systems |
| Pass or fail signal | Assertions on the reply, often graded by another model | Assertions on final state: balances, approvals, deployments |
| Failure injection | Tool failures come from your own mocks or harness | Five built-in fault types at an exact tool call, before or after commit |
| Version comparison | Side-by-side outputs for prompts and providers | Same-scenario-revision comparison, with exit code 3 on regression |
| Adversarial input | Red-teaming of the model's replies | Injection through a simulated support ticket, checked against final state |
| Evidence | Eval results and a results viewer | Recorded actions, state diffs, an event hash chain and deterministic replay |
Prompt eval tools are the right fit for grading replies across many providers. Verixa is for rehearsing the actions an agent takes, against simulated tool state.
Get started
Rehearsing in minutes.
Python 3.11+, no runtime dependencies, no frontend build and no API key for the demo. The demo seeds eleven simulator runs: ten reference runs and one known regression, from scripted fixtures.
Point it at any OpenAI-compatible tool-calling model, then gate CI on exit codes: 0 passed, 1 error, 3 failed outcome or regression.
git clone https://github.com/zyvorai/verixa && cd verixa
VERIXA_TOKEN=Admin@321 python3 -m verixa serve --demo
# open http://127.0.0.1:8788 — sign in as admin / Admin@321
python3 -m verixa run all --junit artifacts/reference.xml
python3 -m verixa run ticket-injection --agent regression # exits 3
Open, and honest about its limits
Apache-2.0. CI on every push runs the Python suite on 3.11–3.13, a console-to-API harness, real Chromium tests and a container build. Evidence is labelled simulated-tool-state: Verixa proves what your agent did against simulated systems, not that it will behave identically in production. The event hash chain detects edits; it is not a signature. Model execution is CLI-only in 0.1.
Rehearse your agents before they touch production.
Run the demo in one command, then point Verixa at your own model and gate your next release on outcomes. Zyvor Enterprise adds supported releases, deployment guidance and priority triage.
