Okareo MCP Server
The Okareo MCP (Model Context Protocol) server lets you interact with the Okareo platform directly from any MCP-ready editor — including Claude Code, Cursor, Cline, GitHub Copilot, and Windsurf. Your copilot can analyze your project, generate test scenarios from your codebase, run evaluations, simulate multi-turn conversations, and compare results — all from natural language prompts in your editor.
See the MCP User Guide for a walkthrough of key use cases and patterns, or jump straight to Configuration for setup instructions.
Getting Started
See the Configuration page for setup instructions.
Key Use Cases
Create Scenarios from Your Codebase
Ask your copilot to analyze your project and produce test scenarios. It can read your code, understand your domain, and generate realistic input/result pairs as Okareo scenarios — no manual data entry required.
Set Up Drivers for Multi-Turn Simulations
Define simulated user personas that interact with your system. Your copilot can create drivers with specific behaviors, frustration levels, or domain expertise tailored to your application's user base.
Run Simulations and Evaluations
Execute multi-turn simulations and single-turn evaluations. Your copilot orchestrates the full flow: creating targets, registering models, selecting checks, and running tests.
Analyze and Compare Results
Retrieve detailed results from test runs and simulations. Your copilot can compare runs side-by-side, identify regressions, and surface patterns in evaluation scores and conversation transcripts.
Available Tools
The MCP server exposes Okareo's core capabilities as tools your AI assistant can call:
| Category | Tools | Description |
|---|---|---|
| Scenarios | save_scenario, list_scenarios, get_scenario, create_scenario_version, preview_delete_scenario, delete_scenario | Create and manage test datasets |
| Models | register_generation_model, list_generation_models, get_generation_model, list_available_llms, update_generation_model, delete_generation_model | Register and configure models under test |
| Checks | create_or_update_check, generate_check, list_checks, get_check, delete_check | Define evaluation criteria — prompt-based, code-based, or AI-generated |
| Tests | run_test, list_test_runs, get_test_run_results | Run evaluations and inspect results |
| Simulations | create_or_update_target, get_target, list_targets, delete_target, create_or_update_driver, get_driver, list_drivers, run_simulation, list_simulations | Define targets and driver personas, execute multi-turn conversations, and manage simulation lifecycle |
| Docs & Templates | get_docs, get_templates | Query documentation and retrieve prompt templates |
Configuration Reference
| Environment Variable | Default | Description |
|---|---|---|
OKAREO_API_KEY | (required) | Your Okareo API token |
OKAREO_BASE_URL | https://api.okareo.com | Custom backend URL (for on-prem deployments) |
Provider API Keys
The MCP server automatically detects provider API keys from your environment (e.g., OPENAI_API_KEY, ANTHROPIC_API_KEY) and injects them into evaluations. You don't need to pass these explicitly when running tests.
Troubleshooting
- Python version: The MCP server requires Python 3.10–3.12. Python 3.13+ is not yet supported.
- Server not starting: Run
uvx okareo-mcpdirectly in your terminal to see error output. - Authentication errors: Verify your
OKAREO_API_KEYis valid by testing withcurl -H "api-key: $OKAREO_API_KEY" https://api.okareo.com/v0/projects.