AgentEvals
Supports UIby agentevals-dev
Framework-agnostic AI agent evaluation using OpenTelemetry traces to score performance and inference quality without re-execution.
What it does
AgentEvals connects to AI agent execution traces via OpenTelemetry (OTel) to provide deterministic scoring of agent behavior. It allows developers to benchmark agents before production by analyzing tool trajectories and response quality from existing traces, eliminating the need for expensive and slow re-runs.
Tools
list_metrics: Displays all available built-in and community evaluation metrics.evaluate_traces: Processes local OTLP or Jaeger trace files to generate scores.list_sessions: Lists active streaming sessions for real-time evaluation.summarize_session: Provides a structured summary of an agent session's tool calls.evaluate_sessions: Scores live sessions against a defined golden reference set.
Installation
Add to your claude_desktop_config.json:
{
"mcpServers": {
"agentevals": {
"command": "agentevals",
"args": ["mcp"]
}
}
}
Supported hosts
- Claude Desktop
- Claude Code
Quick install
pip install agentevals-cliInformation
- Pricing
- free
- Published
- 6/18/2026
- stars
- 0
Categories
Choose your AI client and follow the steps below.
Claude Desktop
{"mcpServers": {"agentevals": {"command": "agentevals", "args": ["mcp"]}}}Related Apps
DiffSitter MCP
MCP ServerAI-powered structural code navigation using tree-sitter ASTs for semantic understanding across 14+ languages.
OpenAI Apps SDK Examples
MCP AppOfficial example gallery of interactive MCP widgets for ChatGPT — 3D viewers, maps, carousels, shopping carts, and more.
Human MCP
MCP ServerGive AI agents human-like senses: visual analysis, image/video generation, speech synthesis, browser automation, and advanced reasoning — 29 MCP tools in one se
Shopify MCP Server
MCP ServerDirect interaction with Shopify store data via GraphQL API for managing products, customers, and orders.
Git MCP Server
MCP ServerFull-featured Git MCP server exposing 28 tools for AI agents to clone, commit, branch, diff, merge, rebase, and more via STDIO or Streamable HTTP.
CodexPotter
MCP ServerAutonomous reconciliation loop that drives Codex to align your codebase with instructed states.
Smart Coding MCP
MCP ServerSemantic code search for AI assistants using local embeddings — find code by meaning, not just keywords.
Snowflake MCP Server
MCP ServerDirect Snowflake database interaction via MCP, enabling AI to run SQL queries, explore schemas, and manage data insights.