Waza

What it does

Waza provides a professional toolkit for developers to build and benchmark AI agent skills. It allows you to scaffold evaluation suites, run rigorous benchmarks, and compare the effectiveness of different LLMs against specific skill requirements.

Tools

• init: Initialize a project workspace with skills and evals.
• new skill: Scaffold a new agent skill with a standard structure.
• run: Execute evaluation benchmarks against a spec file.
• compare: Side-by-side comparison of results from multiple evaluation runs.
• check: Comprehensive readiness report for skill submission.
• quality: LLM-as-Judge evaluation of skill content across clarity, completeness, and precision.

Installation

Add the following to your claude_desktop_config.json:

{
  "mcpServers": {
    "waza": {
      "command": "waza",
      "args": ["run", "eval.yaml"]
    }
  }
}

(Note: Requires waza binary installed via curl -fsSL https://raw.githubusercontent.com/microsoft/waza/main/install.sh | bash)

Supported hosts

• Claude Desktop
• GitHub Copilot