MCP Apache Spark History Server

What it does

Bridges AI agents with Apache Spark infrastructure, enabling natural language querying of historical job execution data, resource usage, and SQL performance.

Tools

• list_applications: Retrieve available Spark applications with filters.
• get_job_bottlenecks: Identify performance bottlenecks with actionable recommendations.
• compare_job_performance: Compare metrics between two Spark jobs for regression detection.
• list_slowest_sql_queries: Find top slowest SQL queries with execution metrics.
• get_resource_usage_timeline: View chronological resource allocation patterns.

Installation

Add the following to your claude_desktop_config.json:

{
  "mcpServers": {
    "spark-history": {
      "command": "uvx",
      "args": ["--from", "mcp-apache-spark-history-server", "spark-mcp"]
    }
  }
}

Supported hosts

Confirmed support for Claude Desktop, Amazon Q CLI, and LangGraph.