OmniMCP
by openadaptai
AI-driven UI interaction and visual perception using Microsoft OmniParser and MCP.
What it does
OmniMCP bridges the gap between LLMs and complex user interfaces. By leveraging Microsoft's OmniParser, it allows AI models to visually perceive the screen, identify UI elements, and execute precise mouse and keyboard actions to complete goals autonomously.
Tools
- Visual Perception: Analyzes screenshots to identify and label interactive UI components.
- LLM Planner: Generates a sequence of actions based on current visual state and goal.
- Agent Executor: Orchestrates the perceive-plan-act loop for continuous task execution.
- Input Controller: Performs physical interactions via pynput for mouse and keyboard control.
Installation
{
"mcpServers": {
"omnimcp": {
"command": "python",
"args": ["/path/to/OmniMCP/cli.py"]
}
}
}
Supported hosts
- Claude Desktop
- Linux (X11/Wayland)
Quick install
git clone https://github.com/OpenAdaptAI/OmniMCP.git && cd OmniMCP && ./install.shInformation
- Pricing
- free
- Published
- 5/1/2026
- stars
- 0
Categories
Choose your AI client and follow the steps below.
Claude Desktop
Add to claude_desktop_config.json with python and cli.py pathRelated Apps
DiffSitter MCP
MCP ServerAI-powered structural code navigation using tree-sitter ASTs for semantic understanding across 14+ languages.
OpenAI Apps SDK Examples
MCP AppOfficial example gallery of interactive MCP widgets for ChatGPT — 3D viewers, maps, carousels, shopping carts, and more.
Human MCP
MCP ServerGive AI agents human-like senses: visual analysis, image/video generation, speech synthesis, browser automation, and advanced reasoning — 29 MCP tools in one se
Shopify MCP Server
MCP ServerDirect interaction with Shopify store data via GraphQL API for managing products, customers, and orders.
Git MCP Server
MCP ServerFull-featured Git MCP server exposing 28 tools for AI agents to clone, commit, branch, diff, merge, rebase, and more via STDIO or Streamable HTTP.
CodexPotter
MCP ServerAutonomous reconciliation loop that drives Codex to align your codebase with instructed states.
Smart Coding MCP
MCP ServerSemantic code search for AI assistants using local embeddings — find code by meaning, not just keywords.
Snowflake MCP Server
MCP ServerDirect Snowflake database interaction via MCP, enabling AI to run SQL queries, explore schemas, and manage data insights.