Back to Apps
T
Tokio Prompt Orchestrator
by Mattbusel
Production-grade, multi-core orchestration for LLM pipelines with circuit breakers and rate limiting.
0 stars
4 views
Works in:claude
Exposes:Tools
What it does
Provides a high-performance, asynchronous orchestration layer for LLM inference pipelines. It solves critical production issues like thundering herds, provider instability, and cost opacity by implementing a bounded-backpressure DAG with resilience layers.
Tools
infer: Executes a prompt through the optimized pipeline.batch_infer: Performs multi-prompt inference with smart micro-batching.pipeline_status: Returns real-time health metrics of the pipeline stages.configure_pipeline: Dynamically updates pipeline parameters.replay_dlq: Re-injects failed requests from the dead-letter queue.
Installation
Run the MCP binary and add to your claude_desktop_config.json:
{
"mcpServers": {
"orchestrator": {
"url": "http://127.0.0.1:8080"
}
}
}
Supported hosts
- Claude Desktop
Quick install
cargo run --bin mcp --features mcpInformation
- Pricing
- free
- Published
- 4/17/2026
- stars
