Millwright
Millwright is an open-source, self-hosted LLM router that routes AI requests to the lowest-cost model providers based on policy, preserving prompt caches and controlling spend.
Millwright
What is Millwright?
Millwright vs Similar AI Tools
| Pricing Model | Free | Custom Pricing | Paid | Free |
| Free Credits | ||||
| Key Features |
|
|
|
|
| Pros |
|
|
|
|
| Cons |
|
|
|
|
| Best For |
|
|
|
|
How to use Millwright?
- 1git clone https://github.com/Northwood-Systems/millwright.git
- 2cd millwright
- 3cargo install --path . --locked
- 4millwright init
- 5millwright serve
- 6Alternatively, for a local demo:
- 7make compose-up
- 8curl http://localhost:8080/v1/chat/completions -H 'Authorization: Bearer sk-millwright-local-demo' -H 'Content-Type: application/json' -d '{"model":"millwright-fast","messages":[{"role":"user","content":"Hello"}]}'
Millwright Key Features
- Policy-controlled routing with task and risk headers
- Cache-aware concurrency maintaining independent affinity lanes for cheap, mid, and frontier roles
- Spend tracking and cost analysis via ledger, CLI commands, and management APIs
- Provider and protocol support for OpenAI Chat Completions, Anthropic Messages, and Amazon Bedrock
- Self-hosting as a single Rust binary with Docker support; provider credentials stay on your infrastructure
Millwright Use Cases
- Reduce LLM costs by routing requests to the cheapest model that meets policy requirements
- Maintain prompt cache reuse across a session by preserving provider/model affinity
- Switch between providers without changing application code
- Inspect and audit routing decisions with response headers and ledger entries
Millwright Pricing & Free Credits
Millwright currently operates on a Free model.
This tool is completely free to use
Millwright Pros & Cons
Pros
- Cost control by using cheapest appropriate models
- Preserves prompt caches across requests
- Provider-agnostic; easy to switch without code changes
- Fully self-hosted; data and credentials remain on your infrastructure
- Inspectable routing decisions with detailed logging
Cons
- Requires self-hosting and setup
- Limited to supported protocols (OpenAI, Anthropic, Bedrock)
- Not an agent orchestrator; no prompt inspection or rewriting
What is Millwright best for?
- Teams managing LLM costs
- Developers needing policy-based routing
- Enterprises requiring self-hosted privacy
Millwright FAQ
Top free alternatives to Millwright
Manifest converts any URL into a structured JSON map of what AI agents can interact with on a page—buttons, forms, inputs, and required fields.
A cooperative gate for shell commands that AI agents run, providing previews, backups, policy enforcement, and audit for tools like Claude Code and Cursor.
Agentcard provides agent-friendly card issuing and payment infrastructure for AI agents, enabling 5-minute setup and autonomous purchases.
Oodle AI provides agent observability with fast trace search, S3-based storage, and out-of-the-box insights to detect silent failures in AI agents.
Jacquard is a small programming language designed for running, reviewing, and trusting programs written by machine-learning models and reviewed by people.
Autonomous security testing platform that finds and fixes access control vulnerabilities in live AI-built apps.
AI-powered social listening tool that monitors Reddit, X, LinkedIn, and 10+ other platforms, filters mentions with AI, and delivers them to your stack via API, Slack, or webhooks.
Best alternatives AI Tools to Millwright
CartAI provides an API and MCP server to deploy AI agents that autonomously navigate web properties and complete transactions like checkout, subscriptions, and invoice payments.
A marketplace for buying and selling GPU hours by the week, offering instant liquidity and flexible compute for AI workloads.
szr is a Go-native CLI proxy that reduces noisy terminal output before it reaches an LLM context, saving tokens and preserving command exit codes.
CodeAlmanac is a living wiki for your codebase, maintained by AI coding agents, capturing decisions, flows, invariants, and gotchas in plain markdown.
Stele provides shared memory for AI coding agents, recording decisions, tasks, and lessons to maintain context across tools and sessions.
Self-hosted network monitor that runs hourly speed tests, LAN device scans, logs to SQLite, and delivers sarcastic AI-generated reports to Telegram.
A macOS menu bar app that connects to Stripe, GA4, and Google Search Console to celebrate real-time metrics with emoji rain, sounds, and full-screen reactions.