Millwright
Millwright is an open-source, self-hosted LLM router that routes AI requests to the lowest-cost model providers based on policy, preserving prompt caches and controlling spend.
Millwright
What is Millwright?
Millwright vs Similar AI Tools
| Pricing Model | Free | Free | Free | Free |
| Free Credits | ||||
| Key Features |
|
|
|
|
| Pros |
|
|
|
|
| Cons |
|
|
|
|
| Best For |
|
|
|
|
How to use Millwright?
- 1git clone https://github.com/Northwood-Systems/millwright.git
- 2cd millwright
- 3cargo install --path . --locked
- 4millwright init
- 5millwright serve
- 6Alternatively, for a local demo:
- 7make compose-up
- 8curl http://localhost:8080/v1/chat/completions -H 'Authorization: Bearer sk-millwright-local-demo' -H 'Content-Type: application/json' -d '{"model":"millwright-fast","messages":[{"role":"user","content":"Hello"}]}'
Millwright Key Features
- Policy-controlled routing with task and risk headers
- Cache-aware concurrency maintaining independent affinity lanes for cheap, mid, and frontier roles
- Spend tracking and cost analysis via ledger, CLI commands, and management APIs
- Provider and protocol support for OpenAI Chat Completions, Anthropic Messages, and Amazon Bedrock
- Self-hosting as a single Rust binary with Docker support; provider credentials stay on your infrastructure
Millwright Use Cases
- Reduce LLM costs by routing requests to the cheapest model that meets policy requirements
- Maintain prompt cache reuse across a session by preserving provider/model affinity
- Switch between providers without changing application code
- Inspect and audit routing decisions with response headers and ledger entries
Millwright Pricing & Free Credits
Millwright currently operates on a Free model.
This tool is completely free to use
Millwright Pros & Cons
Pros
- Cost control by using cheapest appropriate models
- Preserves prompt caches across requests
- Provider-agnostic; easy to switch without code changes
- Fully self-hosted; data and credentials remain on your infrastructure
- Inspectable routing decisions with detailed logging
Cons
- Requires self-hosting and setup
- Limited to supported protocols (OpenAI, Anthropic, Bedrock)
- Not an agent orchestrator; no prompt inspection or rewriting
What is Millwright best for?
- Teams managing LLM costs
- Developers needing policy-based routing
- Enterprises requiring self-hosted privacy
Millwright FAQ
Top free alternatives to Millwright
A voice-controlled IDE that enables developers to initiate AI coding sessions with Codex or Claude Code, approve commands, and review diffs from their phone.
SureWire is a specialized QA platform that stress-tests AI agents for safety, reliability, and compliance using purpose-built testing agents.
Browser infrastructure platform for AI agents to run on the internet at speed with cloud browser sessions, agents, and serverless functions.
An agent-first file transfer tool that enables secure, encrypted file sharing between AI agents via MCP calls without human involvement.
Manifest converts any URL into a structured JSON map of what AI agents can interact with on a page—buttons, forms, inputs, and required fields.
A cooperative gate for shell commands that AI agents run, providing previews, backups, policy enforcement, and audit for tools like Claude Code and Cursor.
Agentcard provides agent-friendly card issuing and payment infrastructure for AI agents, enabling 5-minute setup and autonomous purchases.
Best alternatives AI Tools to Millwright
A collection of deliberately vulnerable MCP servers for training in agentic security, mapped to the OWASP Top 10 for Agentic Applications.
Open-source AI tool for deep infrastructure research, running frontier models across code, cloud, and runtime to deliver verified answers.
Magpie is an opinionated accounting CLI for humans and AI agents, providing double-entry bookkeeping with RBAC and immutable event storage on Jaybase.
Terminal UI to manage AI coding-agent sessions (Claude Code, OpenCode, Codex, Grok Build) in tmux with live status, group tree, and diff review.
A voice-controlled IDE that enables developers to initiate AI coding sessions with Codex or Claude Code, approve commands, and review diffs from their phone.
Zero-config, single-binary software catalog with auto-discovery, dependency visualization, compliance scorecards, and native MCP integration for AI agents.
Browser automation CLI for AI agents to bypass anti-bot walls, hand off to humans, and run parallel tasks.
An AI-powered shopper assistant that provides a live cursor to guide website visitors, point out products, and display price calculations.