colibri
A dependency-free C engine that streams expert weights from disk to run the 744B-parameter GLM-5.2 MoE model on consumer hardware with as little as 25 GB of RAM.
colibri
What is colibri?
colibri vs Similar AI Tools
| Pricing Model | Free | Free | Free | Free, Freemium |
| Free Credits | ||||
| Key Features |
|
|
|
|
| Pros |
|
|
|
|
| Cons |
|
|
|
|
| Best For |
|
|
|
|
How to use colibri?
- 1Clone the repository: git clone https://github.com/JustVugg/colibri.git
- 2Build the engine: cd c && make
- 3Convert the FP8 model to int4: ./coli convert --model /path/to/model
- 4Run chat: COLI_MODEL=/path/to/model ./coli chat
- 5(Optional) Use the web UI or OpenAI-compatible server for a graphical interface.
colibri Key Features
- Pure C implementation with zero external dependencies
- Streaming of expert weights from disk, with LRU cache and optional pinned hot-store
- Faithful GLM-5.2 (glm_moe_dsa) forward pass, validated token-exact
- MLA attention with compressed KV-cache (57x smaller)
- Native MTP speculative decoding for up to 2.8 tokens per forward
- Integer-dot kernels (int8/int4) with AVX2 acceleration
- Online learning cache that adapts to usage patterns
colibri Use Cases
- Running a 744B-parameter MoE model on a consumer laptop or desktop
- Offline chat and inference without cloud dependencies
- Research and experimentation with large MoE architectures
- Educational demonstrations of frontier model capabilities on limited hardware
colibri Pricing & Free Credits
colibri currently operates on a Free model.
This tool is completely free to use
colibri Pros & Cons
Pros
- Runs a 744B-parameter model on consumer hardware with only 25 GB RAM
- Open source and fully transparent (C code, no dependencies)
- Efficient disk streaming with caching enables long-running usage
- Active community benchmarks and improvements
Cons
- Very slow cold decode (0.05-0.1 tok/s on typical NVMe)
- Requires ~370 GB disk space for the converted int4 model
- Only supports GLM-5.2 model (no multi-model flexibility)
- CUDA backend is experimental and limited
What is colibri best for?
- Developers wanting to run massive models locally
- AI researchers exploring MoE architectures on limited hardware
- Enthusiasts interested in frontier model inference without cloud costs
colibri FAQ
Top free alternatives to colibri
A unified AI gateway providing access to 300+ leading models through one EU-hosted, GDPR-compliant API with an OpenAI SDK-compatible interface.
Unified AI API gateway providing access to top-tier LLMs, image, video, and coding models at affordable prices.
One chat interface that gives you access to over 100 AI models, automatically selecting the best model for your task while tracking energy usage and protecting your privacy.
Oxlo.ai is a privacy-first AI inference API offering request-based pricing for over 45 open-source models.
Graphsignal is a production-scale inference profiling platform that helps engineers optimize AI performance across models, engines, GPUs, and other accelerators.
Atlas Cloud is a full-modal AI inference platform offering one API for chat, image, video, and audio models.
Groq provides fast, low-cost AI inference via GroqCloud and its custom LPU stack.
Best alternatives AI Tools to colibri
A specialized local inference engine for large language models, optimized for Apple Silicon and SSD streaming on memory-constrained systems.
Unified, zero-allocation native AI inference runtime for Java, consolidating LLM, vision, and speech pipelines via Project Panama.
A unified AI gateway providing access to 300+ leading models through one EU-hosted, GDPR-compliant API with an OpenAI SDK-compatible interface.
A unified API platform for LLM inference with cost optimization, routing, observability, and automatic failover.
Unified AI API gateway providing access to top-tier LLMs, image, video, and coding models at affordable prices.
Frugon is a free, open-source LLM cost analyzer that identifies where your LLM bill leaks by analyzing your call logs locally.