Thinking Machines Lab has released Inkling 2 Mini, a 26.4 billion-parameter open-weight model built to run on a laptop. The model accepts text and image inputs, generates text, and is available now on Hugging Face under an Apache 2.0 license.
What Is Inkling 2 Mini and Its Unique Architecture
Inkling 2 Mini is a multimodal model designed for developers building AI-powered applications, including agentic systems, coding assistants, chatbots, and retrieval-augmented generation setups. It supports a context window of up to 1 million tokens and handles English, other languages, and multiple coding languages. The model comes from Thinking Machines Lab, a company led by Mira Murati.
The architecture is the headline. Inkling 2 Mini is a looped transformer with 24 physical layers. The first six layers run once, the middle 12 layers run three times using shared weights, and the final six layers run once. Think of it like a reader who goes back and re-reads the middle chapters of a book three times to understand the plot better, without the book getting any thicker. This design creates 48 logical layer applications per token while storing only 24 layers’ worth of weights.
It also uses a Mixture-of-Experts (MoE) design. Each layer has 128 experts, but only 8 are active per token. The model was initially trained with 384 experts; two-thirds were pruned during mid-training, leaving the 128 most-used experts. Attention residuals let each layer learn its own mixture of earlier block outputs, and a gated message passes between loop iterations, learning to admit more information during training.
Key Features and Performance Benchmarks
The model has 26.4 billion total parameters but only 3.4 billion active parameters per token. Because the middle layers loop three times, the effective active parameters rise to 5.4 billion per token once those repeated passes are counted. The company notes that the ordinary active-parameter count alone understates the compute required per token.

According to Thinking Machines, Inkling 2 Mini leads every open-weights model under 100 billion total parameters on DeepSWE v1.1 with 48.2% and Vals Index v2.2 with 46.5%. It ties Qwen3.8-27B at 20.1 on the Artificial Analysis Index. Artificial Analysis reports that Mini generated roughly 38% more output tokens than Qwen on its index tasks, while Qwen used about 2.8 times as much compute under the evaluator’s proxy of active parameters multiplied by tokens. The company acknowledges gaps: the model trails Qwen3.8-27B on multi-turn chat and on instructions with many constraints.
In a demonstration, the model ran offline on a MacBook Pro in airplane mode during a flight and built a playable flight simulator in a few minutes. Another demo used one offline MacBook to power 20 persistent agents in a simulated pixel town called Thinkyville.
Hardware Requirements and Local Deployment Options
The model checkpoint is distributed as a single file of about 20GB in mixed NVFP4 and MXFP8 precisions. Most weights were trained and deployed natively in these formats, so there is no quantization loss.
Running it requires more than the 20GB file size. The model card lists Apple silicon devices with 32GB or more memory (MacBooks, Mac minis, and Studios) as supported. For other GPUs and platforms, 32GB or more of VRAM is recommended. NVIDIA’s DGX Spark is also listed as supported hardware.
Open-source inference is supported from day one on SGLang, vLLM, Ollama, Unsloth, NVIDIA Dynamo, and TokenSpeed. On Apple silicon, the model card lists Ollama and Exo Labs support, and Hugging Face announced MLX support. Quantized GGUF versions are available for llama.cpp via Unsloth. Hugging Face also announced Transformers and Llama App support. For fine-tuning, RL frameworks include SkyRL, Miles, and NVIDIA AutoModel.
Developers can also access the model without a local installation through the Tinker platform for inference and fine-tuning, or the Tinker Playground for inference-only chat.
Intended Use Cases and Target Audience
Inkling 2 Mini is built for developers building AI-powered applications. It is also suitable for general-purpose conversational use, instruction-following, and other natural language and visual tasks. The Apache 2.0 license allows for commercial use, modification, and integration into third-party products.
The release follows Inkling 2 Small, a 594 billion parameter model that some users found too large to run locally. Mini is the direct answer to that feedback. Thinking Machines is working with the National Deep Inference Fabric to bring the model to interpretability researchers and gave Goodfire early access to study how looped models work internally.
How It Compares to Other Open-Weight Models
Inkling 2 Mini sits in the 27 billion parameter class alongside models like Qwen3.8-27B. Its looped architecture is a structural difference: it achieves 48 logical layers from 24 physical ones, keeping the memory footprint around 20GB while applying more compute per token. Traditional small models typically trade reasoning depth for size; the loop structure attempts to blur the line between local models and high-performance cloud APIs.
On benchmarks, it leads the under-100B open-weight class on DeepSWE, Vals Index, BrowseComp, DeepSearchQA, and MCP Atlas. It matches Qwen3.8-27B on the Artificial Analysis Intelligence Index. The trade-off appears in multi-turn chat and complex constraint following, where Qwen currently holds an edge. The model uses search and tools to compensate for the limited knowledge a small model can store.
What’s Next for the Model and Its Ecosystem
Thinking Machines also released a variable-loop research variant, TML-d24-varloop-research, on Hugging Face. In experiments, the variable-loop model’s no-chain-of-thought time horizon grew from 16 seconds at one pass to 37 seconds at three passes; the written chain of thought got shorter but stayed legible, and scores on agentic sabotage tests did not trend higher as loops were added. Hugging Face says it tested that variant but has not evaluated it in depth. The standard Mini model uses three fixed passes.
Training comparisons showed an unpruned non-looping model cost 7.5% less compute to train but required 140% more inference compute to match the looped version’s loss on 512,000-token agentic traces. The model trained on 1 trillion tokens distilled from a larger internal Inkling model after expert pruning.
Sources
- thinkingmachines/Inkling-2-Mini · Hugging Face
- Thinking Machines releases Inkling 2 Mini open-weight model
- Inkling 2 Mini arrives for local AI, with its middle layers on triple duty
- Inkling 2 Mini: 26.4B Open-Weight Model, Apache 2.0
- Thinking Machines Lab Releases Inkling 2 Mini for Local Devices
- Local AI Transformed by Loop Structures: Why the 26B Model Rivals Qwen|Lucky AI Lab
Photo: Parker Byrd parkerabyrd (CC0)





