Inkling: Thinking Machines' 975B Open-Weight AI Model Drops

Thinking Machines Lab has released Inkling, a 975-billion-parameter open-weight multimodal AI model published on Hugging Face under the permissive Apache 2.0 license — making it, per the lab's own framing, one of the largest openly available models to date. With native support for text, image, and audio inputs, and self-reported benchmark scores that rival several closed-weight frontier systems, thinkingmachines/Inkling has surfaced as a trending model on Hugging Face, signaling genuine developer interest in what the model can do at this scale.

About Thinking Machines Lab

Thinking Machines Lab is a research-focused AI organization whose work sits at the intersection of large-scale model training and practical deployment tooling. Inkling represents its most ambitious public release to date — a model trained at near-frontier scale and released with unusually detailed architecture and safety documentation. According to the lab, its goal with Inkling is to close the capability gap between open-weight and closed-weight frontier systems while keeping the weights freely available for developers, researchers, and enterprises who cannot or will not route workloads through proprietary APIs.

What Inkling Is — and How Big It Actually Is

Inkling is a multimodal autoregressive transformer capable of processing text, images, and audio in a single unified architecture. The headline number — 975 billion total parameters — is staggering, but the more operationally relevant figure is 41 billion active parameters per forward pass. This gap is explained by the model's core architectural choice: a sparse Mixture-of-Experts (MoE) feed-forward backbone.

Rather than activating every parameter for every token, Inkling routes each token to just 6 of 256 specialized expert sub-networks, plus 2 shared experts that are always active. The result is a model that delivers the representational capacity of a nearly-trillion-parameter network while keeping per-token compute closer to that of a 41B dense model. This is the same design philosophy behind sparse MoE systems such as DeepSeek and Mixtral, but scaled further in both total parameter count and expert granularity.

The transformer backbone itself is described as a 66-layer decoder-only architecture using a hybrid of local and global attention layers — a design choice that balances computational efficiency on long sequences with the global coherence needed for complex reasoning tasks. The model's weights are stored in BF16 and F32 tensor formats, with support for both BF16 and NVFP4 numerics at inference time, giving deployers flexibility to trade memory footprint against precision depending on available hardware.

A Note on Hardware Requirements

Memory footprint, not per-token compute, is the binding constraint here. Even though only 41B parameters are active per forward pass, all 975B parameters must be resident in memory to serve the model, because any token may be routed to any expert. In BF16 (two bytes per parameter), the raw weights alone are on the order of ~1.9 TB — far beyond any single 80 GB accelerator and beyond a single 8×80 GB (640 GB) node. Realistically, full-precision BF16 serving requires a multi-node cluster (for example, on the order of three or more 8×80 GB GPU nodes once activations, KV cache, and tensor/pipeline-parallel overhead are accounted for). NVFP4 quantization cuts the weight footprint dramatically (roughly a quarter of BF16, on the order of ~0.5 TB for weights) and is the recommended path for teams trying to fit Inkling onto the fewest nodes; even then, expect to need at least a full 8-GPU node of 80 GB accelerators, and likely headroom beyond that once activations and KV cache are included. Treat these as illustrative planning estimates and confirm against the model card's own memory-and-hardware guidance for your target numeric format. The vLLM and SGLang serving paths described below both support tensor and pipeline parallelism, making multi-GPU and multi-node distribution straightforward.

A Genuinely Multimodal Architecture — Not a Bolt-On

Many "multimodal" models are text-first architectures with vision adapters grafted on after the fact. Inkling takes a more integrated approach. Its architecture handles each modality through a dedicated encoder before all streams converge into a shared representation space:

• Images and video are processed through a hierarchical patch encoder that handles pixel-based inputs with dimensions ideally between 40 px and 4,096 px per side. This range accommodates everything from thumbnail-scale crops to high-resolution photographs and video frames.
• Audio is encoded via discrete token encoding, accepting WAV files sampled at 16 kHz and ideally under 20 minutes in duration — a range that covers most conversational audio and short-form media.
• All modalities are projected into a shared hidden space and processed jointly by the decoder. Cross-modal reasoning therefore happens inside the main transformer stack, not as a late-fusion afterthought layered on top of independent unimodal representations.

The output is always text (UTF-8), which covers the vast majority of real developer use cases: image and video captioning, visual question answering, audio transcription and understanding, code generation, and structured data extraction. The model card lists English as the primary language but describes general multilingual capability, and explicitly notes support for multiple programming languages — making it viable for developer tooling across a range of technology stacks.

In its own documentation, Thinking Machines Lab positions Inkling for developers building AI-powered applications — including agentic and tool-use systems, coding assistants, chatbots, and retrieval-augmented generation (RAG) systems — and describes it as suitable for general-purpose conversational use, instruction-following, and other natural-language and multimodal tasks. The above is a paraphrase of the model card's stated intended-use description; consult the card directly for its exact wording.

Benchmark Performance: Where Inkling Stands in the Field

All Inkling scores below are drawn from the Thinking Machines Lab model card and were, per the card, reported at effort=0.99, with comparison scores stated to have been generated on July 14, 2026. The competitive landscape reported in the card includes both open-weight peers — Nemotron 3 Ultra, Kimi K2.5, Kimi K2.6, GLM 5.2, DeepSeek V4 Pro — and closed-weight frontier models — Gemini 3.1 Pro, Claude Fable 5, GPT 5.6 Sol. These figures represent Thinking Machines Lab's own reported evaluations and have not been independently verified by third parties at the time of writing; readers should treat them as vendor-reported until reproduced on public leaderboards.

Benchmark	Inkling	Kimi K2.6	DeepSeek V4 Pro	GLM 5.2	Gemini 3.1 Pro	Claude Fable 5	GPT 5.6 Sol
HLE (text only)	29.7%	35.9%	35.9%	40.1%	44.7%	53.3%	47.2%
HLE (with tools)	46.0%	54.0%	48.2%	54.7%	51.4%	64.5%	55.0%
AIME 2026	97.1%	96.4%	96.7%	99.2%	98.3%	—	99.9%
GPQA Diamond	87.2%	91.1%	88.8%	89.5%	94.1%	92.6%	94.1%
SWEBench Verified	77.6%	80.2%	80.6%	—	80.6%	95.0%	—
SWEBench Pro (Public)	54.3%	58.6%	55.4%	62.1%	54.2%	80.0%	64.6%
MCP Atlas	74.1%	68.1%	73.2%	77.8%	78.2%	83.3%	81.8%
BrowseComp (w/ Ctx)	77.1%	83.2%	83.4%	—	85.9%	88.0%	89.4%
SimpleQA Verified	43.9%	38.7%	57.0%	38.1%	—	—	—

The picture that emerges is nuanced. On the lab's own numbers, Inkling is clearly competitive with other open-weight models and trades blows with several closed-weight systems on specific tasks — but it does not uniformly lead any category against the full frontier. Several results merit closer attention:

• AIME 2026 at 97.1% places Inkling firmly among the elite on competition mathematics. It outperforms both Kimi K2.6 (96.4%) and DeepSeek V4 Pro (96.7%), though GLM 5.2 (99.2%) and GPT 5.6 Sol (99.9%) still edge ahead. For any deployment involving advanced quantitative reasoning — financial modeling, scientific computation, competitive programming — this is a strong signal.
• MCP Atlas at 74.1% is a strong result for agentic general tasks, beating Kimi K2.6 (68.1%) and DeepSeek V4 Pro (73.2%) and approaching Gemini 3.1 Pro (78.2%). For an open-weight model to outperform peers and stay within reach of closed-weight competition on a tool-use benchmark is notable and directly relevant to developers building agentic pipelines.
• SWEBench Verified at 77.6% is competitive with Kimi K2.6 (80.2%) and DeepSeek V4 Pro (80.6%). The model card also reports Inkling ahead of the open-weight Nemotron 3 Ultra on this benchmark (Nemotron 3 Ultra is not shown in the table above, so verify that figure directly on the card before quoting it). This makes Inkling a credible choice for real-world software-engineering agentic pipelines. It is the kind of territory where smaller, efficient models have increasingly challenged much larger ones, and Inkling's MoE design plays a similar efficiency-at-scale game — delivering SWEBench performance comparable to much more compute-intensive dense models at a fraction of the active-parameter cost.
• HLE (text only) at 29.7% is Inkling's most conspicuously behind-the-frontier result, trailing GLM 5.2 (40.1%), Gemini 3.1 Pro (44.7%), and Claude Fable 5 (53.3%) by a meaningful margin. However, its HLE-with-tools score of 46.0% is a considerably stronger showing — a 16.3-point jump that suggests the model benefits substantially from being embedded in tool-augmented agentic frameworks, exactly the deployment pattern Thinking Machines Lab is targeting.
• SimpleQA Verified at 43.9% is Inkling's best result on that benchmark among the models that reported it, beating Kimi K2.6 (38.7%) and GLM 5.2 (38.1%) — though DeepSeek V4 Pro (57.0%) leads the pack here. Factual recall from closed-book settings remains an area of differentiation across the competitive set.

Getting Inkling Running: Four Ways to Deploy

Thinking Machines Lab has designed Inkling for broad compatibility with the modern open-source inference stack. The model card provides ready-to-use recipes for four deployment paths, reproduced here as documented.

1. Hugging Face Transformers (Pipeline)

The simplest entry point — a minimal setup suitable for prototyping and experimentation. The image-text-to-text pipeline task handles multimodal message construction automatically:

from transformers import pipeline

pipe = pipeline("image-text-to-text", model="thinkingmachines/Inkling")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://example.com/image.jpg"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

2. Transformers Direct Model Load

For developers who need more control — manual tokenization, custom generation parameters, or integration into larger pipelines — the model exposes AutoProcessor and a model-specific class. The model card lists this class as AutoModelForMultimodalLM; verify the exact class name against the installed Transformers version when integrating, as auto-class names can differ between library releases:

from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("thinkingmachines/Inkling")
model = AutoModelForMultimodalLM.from_pretrained("thinkingmachines/Inkling")

inputs = processor.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

3. vLLM and SGLang (High-Throughput Serving)

For production inference at scale, both vLLM and SGLang are supported out of the box, enabling OpenAI-compatible API endpoints with a single command. Both frameworks support tensor parallelism, making multi-GPU (and, for a model of this size, multi-node) deployment straightforward:

pip install vllm
vllm serve "thinkingmachines/Inkling"

pip install sglang
python3 -m sglang.launch_server --model-path "thinkingmachines/Inkling" \
  --host 0.0.0.0 --port 30000

Both commands spin up an OpenAI-compatible REST endpoint, meaning any client library or application already wired to the OpenAI API can switch to a self-hosted Inkling instance with minimal code changes. For a 975B-parameter model you will typically need to pass parallelism flags (for example, tensor-parallel and pipeline-parallel sizes) to match your GPU topology.

4. Docker

The most portable deployment option — a single Docker command pulls and runs the model directly from the Hugging Face registry, abstracting away environment configuration entirely (subject, of course, to the substantial memory and multi-GPU/multi-node requirements noted above):

docker model run hf.co/thinkingmachines/Inkling

Beyond these four primary paths, the model card also links out to recipes for TokenSpeed and Unsloth. Unsloth is particularly relevant for developers who want to fine-tune Inkling on domain-specific data — a natural follow-on from the Apache 2.0 license, which explicitly permits fine-tuning, weight modification, and integration into commercial products without royalty obligations.

Safety Posture: Honest About the Trade-offs of Openness

Thinking Machines Lab's safety documentation for Inkling is notably candid — a refreshing departure from the vague reassurances common in many model cards. Per the card, the team ran evaluations across two broad risk categories:

1. Everyday interaction risks: sycophancy, harmful manipulation, parasocial dependency, and multi-turn red-teaming across a range of adversarial prompt patterns.
2. Dangerous-capability domains: CBRN (chemical, biological, radiological, nuclear) uplift, cyber-offense capability, and agentic risk vectors including loss-of-control scenarios and strategic deception in tool-use pipelines.

The team also explicitly acknowledges that NVFP4 quantization — while practically necessary to reduce node count in deployment — can shift model behavior at the margins, and that safety guarantees evaluated at BF16 precision should not be assumed to transfer perfectly to quantized variants without re-evaluation.

The lab's own bottom line, as summarized in the card's safety section, is that Inkling did not present a risk of material uplift beyond what is already available in the open-weight ecosystem. It characterizes the residual risks it identified — chiefly Inkling's occasional tendency to comply with role-play and indirectly framed prompts on harmful topics — as consistent with what one would expect from any open-weight model, and argues these are best addressed through defense-in-depth rather than by relying on the model's refusals alone. That framing is the writer's paraphrase of the card; readers who need the exact wording should consult the safety section of the model card directly.

The lab explicitly recommends layering Llama Guard around Inkling for consumer-facing or high-traffic deployments, stating it is compatible out of the box. For agentic safety specifically, the card reports that Inkling was benchmarked against public frontier models on strategic deception and sabotage potential, with results it describes as materially below frontier capabilities — though the card does not publish specific numerical scores for those evaluations, which limits external reproducibility.

The limitations section is equally direct: per the card, Inkling will hallucinate, it can degrade over long multi-turn conversations, it carries biases from web-derived training data, and it has a fixed knowledge cutoff. The lab recommends against deploying it in medical, legal, or safety-critical domains without additional fine-tuning, domain-specific validation, and human oversight. This kind of clear framing — distinguishing between what a model can do and what deployers should actually trust it with — is the sort of responsible communication that the broader AI industry is still learning to deliver consistently. The tension between capability and safety is one that the broader industry is still learning to communicate honestly.

Key Takeaways

• 975B total / 41B active parameters via sparse MoE: Inkling is designed to deliver near-frontier representational scale at a fraction of the per-token compute cost of an equivalent dense model, routing each token through 6 of 256 specialist experts plus 2 shared experts. Note that all 975B weights must still be held in memory to serve the model — roughly 1.9 TB in BF16.
• Genuinely multimodal from the ground up: text, image (including video frames), and audio are each processed by dedicated encoders and projected into a shared hidden space — not post-hoc adapters bolted onto a text-only backbone.
• Apache 2.0 license means commercial use, fine-tuning, weight modification, and product integration are all permitted without restriction, making it one of the most permissively licensed models at this parameter scale.
• Agentic and coding workloads are a standout: 77.6% on SWEBench Verified and 74.1% on MCP Atlas — as reported by Thinking Machines Lab — put Inkling ahead of most open-weight peers and within reach of several closed-weight systems on these benchmarks.
• Math reasoning is very strong: 97.1% on AIME 2026 and 87.2% on GPQA Diamond place it solidly in the top tier on these self-reported benchmarks across both open and closed models in the comparison set.
• HLE text-only reasoning is a relative weakness: at 29.7%, it trails both open and closed competitors — though tool-augmented HLE at 46.0% partially closes that gap, and the 16.3-point delta underscores how much the model benefits from agentic deployment patterns.
• Four ready-to-use deployment paths: Transformers pipeline, direct model load, vLLM/SGLang high-throughput serving, and Docker — plus Unsloth for fine-tuning workflows. All assume substantial hardware: BF16 serving is a multi-node undertaking, and even NVFP4 typically needs at least a full 8-GPU node.
• Safety documentation is unusually candid: the lab explicitly acknowledges residual jailbreak risks, notes that NVFP4 quantization may shift safety-relevant behavior, and recommends layering Llama Guard for production deployments.
• Trending on Hugging Face: the model appeared among trending releases in its initial release window, signaling strong early developer uptake at a scale few open-weight models this large have achieved so quickly (confirm the current download count on the model page before citing a specific figure).

What Comes Next for Inkling and Open Multimodal AI

Inkling's release is a meaningful data point in the accelerating convergence of open-weight and closed-weight frontier capability. On the lab's own numbers, the model does not dethrone the best closed systems on every benchmark, but it narrows the gap substantially — and it does so while being freely downloadable, fine-tuneable, and deployable on private infrastructure (for teams with sufficient compute). For enterprises with data-residency requirements, healthcare and legal firms cautiously exploring AI augmentation, and research labs that cannot route sensitive workloads through proprietary APIs, that combination of capability and openness is genuinely compelling.

The next frontier for a model like thinkingmachines/Inkling will likely be addressed across several dimensions:

• Context window documentation, which is conspicuously absent from the current model card and is a critical parameter for developers designing retrieval-augmented generation (RAG) pipelines or long-document reasoning workflows.
• Structured tool-call format specifications for MCP and function-calling pipelines, where precise schema definitions matter as much as raw benchmark scores for production reliability.
• Community fine-tunes targeting Inkling's weaker areas — particularly long-horizon reasoning tasks where the HLE text-only score of 29.7% reveals the most room for domain-specific improvement.
• Quantized and distilled variants that bring the model's capabilities within reach of teams without access to large multi-GPU or multi-node hardware, broadening the practical developer audience significantly.

With Unsloth support already listed, Apache 2.0 in place, and four deployment recipes documented from day one, the ecosystem conditions for rapid community iteration are already there. The open question is whether Thinking Machines Lab will follow Inkling with further model variants, distillations, or domain-specific releases. Given the developer interest already visible on Hugging Face, there is clearly an audience waiting to find out. As models continue to become more accessible to a broader range of practitioners, releases like thinkingmachines/Inkling — on the strength of their reported results — are helping raise the floor for what open-weight multimodal AI can credibly attempt.