robbyant/lingbot-video-moe-30b-a3b

Robbyant LingBot-Video-MoE-30B-A3B, released on July 9, 2026 by Ant Group's robotics AI team, is presented as the first open-source large-scale Mixture-of-Experts video generation model purpose-built for embodied intelligence — the kind of physical-world reasoning that robots and autonomous agents require. With 30 billion total parameters but only 3 billion active at any moment, it is designed to deliver roughly three times the inference speed of a comparable dense model. At launch, it ranked first on the RBench open-source leaderboard for embodied video generation, according to Robbyant's own published benchmarks.

What Robbyant and the LingBot Platform Actually Are

Most developers encountering the model on Hugging Face will immediately ask: who is Robbyant? The answer has strategic implications for the whole project. Robbyant is a research team associated with Ant Group — the fintech company behind Alipay — with a public presence at technology.robbyant.com, and the model card links its work to Ant's broader AI research efforts. Their stated mission is to build a foundational platform that bridges digital intelligence and the physical world; LingBot is the product family that operationalizes that goal.

The robbyant lingbot ecosystem spans multiple publicly hosted models across several Hugging Face collections. The full family covers video generation, 3D spatial mapping, depth estimation, world modeling, and vision-language-action (VLA) control — essentially every layer of the perception-planning-action stack a robot needs. The robbyant lingbot video series sits at the content-generation and scene-understanding layer, while models like robbyant lingbot-map, robbyant lingbot vla, robbyant lingbot depth, and robbyant lingbot world sit closer to the physical execution layer. Understanding that full stack is key to understanding why the MoE video model matters and why it was optimized for physical rationality rather than pure aesthetics.

The MoE Architecture: 30B Parameters, 3B Active

The "30B-A3B" designation in robbyant lingbot video moe 30b a3b is shorthand for a sparse Mixture-of-Experts design: the model has 30 billion parameters distributed across many expert sub-networks inside a DiT (Diffusion Transformer) backbone, but any given inference token or patch routes through only approximately 3 billion of them. (Robbyant also ships a smaller LingBot-Video-Dense 1.3B variant for lighter workloads.) This is the same efficiency principle used in language models like Mixtral and DeepSeek-V2, now applied to video diffusion.

The practical consequence is a substantial reduction in per-forward-pass compute — Robbyant claims on the order of 3× faster inference compared to a dense 30B video diffusion model — while retaining the representational capacity of a full 30B parameter pool. For developers running video generation workloads where latency and GPU cost are real constraints, that ratio matters enormously: a dense model of equivalent quality would require commensurately more VRAM and compute per forward pass. (The 3× figure is Robbyant's own estimate and will vary with batch size, sequence length, and routing configuration.)

The MoE package also ships with a bundled Refiner component — a separate Refiner DiT that post-processes raw generation outputs to improve visual fidelity. This two-stage pipeline (DiT backbone + Refiner) is increasingly common in high-quality video diffusion, where the base model handles semantics and motion while the refiner handles texture and sharpness.

Supported Tasks

• T2I (Text-to-Image): Generate static frames from natural language prompts.
• T2V (Text-to-Video): Generate video clips from text descriptions, with motion conditioned on physical plausibility.
• TI2V (Text + Image to Video): Condition generation on both a prompt and an input image, enabling animated continuations of real scenes — particularly useful for robot trajectory visualization.
• Refinement: Post-process any of the above outputs with the bundled Refiner model for higher visual quality and sharper textures (available in the MoE package).

Training Data: 70,000+ Hours of Embodied Intelligence

What separates robbyant lingbot video from general-purpose video generators is its training distribution. Per the model card, the model was trained on two categories of data simultaneously: a large corpus of web video (for general visual quality and diversity) and more than 70,000 hours of embodied data — footage explicitly tied to robots, manipulation tasks, and physical-world interactions. This dataset scale is the primary differentiator for physical rationality over models trained entirely on internet video.

A multi-reward training objective reinforces three distinct quality axes during training:

1. High aesthetics: Visual quality and coherence comparable to general video generators.
2. Physical rationality: Generated motion must obey physics — objects do not pass through each other, gravity applies, and contact forces look plausible.
3. Task completion: For embodied scenarios, the generated video must depict a robot actually accomplishing the described action, not merely moving convincingly.

This multi-reward design is a significant departure from standard video generation training, which typically optimizes only for perceptual quality metrics. By explicitly rewarding physical rationality and task completion, Robbyant frames the model as a simulator of physical interaction, not just a visual interpolator. That is intended to make it directly useful as synthetic training data for downstream robotics policies — a use case the team highlights in its documentation.

Benchmark Performance: Topping the Open-Source RBench Leaderboard

Robbyant published results on the RBench leaderboard, a benchmark designed to evaluate video generation models on embodied intelligence tasks. As of the model's July 9, 2026 release date, LingBot-Video ranked first overall among open-source models with an average score of 0.620. The published comparison reports core capability dimensions (manipulation, spatial reasoning, multi-entity interaction, long-horizon, and reasoning) alongside per-embodiment breakdowns (single arm, dual arm, quadruped, humanoid). The table below reproduces the quadruped and humanoid embodiment figures from Robbyant's official model card; consult the model card for the single-arm and dual-arm columns.

Model	Open-Source	Avg.	Manip.	Spatial	Multi-entity	Long-hor.	Reasoning	Quadruped	Humanoid
LingBot-Video (Robbyant)	✅	0.620	0.578	0.643	0.444	0.634	0.505	0.758	0.689
Wan 2.6	❌ (proprietary)	0.607	0.546	0.656	0.479	0.514	0.531	0.723	0.667
Seedance 1.5 pro	❌ (proprietary)	0.584	0.577	0.495	0.484	0.570	0.470	0.680	0.692
Cosmos3 Super	✅	0.581	0.487	0.642	0.444	0.591	0.395	0.739	0.691
Wan 2.2 A14B	✅	0.507	0.381	0.454	0.373	0.501	0.330	0.690	0.648
HunyuanVideo 1.5	✅	0.460	0.442	0.316	0.312	0.438	0.364	0.634	0.595
LongCat-Video	✅	0.437	0.372	0.310	0.220	0.384	0.186	0.681	0.621

Scores sourced from Robbyant's official model card RBench comparison. Proprietary models (Wan 2.6, Seedance 1.5 pro) are shown for context; their full sub-scores are disclosed in the published comparison.

LingBot-Video leads the overall average (0.620), edging out the proprietary Wan 2.6 (0.607) and Seedance 1.5 pro (0.584) — a notable result, since Wan 2.6 is a strong commercial baseline. Its overall lead is driven by strengths in reasoning (0.505 vs. Cosmos3 Super's 0.395) and long-horizon planning (0.634 vs. 0.591), where the embodied training corpus appears to pay dividends. On the manipulation sub-task, LingBot-Video (0.578) is the strongest open-source entry and also edges the proprietary Wan 2.6 (0.546), sitting essentially level with Seedance 1.5 pro (0.577). Robbyant does not lead every dimension, however: Wan 2.6 posts higher spatial (0.656), multi-entity (0.479) and reasoning (0.531) scores, and Cosmos3 Super narrowly leads on the humanoid embodiment (0.691 vs. LingBot's 0.689). The headline claim — best open-source model on RBench overall — holds against the disclosed figures.

Getting Started: Installation and Minimal Usage

The model ships as safetensors files and integrates with the Hugging Face diffusers library. The recommended environment requires Python 3.10 or later. The following installation steps and code examples are drawn from the official model card and GitHub repository.

pip install -U diffusers transformers accelerate

For the full repository — which includes the Refiner, the prompt rewriting pipeline, and advanced inference scripts for T2V and TI2V — clone and install from source:

git clone https://github.com/Robbyant/lingbot-video
cd lingbot-video
python -m venv .venv
source .venv/bin/activate
python -m pip install -U pip
pip install -r requirements.txt
pip install -e .

A minimal text-to-image inference call, as documented in the official model card, looks like this. Note that text-to-video and TI2V generation require additional parameters (frame count, FPS, conditioning image) as specified in the repository README:

import torch
from diffusers import DiffusionPipeline

# Load in bfloat16 precision; substitute device_map="mps" for Apple Silicon
pipe = DiffusionPipeline.from_pretrained(
    "robbyant/lingbot-video-moe-30b-a3b",
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k"
# For T2V, pass additional arguments: num_frames, fps — see README for full API
image = pipe(prompt).images[0]

The model runs in bfloat16 precision by default, and the MoE backend uses grouped expert execution (for example, via LINGBOT_MOE_EXPERT_BACKEND=grouped_mm). For production serving workloads, the repository also includes optional SGLang dependencies for accelerated inference:

python -m pip install --no-deps -r requirements-sglang.txt

The recommended package versions for a reproducible environment are listed in the official model card as follows. These strings are reproduced verbatim from the model card — including the unusual pre-release torch build — so verify them against the current requirements.txt before pinning:

Package	Recommended Version (per model card)
Python	≥ 3.10
torch	2.12.0.dev20260220+cu130
transformers	5.8.1
diffusers	0.39.0
peft	0.19.1
safetensors	≥ 0.4.5
decord	≥ 0.6.0
json_repair	≥ 0.30

These versions are unusually high relative to packages available before 2026, consistent with the model's July 2026 release date. Always verify against the official requirements.txt before pinning a production environment.

Prompt Rewriting Pipeline

The repository bundles two companion models that handle prompt expansion before video generation — reducing the engineering burden on developers who want high-quality outputs without carefully crafted prompts:

• Rewriter-Base: Built on Qwen3.6-27B (the base model named in the model card's download table), this model expands terse natural language prompts into detailed, structured descriptions that the video backbone can use effectively.
• LoRA Adapter: A lightweight adapter that produces structured JSON output suitable for programmatic pipelines, enabling downstream systems to parse scene descriptions, camera movements, and entity behaviors as structured data rather than free text.

This two-stage prompt architecture — rewrite first, then generate — is a practical acknowledgment that end users rarely write prompts that fully specify the physical and temporal details a video model needs for coherent embodied scene generation.

The Broader LingBot Ecosystem: From Video to Physical Robots

The robbyant lingbot video moe 30b a3b model is one node in a considerably larger integrated system. The following subsections describe each major component, its architecture, and its role in the full perception-to-action stack.

Robbyant LingBot-Map: Streaming 3D Reconstruction

The robbyant lingbot-map model is a feed-forward 3D foundation model built on a novel Geometric Context Transformer (GCT) architecture that unifies coordinate grounding, dense geometric cues, and long-range drift correction in a single streaming framework. It performs streaming 3D reconstruction from monocular video at approximately 20 FPS on 518×378 resolution, remaining stable over sequences exceeding 10,000 frames (a featured demo runs a ~25,000-frame, 13-minute indoor walkthrough). Its outputs include camera poses (camera-to-world matrices), dense depth maps, and point clouds — everything a robot needs to anchor itself in 3D space from video input alone.

The github robbyant lingbot map workflow documents three distinct checkpoints:

• lingbot-map — balanced general-purpose checkpoint used in the paper, benchmark, and offline demo.
• lingbot-map-long — optimized for large-scale outdoor or long-corridor scenes where temporal context exceeds what the base model handles efficiently (windowed mode is recommended for sequences longer than ~3,000 frames).
• lingbot-map-stage1 — an earlier training-stage checkpoint that can be loaded into a VGGT model for bidirectional inference, useful for offline batch reconstruction where real-time constraints do not apply.

LingBot-Map is the perception backbone that makes downstream world modeling and VLA control geometrically grounded rather than purely semantic.

Robbyant LingBot-Depth: Geometric Scene Understanding

The robbyant lingbot depth component provides monocular depth estimation, separate from the full 3D reconstruction pipeline of LingBot-Map. Depth outputs from LingBot-Depth are described as being consumed downstream by the LingBot-VLA model via its Dual-Query Distillation mechanism, providing geometric priors that improve the VLA's ability to reason about object distance, reachability, and grasp geometry. It is referenced in the VLA architecture documentation as an upstream cue provider; developers should consult the current Hugging Face collections for its release status rather than assuming standalone availability.

Robbyant LingBot-VA: Visual Affordance Estimation

The robbyant lingbot va component handles visual affordance estimation — predicting where and how a robot can interact with objects in a scene. Affordance maps encode actionable regions: the handle of a cup, the edge of a drawer, the button on a panel. Like LingBot-Depth, LingBot-VA is referenced in the VLA architecture as an upstream cue provider; its standalone availability should be confirmed against Robbyant's current Hugging Face collections. Together with depth, it forms the geometric-semantic perception layer sitting between raw video input and motor command generation.

Robbyant LingBot-World: Interactive World Modeling

The robbyant lingbot world collection contains the LingBot-World-V2 model (14B parameters, causal-fast variant), an image-to-video diffusion model capable of driving 720p video at 60 fps in real time. It uses a dual-agent architecture: a Pilot agent plans character behaviors and a Director agent synthesizes novel environmental elements on the fly. The model is built on the Wan2.2 codebase and uses causal inference with KV caching to support long interaction horizons.

An important licensing distinction applies here: LingBot-World is released under CC BY-NC-SA 4.0, restricting it to non-commercial use — a significant difference from the Apache 2.0 license governing the video generation model. Developers building commercial robotics applications should be careful not to conflate the two licenses when integrating components from the broader ecosystem.

Robbyant LingBot-VLA: Vision-Language-Action Control

The robbyant lingbot vla v2 model (approximately 6B parameters) is the component closest to physical robot execution in the stack. It is a Vision-Language-Action foundation model trained on roughly 60,000 hours of data: about 50,000 hours of robot trajectories spanning 20 distinct robot configurations, plus about 10,000 hours of egocentric human video. Its architecture includes:

• An MoE action expert that enables cross-embodiment generalization across the 20 robot types in the training set.
• A Dual-Query Distillation (DQD) mechanism that integrates geometric cues from LingBot-Depth and semantic temporal priors from DINO-Video, fusing both signal types before action decoding.
• A language conditioning pathway that accepts natural language task descriptions and translates them into low-level motor commands via the fused visual-geometric representation.

The research underpinning LingBot-VLA-v2 has been published on arXiv (identifier reported as 2508.02317), roughly a year ahead of the video generation model's July 2026 release. This timeline reflects Robbyant's methodical, layer-by-layer public release strategy: the action-control research was established first, and the video generation layer that feeds it synthetic training data was released subsequently. Readers should confirm the exact citation and date on arXiv before quoting it.

Why the full stack matters: The LingBot architecture is not a collection of isolated experiments. Video generation (MoE-30B) produces synthetic training data and simulates future states for policy learning. Mapping (LingBot-Map) anchors the robot in 3D space from monocular video. Depth and affordance estimation (LingBot-Depth, LingBot-VA) provide geometric and actionability priors. World modeling (LingBot-World) predicts how scenes evolve under agent actions. The VLA model (LingBot-VLA) translates all of that upstream context into motor commands. This is a vertically integrated, open-source embodied AI platform — and its video generation model is a load-bearing component of the simulation layer, not an ancillary product.

Key Takeaways

• First open-source MoE video model for embodied intelligence: Robbyant positions it as the first open-source model to combine large-scale MoE architecture with a training objective explicitly optimized for physical rationality and robotic task completion.
• Efficiency via sparse activation: 30B total parameters, 3B active per forward pass, targeting roughly 3× faster inference than a dense 30B equivalent (Robbyant's own estimate) — meaningful for any deployment budget or latency requirement.
• Apache 2.0 license: Commercial use is permitted, making LingBot-Video viable for startups and research labs without legal friction. Contrast with LingBot-World's non-commercial CC BY-NC-SA 4.0 terms.
• RBench #1 open-source: Outperforms Cosmos3 Super, HunyuanVideo 1.5, Wan 2.2 A14B, and LongCat-Video on the embodied video benchmark, and edges out the proprietary Wan 2.6 and Seedance 1.5 pro on the overall average (0.620 vs. 0.607 and 0.584), though Wan 2.6 leads on several individual sub-dimensions.
• Prompt rewriting pipeline included: Two companion models (Rewriter-Base on Qwen3.6-27B and a LoRA adapter for structured JSON output) handle prompt expansion, reducing the engineering burden for developers who need consistent, high-quality generation without hand-crafted prompts.
• Part of a complete robotics stack: LingBot-Map (streaming 3D reconstruction), LingBot-Depth (geometric priors), LingBot-VA (affordance estimation), LingBot-World (interactive simulation), and LingBot-VLA (motor control) complete the perception-to-action pipeline.
• 70,000+ hours of embodied training data: This dataset scale is the primary differentiator for physical rationality over general-purpose video generators trained exclusively on web video.

What Comes Next

Adoption is likely to move quickly following the model's release. Several near-term developments are plausible based on community activity and Robbyant's own release trajectory:

• Domain-specific finetunes: Community finetunes targeting specific robot embodiments (e.g., bipedal locomotion, surgical robotics, warehouse manipulation) are the most likely first wave of derivative work, enabled directly by the Apache 2.0 license.
• Quantized variants: Quantized versions at INT8 or INT4 precision would lower the VRAM floor substantially, opening the model to researchers without access to high-memory datacenter GPUs.
• Simulation framework integration: Integration into robotics simulation environments such as NVIDIA Isaac Sim or the Genesis physics engine would let robotics teams use LingBot-Video as a real-time scene synthesizer alongside their policy training loops.
• Standalone LingBot-Depth and LingBot-VA releases: Both components are referenced in the VLA-v2 architecture; their release as independent models would complete the publicly accessible perception layer of the ecosystem.
• Unified video-to-policy training pipeline: The logical culmination of the LingBot stack is a joint training loop in which LingBot-Video generates synthetic rollouts, LingBot-Map and LingBot-Depth annotate them geometrically, and LingBot-VLA trains directly on the resulting labeled data — closing the simulation-to-real loop without requiring additional human demonstration collection.

For developers building on this foundation today, the Apache 2.0 terms on LingBot-Video mean the community can iterate on what Ant Group's team has built without the commercial restrictions that limit components like LingBot-World. The robbyant lingbot depth and robbyant lingbot va components referenced in the VLA architecture suggest that Robbyant is following a deliberate, staged public release strategy — and that the ecosystem visible today represents an early, not final, state of what the team intends to publish.