empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF

What Empero AI Actually Built — and How

Empero Qwythos — the 9-billion-parameter reasoning model published by Empero AI on Hugging Face under the repository empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF — has quietly become one of the platform's fastest-rising trending models, accumulating nearly two million downloads (about 1.99M at time of writing) before much of the broader developer community had taken notice. Built on the Qwen3.5-9B base model and post-trained on over 500 million tokens of Claude Mythos chain-of-thought traces, empero qwythos claude ships exclusively as a GGUF artifact engineered for consumer and prosumer hardware — making frontier-grade structured reasoning, plus a one-million-token context window, accessible without cloud API billing.

For developers watching the open-weight space closely, empero qwythos sits at an unusual intersection: it fuses a permissively licensed base model, a distillation-style training pipeline drawing from one of the most capable proprietary reasoning chains available, and a packaging format that runs on laptops, NAS boxes, and single-GPU rigs. Understanding the architecture, the quantization trade-offs, and the v2 upgrade that recently landed is essential for anyone evaluating local inference in mid-2025.

Empero AI is the developer behind the model, operating under the domain empero.org. The organization describes Qwythos-9B as a full-parameter reasoning model — no LoRA adapters, no prefix tuning — meaning every weight in the 9B-parameter network was updated during supervised fine-tuning. The model's lineage, as stated on its model card, runs: Qwen/Qwen3.5-9B-Base → Qwen/Qwen3.5-9B (the instruction-tuned checkpoint released by the Alibaba Qwen team) → empero-ai/Qwythos-9B-Claude-Mythos-5-1M (Empero's post-training pass) → this GGUF quantization.

The post-training corpus is the heart of the project. Empero trained on 500 million tokens of what the model card labels Claude Mythos chain-of-thought traces — high-quality, elaborately structured reasoning examples generated by Claude operating in its most deliberate inference modes. The model card separately references Claude Fable traces (the same lineage that powers Empero's companion Qwable-9B-Claude-Fable-5 release) as part of the broader Empero training program, though the primary dataset identifier in this repository's name is Claude-Mythos-5. Critically, per the model card these traces were produced in-house using Empero's proprietary rethink tool rather than scraped from any public dataset — though Empero has not publicly documented the rethink pipeline's architecture or methodology beyond that description. The result is a model that replicates the structured deliberation pattern popularized by reasoning models: every response opens with a <think>...</think> block before committing to a final answer, giving users complete visibility into the model's reasoning chain.

Architecturally, the model follows the qwen35 decoder specification. Empero's supervised fine-tuning pass was text-only, which means the vision encoder and multimodal projector (mmproj) inherited from Qwen3.5-9B were left unchanged throughout training. That is an important practical caveat: in runtimes that load both the main GGUF and the companion mmproj file, the model may accept image inputs, but image-understanding quality reflects whatever Qwen3.5-9B already possessed — not anything Empero additionally trained. Operators should test vision performance independently rather than assuming it matches the text-reasoning gains.

The 1M-Token Context Window: What It Actually Takes

The headline context figure — 1,048,576 tokens — is enabled via YaRN rope-scaling, which extends the model's native 262,144-token positional encoding by a factor of exactly four. YaRN (Yet another RoPE extensioN) is an interpolation technique that rescales rotary position embeddings so the model can generalize to sequence positions it never encountered during training; per the model card it is enabled by default and baked into all GGUF files Empero ships.

Achieving the full 1M context in practice, however, demands hardware that most developers do not keep on a desk. According to the model card's own guidance:

• A single H100 or H200 GPU handles 256k–512k tokens comfortably within VRAM.
• Reaching the full 1,048,576-token ceiling requires either tensor-parallel multi-GPU inference or aggressive KV-cache offloading to system RAM — a setup that introduces significant memory-bandwidth bottlenecks.
• For consumer hardware, the model card recommends starting at -c 16384 and scaling upward based on available VRAM and unified memory.

The reference command for running the llama.cpp server at near-maximum context, following the model card's -c 1010000 guidance for full-1M usage, is:

llama-server \
  -m Qwythos-9B-Claude-Mythos-5-1M-Q4_K_M.gguf \
  -c 1010000 \
  --port 8080

For the vast majority of real workloads — code review, long document summarization, extended agentic loops — a context of 32k to 128k covers the task, and those ranges are readily achievable on modern consumer GPUs with the right quantization tier. The 1M figure is best understood as a ceiling for specialized retrieval-augmented or ultra-long-document workflows rather than a day-to-day operating point.

GGUF Quantization Options: Picking the Right File

The empero qwythos claude mythos gguf release ships with ten quantization variants across two families: standard text-only weights and MTP-enabled (Multi-Token Prediction) variants that include an additional draft head for speculative decoding. The full comparison is below.

File Variant	Quantization	Size (GiB)	MTP	Best For
Q4_K_M	4-bit K-quant	5.24	No	Recommended default; 8 GB VRAM systems (weights only)
Q5_K_M	5-bit K-quant	6.02	No	Balanced quality/size on 10–12 GB VRAM
Q6_K	6-bit K-quant	6.85	No	High-quality inference, 12 GB VRAM
Q8_0	8-bit	8.87	No	Near-lossless quality, 12–16 GB VRAM
BF16	Full bfloat16	16.69	No	Maximum precision, 24 GB VRAM
MTP Q4_K_M	4-bit + MTP draft head	5.48	Yes	Speculative decoding on 8 GB VRAM
MTP Q5_K_M	5-bit + MTP draft head	6.26	Yes	Balanced quality + throughput
MTP Q6_K	6-bit + MTP draft head	7.09	Yes	High-quality speculative decoding
MTP Q8_0	8-bit + MTP draft head	9.11	Yes	Near-lossless quality + throughput
MTP BF16	Full BF16 + MTP draft head	17.14	Yes	Maximum precision + maximum throughput

Note that the sizes above cover model weights only. Once the KV cache for a non-trivial context window is added on top, a "fits in 8 GB" file like Q4_K_M will use meaningfully more memory at runtime — so leave headroom, or offload the KV cache to system RAM, when you push the context length up.

The MTP draft head enables speculative decoding: a lightweight draft module proposes a batch of candidate tokens in a single forward pass, and the main model verifies (or rejects) them in parallel — lifting tokens-per-second throughput without altering output quality when candidates are accepted. The ~0.24–0.45 GiB size premium over the non-MTP equivalents reflects the weight of this additional draft head. Activating speculative decoding requires a recent llama.cpp build that exposes the --spec-type draft-mtp flag; older runtimes should use the standard non-MTP files to avoid compatibility issues.

Vision capability requires one additional file: mmproj-Qwythos-9B-Claude-Mythos-5-1M-F16.gguf at 0.86 GiB, loaded alongside the main weight file in runtimes that support multimodal GGUF configurations such as llama.cpp and LM Studio.

Performance Numbers and What They Mean

Empero benchmarked Qwythos-9B against the stock Qwen3.5-9B baseline using the lm-eval-harness framework under matched inference conditions. The self-reported deltas are substantial:

• +34 points on MMLU — the standard academic knowledge benchmark spanning 57 subjects across science, humanities, law, and social sciences.
• +30 points on GSM8K-strict — grade-school math problems evaluated under exact-match output criteria.
• +19 points on GSM8K-flex — the same problem set evaluated with normalized output matching, which is more lenient about formatting variations.

These gains are measured relative to the model's own Qwen3.5-9B starting point, not against frontier closed-source models, so they quantify what 500 million tokens of chain-of-thought distillation does to a mid-size open model's reasoning surface. It is worth noting that all benchmark figures are self-reported by Empero, which is standard practice in the open-source ML community where independent third-party evaluations routinely lag model releases by weeks or months. Treat the numbers as credible directional signals pending corroboration by groups such as EleutherAI's LM Eval working group or LMSYS.

Empero also conducted a tool-use evaluation using a seven-prompt internal harness that paired the model with a Python code executor and DuckDuckGo search integration. Per the model card, Qwythos-9B produced source-cited, factually correct answers on all seven prompts (7/7) — recovering four closed-book failure modes that the Qwen3.5-9B baseline could not handle without tool access. This is a deliberately small test suite and should be understood as an illustrative internal benchmark rather than a rigorous agentic capability evaluation; nevertheless, it positions the model as a credible candidate for agentic workflows where the model must autonomously decide when to invoke tools, parse structured results, and synthesize a sourced final answer.

Why it matters: A 9B-parameter model that gains 30+ points on rigorous math benchmarks through chain-of-thought distillation alone — without scaling compute or parameter count — is a meaningful data point in the ongoing debate about whether sophisticated reasoning can be taught rather than simply grown through scale. As frontier model pricing continues to pressure developer budgets, that argument has real and immediate economic stakes.

The v2 Upgrade: Squashing Loops and Restoring MTP

The model card carries a prominent notice: v2 has been released as empero-ai/Qwythos-9B-v2 and empero-ai/Qwythos-9B-v2-GGUF. Empero frames it explicitly as "a hygiene upgrade, not a capability regression" rather than a capability leap, but the fixes address production-blocking issues present in v1:

1. Output looping eliminated. Repetition and output degeneration under greedy or low-temperature decoding fell from a self-reported 6.7% of prompts to 0%. In v1, workarounds like aggressively raising repetition_penalty were necessary; v2 addresses the pathology at the training level rather than papering over it with sampler settings.
2. Benchmark scores held or improved. MMLU, GSM8K, GPQA, ARC, and HumanEval all held at (or above) v1 levels — confirming, per the model card, that the degeneration fix was not achieved by softening the model's output distribution at the cost of accuracy.
3. MTP draft head restored. The native multi-token-prediction module was dropped from a previous export of the GGUF files; v2 restores it so that config and weights agree, meaning speculative-decoding configurations that silently fell back to standard decoding on v1 exports now work as intended.
4. Cleaner identity behavior. The model no longer prefaces unrelated responses with unprompted self-identification; it introduces itself only when directly asked. This is a small but real usability improvement for chat interfaces and agentic pipelines where extraneous preamble inflates token counts and disrupts downstream parsing.

One characteristic that did not change between versions — and is explicitly documented on the model card — is that Qwythos-9B remains intentionally uncensored for research, cybersecurity, red-teaming, biology, chemistry, pharmacology, and clinical applications. The card's own Limitations section states plainly that operators should "add your own application-level review/safety layer for end-user-facing deployments where that matters." Empero presents the uncensored posture as a deliberate design choice for professional and academic users rather than an oversight; compliance with applicable regulations and responsible deployment policies is the operator's responsibility.

Any deployment currently running v1 should migrate to v2: the looping regression alone represents a meaningful reliability risk in production, and the MTP restoration is a free throughput gain on hardware that can use it.

Running Empero Qwythos: A Developer Quick-Start

The model integrates with every major GGUF-compatible runtime: llama.cpp, Ollama, LM Studio, Jan, KoboldCpp, and Docker-based deployments. On a recent llama.cpp build, the fastest path to a local endpoint is to let llama-server pull the weights straight from Hugging Face with the -hf flag and the desired quantization tag, which also exposes an OpenAI-compatible API:

# Serve the model with an OpenAI-compatible endpoint (recent llama.cpp build)
llama-server \
  -hf empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF:Q4_K_M \
  --port 8080

If you prefer to point at a local file you have already downloaded, the equivalent invocation from the model card is a plain llama-cli or llama-server run against the .gguf path:

llama-cli \
  -m Qwythos-9B-Claude-Mythos-5-1M-Q4_K_M.gguf \
  --temp 0.6 --top-p 0.95 --top-k 20 --repeat-penalty 1.05 \
  -c 16384

For Ollama users, a single command pulls and runs the recommended quantization directly from the Hugging Face repo:

ollama run hf.co/empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF:Q4_K_M

Empero's recommended sampler configuration aligns with Qwen3.5's official thinking-mode guidance and is designed to keep the <think> block coherent without runaway verbosity:

• temperature: 0.6 — low enough to keep reasoning structured, high enough to avoid degenerate repetition
• top_p: 0.95 — nucleus sampling threshold
• top_k: 20 — vocabulary truncation before nucleus filtering
• repeat_penalty: 1.05 — light repetition suppression (v2 no longer requires aggressive values here)
• max_new_tokens: 16384 — generous budget to accommodate the full <think> preamble plus the final answer, which can be verbose for multi-step problems

To activate speculative decoding using an MTP-enabled variant, the llama-server invocation is:

llama-server \
  -m Qwythos-9B-Claude-Mythos-5-1M-MTP-Q4_K_M.gguf \
  --spec-type draft-mtp \
  --spec-draft-n-max 6 \
  -c 16384 \
  --port 8080

The --spec-draft-n-max 6 flag tells the draft head to propose up to six tokens per speculative step; the main model then verifies accepted tokens in a single forward pass. Tune this value downward if you see high rejection rates on your specific workload.

The license is Apache-2.0, inherited directly from the Qwen3.5-9B base, which permits commercial use without royalties or additional licensing agreements — a meaningful distinction from models carrying custom non-commercial or research-only licenses. The model card credits the Alibaba Qwen team for the base model, ggml-org for the llama.cpp quantization toolchain, and Unsloth for the original GGUF conversion of the vision projector (mmproj), which Empero re-hosts with the vision tower unchanged.

The open-weight ecosystem keeps proving its resilience: distributable, runnable artifacts like GGUF files survive in ways that centralized API access does not. As open data preservation has demonstrated in other contexts, artifacts in community hands outlast the organizations that created them.

Key Takeaways

• Nearly 2 million downloads (about 1.99M) on Hugging Face have made empero qwythos one of the platform's top trending models, reflecting strong developer demand for locally runnable reasoning models at the 9B scale.
• 500 million tokens of Claude Mythos-5 chain-of-thought traces form the post-training corpus, generated in-house with Empero's proprietary rethink tool and applied via full-parameter SFT — not LoRA adapters or prefix tuning.
• Self-reported benchmark improvements over the Qwen3.5-9B baseline: +34 pts MMLU, +30 pts GSM8K-strict, +19 pts GSM8K-flex under lm-eval-harness; independent third-party verification is pending.
• The 1M-token context window is real and enabled by YaRN rope-scaling (4× extension of native 262,144-token encoding), but full utilization requires multi-GPU or heavy RAM-offload infrastructure; 32k–256k is the practical operating range for most users.
• Ten GGUF variants span from a 5.24 GiB Q4_K_M (weights fit in 8 GB VRAM, before KV cache) to a 17.14 GiB BF16+MTP file for precision-plus-throughput workloads, with Ollama and llama.cpp one-liners available for immediate deployment.
• v2 reduces the self-reported output-looping rate from 6.7% to 0%, restores the MTP draft head, holds all benchmark scores, and cleans up unprompted identity behavior — migration from v1 is strongly recommended.
• The Apache-2.0 license permits commercial use without additional agreements; the model is explicitly uncensored for research, cybersecurity, and clinical domains, and the model card itself tells operators to add their own safety layer — placing responsible-use compliance on operators.
• Vision capability is inherited from Qwen3.5-9B's unchanged vision tower — the SFT pass was text-only, so multimodal performance reflects the base model, not additional Empero training. Evaluate vision quality independently.

What Comes Next for Empero Qwythos

The v2 release — with its looping fixes, restored MTP head, held benchmark scores, and cleaner identity behavior — shows that Empero is iterating rapidly on community feedback rather than shipping and moving on. The most pressing next step is an independent benchmark card: self-reported numbers, however internally consistent, carry limited comparative weight until corroborated by groups such as EleutherAI's LM Eval working group, LMSYS Chatbot Arena, or the Open LLM Leaderboard. A transparent third-party evaluation would either validate the +30-point GSM8K claim or contextualize it — and either outcome helps the community.

On the architecture side, the unchanged vision tower remains an obvious gap. A future version that includes image-paired data in its SFT pipeline — fully training the multimodal stack rather than inheriting it passively — would meaningfully differentiate Qwythos from the large field of text-only fine-tunes that carry a vision projector as an inert accessory. The vision projector adds only 0.86 GiB; the real cost is curated multimodal training data, which is the harder problem.

Similarly, publishing or open-sourcing the rethink distillation pipeline — even partially, as a methodology paper — would let the community replicate and extend Empero's approach to other base models and reasoning domains. The 500M-token synthetic dataset is the real intellectual contribution here; making the generation process legible would substantially increase scientific and practical impact beyond what a single GGUF release achieves on its own.

For developers evaluating local inference stacks today, empero qwythos claude v2 deserves a serious trial: the Q4_K_M variant's weights fit in 8 GB of VRAM (leave headroom for the KV cache), the Apache-2.0 license removes legal friction entirely, and the chain-of-thought reasoning capability — when the <think> block is firing correctly — consistently outperforms what a raw 9B parameter count would historically predict. The open-weight ecosystem keeps compressing the gap with proprietary APIs, and models like empero qwythos claude mythos gguf are a concrete, runnable example of how far distillation-driven training has advanced in a single training cycle.