Skip to content
NetProjects
Menu

Insights

NVIDIA's looped Dory 4B model now runs on a Mac

Paris Michalas, Partner, NetProjects (Internet Projects Ltd)

Most language models run each layer once per token. NVIDIA's Nemotron-Labs Dory 1.0 4B reuses a block of its layers: the same weights are applied one to four times, and you choose how many. One set of weights becomes a dial between faster and more careful. NVIDIA released it on 16 October 2026 for NVIDIA GPU software stacks. NVIDIA's model card: https://huggingface.co/nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-BF16

We wanted that dial on a laptop, so we wrote the MLX version and released it on 17 October 2026, the day after NVIDIA. As far as we could find on 17 October 2026, it was the first public MLX build of Dory.

What we built

Does the port match the original?

Yes. NVIDIA's own accuracy results are on NVIDIA's model card, linked above. Our checks show the Mac version says the same thing the original does:

  • Against a second implementation. We wrote a plain fp32 PyTorch version from the vLLM implementation of Dory, separately from the MLX one, and compared outputs on the real weights at two loops over 55 tokens. The top token matched at every position, and the probabilities differed by only 0.001 to 0.003 (KL divergence).
  • 8-bit versus bf16. The 8-bit build gives the same top token as bf16 98% of the time, with a KL divergence of 0.003 from it. On WikiText-2 the perplexity (lower is better) is 12.18 / 11.77 / 11.70 at one, two and four loops, against 12.17 / 11.76 / 11.69 for bf16. The 8-bit build is half the size and is the one we recommend.
  • Caches. With the sliding attention window shrunk to 64 tokens, processing a text in chunks or one token at a time gave the same answers as one full pass (top-token agreement of at least 98.7%, KL divergence at most 0.0006).
  • Loops. More loops lowered the perplexity (12.17, then 11.76, then 11.69 for bf16), so the dial works in the direction NVIDIA describes.
  • Prompt length. We verified prompts of up to 24K tokens.

“A looped model gives you a dial instead of a new model. We wanted that dial on a Mac, and we checked that it still reads the same way as the original.”

— Paris Michalas, Partner, NetProjects (Internet Projects Ltd)

How fast is it?

On a MacBook Pro M5 Max (128 GB), single runs, so treat these as approximate:

Loops8-bit, tokens/sbf16, tokens/sPeak memory with a 24K-token prompt (8-bit / bf16)
1about 35about 277.7 GB / 11.3 GB
2 (default)about 26about 209.0 GB / 12.6 GB
4about 15about 1211.5 GB / 15.1 GB

At the default two loops the 8-bit build writes about 26 tokens a second and needs 9 GB with a 24K-token prompt, which fits comfortably on a laptop. One loop gives you about 35 tokens a second when speed matters more. Dory reasons before it answers, so replies include the thinking.

Try it

pip install "git+https://github.com/internet-projects/dory-mlx"
python -m dory_mlx "Explain why the sky is blue in three sentences." --loops 2

The first run downloads the 8-bit model (4.2 GB) from Hugging Face. The repository README has the commands to convert the weights yourself and to rerun the checks: https://github.com/internet-projects/dory-mlx

What we are releasing

Public: the code (Apache-2.0), the model cards, the converter and the check scripts. Everything above can be reproduced from them.

Credits

Dory is by NVIDIA, released on 16 October 2026 under the OpenMDW-1.1 licence; the MLX builds carry the same licence and link to NVIDIA's model card for the full terms. The MLX port is by Paris Michalas, Partner, NetProjects (Internet Projects Ltd), and the code is Apache-2.0 licensed. NetProjects is not affiliated with NVIDIA. We used NVIDIA's vLLM implementation and the llama.cpp implementation as references, and the MLX and MLX-LM projects.

Paris Michalas is a partner at NetProjects, the trading name of Internet Projects Ltd. This article was drafted with AI assistance, then edited and approved by people on our team. You can read more about how we use AI at https://netprojects.com/standards/ai.