NVIDIA's Nemotron 3.5 search models on a Mac: embeddings and reranking for text and page images

Search works best when it understands meaning, not just matching words. That is the job of an embedding model: it turns each document and each question into a list of numbers, a vector, so that things that mean the same land close together. A reranker then takes the best candidates and scores each one against the question more carefully.

NVIDIA's Nemotron 3.5 retrieval models do both. Embed 1B and Embed 8B turn text, or page images (one image per document, optionally with text), into vectors for search; the questions are text. Rerank 8B scores how well a document answers a question. They run on NVIDIA's own software. We wanted them on a Mac, so we ported all three to MLX, Apple's machine-learning framework.

Always up to date. We keep these builds and the engine that runs them current on Hugging Face and GitHub. When NVIDIA releases a new version of the Nemotron 3.5 retrieval models, we rebuild, re-test and publish it there, so the links on this page always point to the latest version. The numbers in this article were measured on the version named in each section; the README on GitHub and the model card on Hugging Face carry the current figures and a dated change log.

• Hugging Face: https://huggingface.co/netprojects
• GitHub: https://github.com/internet-projects
• NetProjects MLX builds collection: https://huggingface.co/collections/netprojects/netprojects-mlx-builds

What the models do

Embed 1B gives 2,048-number vectors and Embed 8B 4,096. Both are Matryoshka models, which means the vectors can be shortened, for example to 512 numbers, to save space, at some cost in quality. They read up to 32,768 tokens. They are built on Ministral 3 (3B and 8B) with a Pixtral vision encoder, so a scanned page or a slide can be searched as an image.

What we released

Six MLX builds on Hugging Face, with group size 64. The vision encoder stays at bf16 and the reranker's score head at float32.

We chose not to publish a 4-bit Embed 1B: its quality was worse, and it would have been only 0.6 GB smaller than the 8-bit build. The release also includes an engine with a local server that speaks the OpenAI embeddings format and the Cohere and Jina rerank format, so existing search code can point at it.

How well it matches the original

We compared every build with NVIDIA's own code running in float32, with the same tokens on every input.

Measured on Nemotron 3.5, our release v0.1.0, 30 October 2026, MacBook Pro M5 Max (128 GB).

A cosine of 1 means the vectors point the same way. Long documents (4,000 tokens or more) and page images hold up too: the lowest cosine is 0.9990 or better for every 8-bit build, while the 4-bit Embed 8B drops to 0.9702 on images. For the reranker we compared the order of scores on 400 SciFact pairs: the Spearman correlation is 0.9998 for the 8-bit build and 0.9920 for the 4-bit build.

Search quality

Measured on Nemotron 3.5, our release v0.1.0, 30 October 2026, MacBook Pro M5 Max (128 GB).

ViDoRe v3 Computer Science asks text questions against 1,360 page images. Our builds land on NVIDIA's own figures.

On SciFact, Recall@100 is 0.96 for both Embed 1B builds. Adding Rerank 8B on top of Embed 8B raised ViDoRe nDCG@10 from 0.7952 to 0.8315 on 50 queries. On a SciFact slice it did the opposite and lowered it, from 0.9019 to 0.8847. Reranking is not a free win on every data set, so test it on your own data before you switch it on.

Speed on a Mac

Measured on Nemotron 3.5, our release v0.1.0, 30 October 2026, on a MacBook Pro M5 Max (128 GB), one request at a time, with inputs of about 500 tokens.

With inputs of about 4,000 tokens, Embed 1B manages about 3.7 to 3.9 documents a second. The 8-bit Embed 1B uses 1.75 GB of memory at batch 1. The 4-bit builds save memory and disk, not time.

Try it

pip install git+https://github.com/internet-projects/nemotron-retrieval-mlx

from nemotron_mlx import Embedder, Reranker
e = Embedder("netprojects/Nemotron-3.5-Embed-1B-mlx-8bit")
docs = ["The Acropolis Museum in Athens displays the Parthenon sculptures.",
        {"image": "slide-12.png"}]
D = e.embed_documents(docs)
q = e.embed_queries(["Where can I see the Parthenon sculptures?"])
print(q @ D.T)   # cosine similarity
r = Reranker("netprojects/Nemotron-3.5-Rerank-8B-mlx-8bit")
print(r.rank("Where can I see the Parthenon sculptures?", docs, top_k=1))

To run the local server:

python -m nemotron_mlx.server --embed <repo> --rerank <repo>

The README lists the server options: https://github.com/internet-projects/nemotron-retrieval-mlx

Honest limits

• Questions are text only, and each document can carry one image.
• We tested on ViDoRe v3 Computer Science and SciFact only; the 8B models on a 1,000-document SciFact slice.
• All figures come from one machine.
• The 4-bit Embed 8B is visibly less exact than the 8-bit build, and weaker when its vectors are shortened to 512 numbers.

Links

• Code on GitHub: https://github.com/internet-projects/nemotron-retrieval-mlx
• Embed 1B bf16: https://huggingface.co/netprojects/Nemotron-3.5-Embed-1B-mlx-bf16
• Embed 1B 8-bit: https://huggingface.co/netprojects/Nemotron-3.5-Embed-1B-mlx-8bit
• Embed 8B 8-bit: https://huggingface.co/netprojects/Nemotron-3.5-Embed-8B-mlx-8bit
• Embed 8B 4-bit: https://huggingface.co/netprojects/Nemotron-3.5-Embed-8B-mlx-4bit
• Rerank 8B 8-bit: https://huggingface.co/netprojects/Nemotron-3.5-Rerank-8B-mlx-8bit
• Rerank 8B 4-bit: https://huggingface.co/netprojects/Nemotron-3.5-Rerank-8B-mlx-4bit
• NetProjects MLX builds collection: https://huggingface.co/collections/netprojects/netprojects-mlx-builds
• NVIDIA's original Embed 1B: https://huggingface.co/nvidia/Nemotron-3.5-Embed-1B-BF16
• NVIDIA's original Embed 8B: https://huggingface.co/nvidia/Nemotron-3.5-Embed-8B-BF16
• NVIDIA's original Rerank 8B: https://huggingface.co/nvidia/Nemotron-3.5-Rerank-8B-BF16
• Our earlier NVIDIA port, Dory 1.0 4B on a Mac: https://netprojects.com/insights/nvidia-dory-4b-on-apple-silicon-mlx
• Decision models on a Mac, Fastino GLiDE: https://netprojects.com/insights/fastino-glide-decision-model-on-apple-silicon-mlx
• How we use AI: https://netprojects.com/standards/ai

Always up to date. We keep these builds and the engine that runs them current on Hugging Face and GitHub. When NVIDIA releases a new version of the Nemotron 3.5 retrieval models, we rebuild, re-test and publish it there, so the links on this page always point to the latest version. The numbers in this article were measured on the version named in each section; the README on GitHub and the model card on Hugging Face carry the current figures and a dated change log.

• Hugging Face: https://huggingface.co/netprojects
• GitHub: https://github.com/internet-projects
• NetProjects MLX builds collection: https://huggingface.co/collections/netprojects/netprojects-mlx-builds

Credits

Nemotron 3.5 Embed 1B, Embed 8B and Rerank 8B are by NVIDIA, under the OpenMDW-1.1 licence; our builds carry the same licence, with NVIDIA's LICENSE and NOTICE files kept in every repository. Our code is Apache-2.0 licensed. The MLX port is by Paris Michalas, Partner, NetProjects (Internet Projects Ltd). Not affiliated with NVIDIA.

Paris Michalas is a partner at NetProjects, the trading name of Internet Projects Ltd: https://netprojects.com. This article was drafted with AI assistance, then edited and approved by people on our team. You can read more about how we use AI at https://netprojects.com/standards/ai.