Skip to content
NetProjects
Menu

Insights

harness-hub's Qwen3.8-27B for Hermes Agent on a Mac: MLX and GGUF builds

NetProjects

harness-hub trained a LoRA adapter for Alibaba Cloud's Qwen3.8-27B with reinforcement learning (AsyncGRPO in TRL), so that it works as the model behind Hermes Agent on software-engineering and terminal tasks. harness-hub reports SWE-bench Pro (hard 51) rising from 29.4% to 43.1% and Terminal-Bench 2.1 from 44.7% to 58.8% against the base model, using under 40% of the base model's turns on the tasks both solve. With thinking off, it says, the adapter is level with the base model thinking at its highest effort while generating 3.5 to 4.4 times fewer tokens. These are harness-hub's figures, from runs under Hermes Agent with thinking off; we have not reproduced them.

We merged the adapter into Qwen3.8-27B, kept the vision encoder and Qwen's own multi-token-prediction (MTP) layer, and published ready-to-run builds for Macs with Apple Silicon (MLX) and for llama.cpp (GGUF), plus the adapter on its own for mlx-lm. Everything is under the Apache-2.0 licence. Sister ports do the same for the pi and OpenCode agents: https://netprojects.com/insights/harness-hub-qwen3-8-27b-pi-adapted-coding-agent-mlx-gguf

Always up to date. We keep these builds current on Hugging Face. When harness-hub releases a new version of the adapter, we rebuild, re-test and publish it there, so the links on this page always point to the latest version. The numbers in this article were measured on the version named in each section; the model cards on Hugging Face carry the current figures.

What we released

Merged from harness-hub's adapter at revision 09fb741 and Qwen3.8-27B at revision 3756181. The MLX builds were converted with mlx-vlm and use affine quantisation with group 64; in the 3-bit build the token embedding, output layer and the more sensitive layers stay at 4-bit, and the 2.5-bit build is text only. Peak memory is for an 8K-token prompt with mlx-lm (text).

Hugging FaceSizeSame next token as bf16GSM8K, 50 questionsPeak memory
netprojects/Qwen3.8-27B-hermes-adapted-MLX-6bit (recommended)23.1 GB98.5%46/5023.4 GB
netprojects/Qwen3.8-27B-hermes-adapted-MLX-8bit30.0 GB99.2%45/5029.7 GB
netprojects/Qwen3.8-27B-hermes-adapted-MLX-4bit16.3 GB96.1%43/5017.1 GB
netprojects/Qwen3.8-27B-hermes-adapted-MLX-3bit13.8 GB91.7%45/5014.6 GB
netprojects/Qwen3.8-27B-hermes-adapted-MLX-2.5bit (text only)11.8 GB85.4%44/5013.8 GB
netprojects/Qwen3.8-27B-hermes-adapted-MLX-bf1655.6 GBreference45/5053.4 GB
netprojects/Qwen3.8-27B-hermes-adapted-MLX-adapter638 MBLoRA only, for mlx-lmon any Qwen3.8-27B MLX buildnot measured

The GGUF repository, netprojects/Qwen3.8-27B-hermes-adapted-GGUF, has Q6_K (22.4 GB, recommended), Q8_0 (29.0 GB), Q4_K_M (16.8 GB), Q3_K_M (13.8 GB), IQ3_XXS (11.6 GB) and IQ2_M (10.5 GB), each with the MTP layer, plus a 0.9 GB vision projector for image input. Compared with the BF16 GGUF, they pick the same next token on 98.9%, 99.7%, 94.8%, 92.9%, 92.4% and 89.8% of positions.

How well it matches the original

Measured on 19 November 2026, MacBook Pro M5 Max (128 GB), with mlx 0.32.4, mlx-lm 0.32.2 and mlx-vlm 0.7.11.

  • Merge: all 256 adapted modules equal the base weight plus 2.0 times the LoRA product, rounded to bf16, checked element by element against an independent computation. Every other tensor is the same as in Qwen3.8-27B, byte for byte.
  • Quantisation: each quantised MLX build is exactly the affine (group 64) quantisation of the bf16 build.
  • Agreement with bf16, on 12 short teacher-forced prompts (953 positions for the 8-bit, 6-bit and 4-bit builds): 8-bit 99.2%, 6-bit 98.5%, 4-bit 96.1%, 3-bit 91.7%, 2.5-bit 85.4%.
  • Tool calling: with the included chat template and thinking off, every quantised MLX build and every GGUF file produced a well-formed bash tool call on an agent-style prompt.
  • Coding: three short Python tasks run against asserts, 3 of 3 on the 8-bit, 6-bit, 4-bit and 3-bit MLX builds and on every GGUF file, 2 of 3 on the 2.5-bit MLX build.
  • Vision: the 8-bit, 6-bit, 4-bit and 3-bit MLX builds and the GGUF files with the vision projector described a test image correctly.
  • Source files: every file of both pinned upstream revisions matches the Hub's SHA-256 hashes.

Try it

Keep thinking off: the adapter was trained and evaluated that way. On a Mac with Apple Silicon, serve it with mlx-lm's OpenAI-compatible server and point Hermes Agent at it as a custom provider; Hermes Agent needs at least 64,000 tokens of context.

pip install -U "mlx-lm>=0.32.2"
mlx_lm.server --model netprojects/Qwen3.8-27B-hermes-adapted-MLX-6bit --port 8000 \
  --chat-template-args '{"enable_thinking": false}'

The model card has the Hermes Agent configuration, image input with mlx-vlm and the MTP command: https://huggingface.co/netprojects/Qwen3.8-27B-hermes-adapted-MLX-6bit. The GGUF card has the llama.cpp commands: https://huggingface.co/netprojects/Qwen3.8-27B-hermes-adapted-GGUF

Honest limits

  • We did not run SWE-bench Pro, Terminal-Bench or any other agent test suite on these builds, so the agent scores above are harness-hub's, from its own setup.
  • Our checks are short: 12 prompts for agreement, one tool-calling prompt, three coding tasks and 50 GSM8K questions.
  • All figures come from one machine. We did not time these builds, so we give no speed figures, with or without MTP.
  • Peak memory is for an 8K-token prompt; Hermes Agent's 64,000-token context needs more.
  • The 3-bit and 2.5-bit builds differ noticeably from the bf16 weights, and the 2.5-bit build has no image input.

Licence

Qwen3.8-27B, harness-hub's adapter and our builds are all under the Apache-2.0 licence. Each of our repositories carries the licence text and a notice that describes the conversion and credits harness-hub and the Qwen team (Alibaba Cloud).

Links

Credits

The adapter is by harness-hub; Qwen3.8-27B is by Alibaba Cloud's Qwen team, under the Apache-2.0 licence. The MLX and GGUF builds are by NetProjects (Internet Projects Ltd). Not affiliated with Hugging Face, harness-hub or Alibaba.

This article was drafted with AI assistance, then edited and approved by people on our team. You can read more about how we use AI at https://netprojects.com/standards/ai.