Granite Switch 4.1 on Apple Silicon: chat, decisions and a safety judge from one 8B model
Paris Michalas, Partner, NetProjects (Internet Projects Ltd)
IBM's Granite Switch 4.1 puts three jobs into one checkpoint. Ask it normally and it is Granite 4.1 8B, a capable multilingual chat model. Switch on its decision adapter and it reads a situation and a question with labelled options, and answers with a probability for each option in a single step: which team should take this ticket, is this page about a beach or a town, should this step go ahead. Switch on guardian-core and it judges whether a piece of text meets a safety criterion, answering yes or no with a confidence.
What makes this practical is how the adapters are built. They are activated LoRAs: the model only changes behaviour from the adapter's control token onwards, and everything before it runs on the unchanged base model. A long document read once can serve a chat reply, a routing decision and a safety check without being reprocessed.
What we built
IBM released Granite Switch 4.1 on 14 October. We have ported it to MLX, Apple's machine-learning framework, and published 8-bit and 4-bit builds on Hugging Face with a small open-source package on GitHub. The 8-bit build is 9.4 GB, generates 56 tokens per second on a MacBook Pro M5 Max and makes a decision on a 1,200-token prompt in about 0.6 seconds; on 1,071 real decisions it gives the same answer as the full-precision model 99.1% of the time. The 4-bit build is 5.2 GB and generates 98 tokens per second, for Macs with less memory.
| Build | Size | Chat generation | Decision on a ~1,200-token prompt | Same decision as bf16 (1,071 real decisions) | Peak memory |
|---|---|---|---|---|---|
| 8-bit (recommended) | 9.4 GB | 56 tokens/s | 0.62 s | 99.1% | 9–10 GB |
| 4-bit | 5.2 GB | 98 tokens/s | 0.55 s | 93.3% | 5–6 GB |
A short decision (about 80 tokens) takes about 65–70 ms. All numbers: MacBook Pro M5 Max (128 GB), no other models running, one request at a time.
Does the port match the original?
On IBM's own example, routing a customer's failed-payouts ticket to one of three teams, our 8-bit build puts 97.9% on the payments team, against 98.2% in IBM's published result. The guardian adapter scores a harmful request at 1.00 and a harmless one at 0.00, in the exact reply format IBM documents. On test models, the MLX code matches IBM's own reference implementation to within four parts in ten million.
Try it
Using it takes one install and one line per job:
import granite_switch_mlx as gs
model, tok = gs.load("netprojects/granite-switch-4.1-8b-mlx-8bit")
gs.decide(model, tok, state="...", question="Which team should handle this ticket?", options=["Payments", "Login", "Other"])
gs.guard(model, tok, "text to check")What we are releasing
The builds are text only, and the code and weights are open under Apache-2.0, like IBM's original. Granite Switch joins the other decision models we run on our Macs, H2O-Lightning-4B and AutoTrust's JEV models. Not affiliated with IBM.
Credits
Granite Switch 4.1 8B and its decision and guardian-core adapters are by IBM. NetProjects is not affiliated with IBM.
Links
- Code on GitHub: https://github.com/internet-projects/granite-switch-mlx
- 8-bit build (recommended): https://huggingface.co/netprojects/granite-switch-4.1-8b-mlx-8bit
- 4-bit build: https://huggingface.co/netprojects/granite-switch-4.1-8b-mlx-4bit
- IBM's original model: https://huggingface.co/ibm-granite/granite-switch-4.1-8b
- IBM Granite Switch project: https://github.com/generative-computing/granite-switch
- Our port of H2O-Lightning-4B on a Mac: https://netprojects.com/insights/h2o-lightning-4b-on-apple-silicon-mlx
- JEV-35B and AutoTrust-JEV-27B on a Mac: https://netprojects.com/insights/autotrust-jev-decision-models-on-apple-silicon-mlx
- How we use AI: https://netprojects.com/standards/ai
Paris Michalas is a partner at NetProjects, the trading name of Internet Projects Ltd: https://netprojects.com. This article was drafted with AI assistance, then edited and approved by people on our team. You can read more about how we use AI at https://netprojects.com/standards/ai.