I load models onto a Mac Studio with 64GB of unified memory, and for a while I treated that memory like a shelf I could keep stacking. Then I watched a second large model take the machine down. I had a 35B mixture-of-experts model already wired on MLX, and I asked the machine to load another big one alongside it. At around 51GB wired, the running model Metal-crashed, and the client on the other end saw the connection drop. The runbook I keep for this now says it plainly: 64GB cannot keep two large MLX models wired at once, and the second large load fails with a Metal "Insufficient Memory" error.
That failure reframed the whole problem for me. Model residency on this tier has room for one big occupant, and everything I built after that crash exists to manage that one slot so I stop managing it by hand.
The whole store of local models on this machine totals 57GB on disk, close to the size of the 64GB of RAM. That total includes transcription, embeddings, and text-to-speech weights, not only the large chat models I schedule. Disk is not the constraint. What is resident in memory is the constraint, and I solved it once with tooling instead of re-solving it every morning.
What stays resident in unified memory
Two model servers stay up all the time. The primary is Qwen3.6-35B-A3B quantized to 4-bit on MLX, about 19GB wired, and it does double duty as my default chat model and the primary for a daily ingestion pipeline. Behind it sits a second, slightly smaller dense model, Qwen3.6-27B at Q4_K_M served by llama.cpp's llama-server, about 16GB. That one is the always-on backup for ingestion.
So the honest version of "one big model at a time" is this: one big model occupies the exclusive swap slot, and a fixed dense backup stays resident next to it. The primary at 19GB plus the backup at 16GB plus ordinary apps leaves near zero truly-free RAM. There is no room for a third large model, which is the entire reason the crash happens when I forget.
For a while I misread the backup. It kept looking flaky, going down at odd moments, and I assumed the server itself was unreliable. It was not. The backup was always up. What I was seeing was memory pressure: whenever I loaded a test model on top of the resident pair, I overcommitted the box, and the OS reached for the largest evictable thing and OOM-killed the backup server, which then cold-reloaded. During that window the ingestion backup was down, and I read a headroom problem as an availability problem. Diagnosing that correctly changed what I built next.
A router that switches on the first message
The load-and-unload logic lives in a router I wrote, part of Cerebellum, the local AI router I open-sourced. It presents one OpenAI-compatible endpoint. On each chat request, it ensures the requested model's profile is loaded, deloading the others to fit the 64GB box, then proxies the request to that model's backend. Each large model is its own exclusive memory profile, because one big model at a time is the rule the hardware enforces. The always-on dense backup never triggers a switch; a request for it finds it already up.
The tricky part was the trigger. My chat client gives no signal when I pick a model from the menu. It signals only when I send a message. So the switch has to ride on the request, not the selection. The first message I send to a cold model is what tells the router to swap the seat.
That means the first message pays for the cold load, and a cold 35B load takes long enough (usually 15 to 60 seconds) to hang a chat UI if I let it. During the load, the router streams keepalives so the UI does not stall. Past a time budget, instead of timing out, it returns a short "resend in a moment" message while the model keeps loading in the background, so my retry lands on a warm model. That small piece of stagecraft turns a stall of half a minute or so into two clean sends.
One rule keeps the router honest: it never improvises memory management. Every load and unload decision routes through a single profile-switcher command that is the sole source of truth for what is resident. The router asks that one command to make the seat correct and does nothing clever on its own. When I debugged residency problems, I only ever had to look in one place. Public bug trackers are full of tools that pretend two models fit. One auto-eviction bypass report documents models loaded past the machine's budget when eviction was skipped, which is how I read it, and centralizing the decision was how I avoided writing my own version of that bug.
The swap tool for testing on the full 64GB
Testing a new model is where the one-slot rule bites hardest. If I try to load a candidate while the 19GB primary and the 16GB backup are both wired, I overcommit the box, and one of two things happens: the test load fails with Metal "Insufficient Memory," or the OS kills the backup and cold-reloads it, and my test runs on a machine that is thrashing. Neither gives me a clean read on the candidate.
So I wrote a small swap tool with the subcommands you would expect: status, free, serve, restore. The free step stops the always-on model servers so a test gets the full 64GB, then restore brings them back, no crash and no cold-reload thrash. The detail I care about most: free records which services it stopped, so restore brings back only those. An on-demand model that was already idle before I ran free stays idle after, so I do not leave extra weights resident. A live round trip on 2026-06-29 freed about 25GB and restored the primary, backup, and router with both endpoints responding.
During a test window, both resident models are down, backup included. That used to mean ingestion could fail if it ran in that gap. I hardened the ingestion summarizer to run on the primary alone, so the backup offline for a few seconds no longer fails a run; sources queue and retry. The swap tool creates a clean testing gap, and the pipeline tolerates the gap. Those two changes together are what let me test candidates without babysitting the whole stack.
What the seat has actually held
The point of a clean testing rig is to say no fast, and most of what I have tested got a no. I want to be clear that these are one-box, one-user reads, not published benchmarks, and none of them are a knock on the model authors.
- Ornith-1.0-35B: tried and deleted on 2026-06-26. About 28GB of GGUF weights removed and its serving lane unwired after my evals came out parity-at-best against the resident stack. Parity at higher cost earned it a delete.
- Qwopus3.6-27B, a fine-tune of the same base as my backup: deleted after its claimed speed advantage failed to show up. The model card promised same-quality-but-faster, and I measured about 22 tokens per second, identical to the plain backup.
- Qwen-AgentWorld, 27GB of weights: deleted on 2026-06-26 once testing showed it is a world model, the kind you feed a state and an action so it predicts the next observation, not a chat assistant, so it never belonged in the rotation.
The most recent occupant, a diffusion-based model, I removed on 2026-07-01 by my own decision, weights and serving wrapper deleted, which returned the rotation to the resident pair. The exclusive slot is open again.
One retirement taught me a lesson about tooling versus documentation. A model I had deleted kept reappearing in my model-inventory docs, and I could not figure out why until I found that the daily inventory generator hardcoded the model list. Retiring a model here takes three steps, not one: remove the weights, unwire the serving lane from the router, and edit the tool that regenerates the docs. Miss any of the three and the ghost comes back.
The honest limitation
The clean framing is "one big model at a time," and that is not literally true of the whole machine. Two models are always resident: the 35B MoE and the 27B backup, about 35GB together. The router is a thin proxy that swaps only the big MLX slot; it does not manage the backup at all. The discipline I can defend is narrower than the slogan: one big model in the exclusive swap slot at a time, plus a fixed always-on dense backup, on a machine that cannot wire two large MLX models at once.
That narrower claim is the one worth taking away. On the 64GB tier, model scheduling is an operations problem, and you solve it once with a router that owns the slot and a swap tool that clears the room. The tools I lean on for the serving itself are ordinary and public, MLX for the daily server and llama.cpp for the backup, and the memory-pressure failure modes are documented in the open. One MLX server report walks through an admission guard rejecting ordinary requests as the KV cache grows. I did not invent the constraint. I built the two small pieces that keep me from paying its cost by hand every day.