When the Model Doesn't Fit on the GPU
A 155GB model on a 96GB card still runs, if it is a mixture-of-experts model and llama.cpp parks the experts in system RAM. We measured fourteen such splits on an RTX 5090, a Radeon AI PRO R9700, an RTX PRO 6000, two RTX 5070 Tis and an H200: what the trick saves, what it costs, why the PC behind the card is what you are waiting on, how well a napkin predicts it, and the twelve-channel EPYC build that gets around it.