3 articles
We ran the same models on an RTX 5090, a Radeon AI PRO R9700, an RTX PRO 6000, a pair of RTX 5070 Tis, a 64GB mining card and three datacenter GPUs, then put every card on one chart against the price it sells for today. Memory decides what you can run, bandwidth decides how fast, and the fastest card is the worst deal.
A 155GB model on a 96GB card still runs, if it is a mixture-of-experts model and llama.cpp parks the experts in system RAM. We measured fourteen such splits on an RTX 5090, a Radeon AI PRO R9700, an RTX PRO 6000, two RTX 5070 Tis and an H200: what the trick saves, what it costs, why the PC behind the card is what you are waiting on, how well a napkin predicts it, and the twelve-channel EPYC build that gets around it.
We ran a 256GB M5 Ultra through the same benchmark as eight other machines: 22 models, five prompt sizes, two engines, cached follow-ups, crowds and power. It holds 300B-class models no desktop card can and writes them at 39 to 131 tokens per second, loses more speed than any PC as the chat grows, and reads twice as fast on MLX as on llama.cpp. What fits, what does not, and when a DGX Spark, an RTX PRO 6000 or an RTX 5090 is the better buy.