22/07/2026 6:25 pm
Topic starter
Notifications
Clear all
0
Ive been running local LLMs on my dual 3090 setup for years now, mostly sticking to Llama 3 derivatives and some Qwen stuff, but DeepSeek V4 Pro is giving me a massive headache. I thought I could just quantize the living daylights out of it and throw it at my VRAM but the performance is just abysmal. Im hitting bottlenecks I havent seen before even with decent memory bandwidth.
Im based in London and my budget is around 3k GBP, I was thinking about grabbing a used A6000 or maybe trying to build out a workstation with more PCIe lanes if I need to split the model across more cards, but honestly im not sure if the architecture even likes that. I need this up and running for some local data analysis tasks by next month. Has anyone actually managed to get this running with decent token gen speeds? Like what are you guys using for the memory backend, just standard llama.cpp or are you doing something custom with vLLM? Im just curious what the actual bare minimum hardware footprint looks like for this without it taking a decade to generate a sentence...
3 Answers
11
22/07/2026 7:30 pm
Been thinking about your setup since I saw your post. Honestly, throwing more money at an NVIDIA RTX A6000 48GB GDDR6 might be overkill if you dont sort out the memory bottleneck first. Splitting these massive models across cards is rough because the interconnect speeds kill your token gen if you aren't using something like Supermicro X12DAI-N6 Motherboard Dual LGA 4189 for proper PCIe lane distribution. Instead of just more VRAM, try these steps:
- Ditch standard llama.cpp and force vLLM with tensor parallelism
- Optimize your power limits on those NVIDIA GeForce RTX 3090 24GB GDDR6X cards to prevent thermal throttling
- Use a faster NVMe like the Samsung 990 PRO 2TB PCIe 4.0 NVMe M.2 to speed up model loading times Honestly, the bottleneck is usually the bus speed, not just raw VRAM capacity. If you fix your PCIe layout, those 3090s still have plenty of life left.
10
22/07/2026 6:40 pm
> I was thinking about grabbing a used A6000 or maybe trying to build out a workstation with more PCIe lanes if I need to split the model across more cards Coming back to this, I actually tried running similar massive models on a pair of NVIDIA RTX 3090 24GB GDDR6X cards before switching to a single NVIDIA RTX A6000 48GB GDDR6 setup. Honestly, the single card experience is way smoother because you arent fighting the sync overhead. Sure, the A6000 is slower on raw compute compared to the 3090, but the 48GB of VRAM is a total game changer for quantizing. Another option I messed around with was the NVIDIA Tesla A100 80GB HBM2e, but finding a decent cooling solution for those is a nightmare. If you want speed, vLLM is definitely the way to go over llama.cpp for this. Im satisfied with the performance now, but it took a bit of fiddling with the context length settings.
3
22/07/2026 6:25 pm
Honestly, that dual setup is likely hitting a bandwidth wall due to PCIe lanes. For a budget under 3k, definitely skip the new gear. I would suggest finding a used NVIDIA RTX A6000 48GB GDDR6 since the VRAM capacity is essential here.
- Ensure your motherboard supports x8/x8 bifurcation
- Use vLLM for better memory management
- Watch your PSU headroom Be careful with the thermals, those blower cards get loud fast.
