What is the best GPU for running DeepSeek-V3 locally?

Question

What GPU should I grab to run DeepSeek-V3 at home without it taking forever to generate a sentence? Im super stoked to try this model because the benchmarks look insane.

I saw some guys on Reddit saying you can squeeze a 4-bit version onto two RTX 3090s but then I read another thread saying you need at least 80GB of VRAM to even load it properly even with heavy quantization so idk who to believe. I have about $3k saved up and I live in a tiny apartment in Seattle so I cant really build a massive server rack. Is one 4090 enough if I use some crazy compression or am I just dreaming here...

tojpegmjth · Accepted Answer

Unfortunately, a single NVIDIA GeForce RTX 4090 24GB GDDR6X just wont cut it for DeepSeek-V3. The VRAM demands are too high for quality quantization.

You realistically need 80GB of VRAM for 4-bit performance.

Multiple NVIDIA GeForce RTX 3090 24GB GDDR6X cards are the only way, but the heat in a small apartment is brutal. Its disappointing that hardware requirements are this extreme for local use.

NorthLondonDerby · Answer

Running DeepSeek-V3 on one NVIDIA GeForce RTX 4090 24GB GDDR6X isn't feasible for decent performance. With a $3k budget, a methodical approach using dual GPUs is more reliable.

Purchase two used NVIDIA GeForce RTX 3090 24GB GDDR6X units.

Ensure your power supply is at least 1200W. This provides 48GB of VRAM. It's the most budget-conscious way to handle heavy quantization while keeping the build small.

BritishMuseumGeek · Answer

This is exactly what I needed to hear. Youre a lifesaver honestly.