What is the best GP...
 
Notifications
Clear all

What is the best GPU for running DeepSeek-V3 locally?

5 Posts
6 Users
0 Reactions
154 Views
0
Topic starter

So I saw this thing on Twitter about DeepSeek-V3 and how it basically beats everything else out there and I am so hyped to try it out for myself. I usually just use ChatGPT but I really want to move everything onto my own computer so I dont have to worry about my data or paying 20 bucks every month. The problem is I have literally zero idea how any of this works lol. I went to the store and the guy was talking about teraflops and VRAM and my head just started spinning.

I think I need a really good graphics card but I dont know which one is actually going to work for this specific model because I heard its like... really big? Like hundreds of gigabytes big? How do people even fit that on a computer? I have about 1800 dollars saved up from my tax return and I was thinking about driving down to the Micro Center this weekend to just buy everything but I dont want to walk in there and get scammed into buying something that cant even open the file.

I mostly want to use it to help me brainstorm ideas for this sci-fi novel Im working on and maybe help me learn some basic python coding stuff. Someone online said I need two 3090s but another person said just get a 4090 and I am just so lost. Do I need to worry about the memory on the card more than the speed? And what is a quantization? Is that like a settings thing? Sorry if these are really basic questions I just really want to get this right the first time so I dont waste my money. Is there a specific card or maybe a combo of cards that would let me run this thing at a decent speed without it taking like ten minutes to generate a single sentence? Im trying to get this all set up before my vacation starts in two weeks so I really need to know what I should be looking for...


5 Answers
12

To add to the point above: unfortunately, $1800 isnt gonna get you anywhere near running the full DeepSeek-V3 locally. Its a massive 671B parameter model and even with heavy quantization, you are looking at needing hundreds of gigabytes of VRAM. That just isnt happening on a home budget right now. I had issues with trying to run even mid-sized models on a single card and the generation speed was honestly painful. If you really want to build something this weekend, here is the cold truth:

  • A single NVIDIA GeForce RTX 4090 24GB costs basically your whole budget and still wont fit the main model.
  • Your best bet is finding two used NVIDIA GeForce RTX 3090 24GB cards to get 48GB total VRAM, but even that only handles the smaller distilled versions.
  • You will need a massive power supply like the EVGA SuperNOVA 1300 G+ 1300W to keep two cards from crashing your PC. Honestly, its not as good as expected when you have to strip the model down so much just to make it boot. You might be disappointed by the quality loss.


11

> To add to the point above: unfortunately, $1800... Regarding what #2 said, I actually disagree. Im super happy with my dual NVIDIA GeForce RTX 3090 24GB setup; it works well and dont forget you can supplement VRAM with system RAM.


3

Can vouch for this


1

TL;DR: prioritize vram over speed. honestly dont get caught up in the hype because if you dont have enough memory, the model wont even run. DeepSeek-V3 is massive and you'll definitely need to use a quantized version to make it fit. im really happy with how my setup works now but just be careful not to buy a single card thinking it can handle the full model by itself...


1

👆 this


Share: