What is the best GP...
 
Notifications
Clear all

What is the best GPU for running DeepSeek models locally?

4 Posts
5 Users
0 Reactions
276 Views
0
Topic starter

I am so incredibly hyped about these new DeepSeek models that everyone is talking about lately! I saw a few demos of the coding version and it looks like it could literally change my life for this small app project i'm trying to build. I really want to run it locally on my own computer because i'm a bit of a privacy freak and I dont want my data being sent off to some random server somewhere. Plus I think it would just be cool to have it working right there on my desk.

The thing is... I have absolutely no clue what I'm doing when it comes to computer parts. I've got about $900 saved up and I'm planning to head to the store this weekend to pick something up but looking at the GPU aisle makes my head spin. Like what even is VRAM?? Is that different from the normal memory in the computer? I see people mentioning the RTX 3060 or maybe a 4070 but then others say you need like 24 gigs of something to make the big models work and I'm just totally lost. I really dont want to buy the wrong thing and end up with a super expensive paperweight that cant even run the software. Sorry if this is a total beginner question but I'm just starting out and its all very overwhelming. What is actually the best GPU I should be looking for to run DeepSeek at home without breaking the bank?


4 Answers
12

To give an accurate recommendation, which specific DeepSeek parameter size are you looking to run? The memory requirements scale linearly with the model size. A quick tip for your $900 budget is to prioritize the price-per-GB of VRAM. While the NVIDIA GeForce RTX 4060 Ti 16GB GDDR6 is a decent new entry, a used NVIDIA GeForce RTX 3090 24GB GDDR6X often fits your budget and provides the capacity needed for larger models.


11

In my experience, VRAM is the single most important factor for running DeepSeek locally. It is dedicated memory on the graphics card, which is totally different from your regular system RAM. Over the years, I have found that 16GB is the absolute minimum sweet spot for these coding models to run at a usable speed. Without enough VRAM, your computer will basically just crawl. With a $900 budget, you should probably get the ASUS GeForce RTX 4070 Ti Super 16GB GDDR6X. It fits your price range and handles the quantized versions of DeepSeek-Coder really well. If you want to spend less, the MSI Ventus GeForce RTX 3060 12GB GDDR6 is a decent entry point tho you will definitely notice the speed drop. Just make sure you dont buy an 8GB card... it wont work for what you want to do. Honestly, VRAM is king here.


3

Re: "In my experience, VRAM is the single most..." - absolutely, but unfortunately capacity isnt the only thing that matters for reliability. I learned that lesson the hard way with the one I got last year.

  • My previous setup had plenty of memory but the cooling was just not as good as expected.
  • I found that running deepseek for hours literally cooked the components because i didnt account for the sustained load.
  • The system would just shut down right when the model was about to finish a long block of code. It was pretty disappointing tbh. Ngl, i focused so much on the specs that I ignored the power requirements. You really gotta make sure your power supply can handle the spikes because these cards pull way more than the advertised tdp when they're actually working... i learned my lesson the hard way.


2

^ This. Also, honestly, when I first jumped into local LLMs, I made the mistake of thinking raw speed mattered more than memory capacity. I remember picking up an NVIDIA GeForce RTX 4070 12GB GDDR6X thinking it would be plenty because the clock speeds were so high. Boy, was I wrong. As soon as I tried to load a decent coding model like DeepSeek-Coder-33B, it just crashed or dumped everything into my slow system RAM, making it totally unusable... like one word every ten seconds. If you're heading to the store with $900, you really want to target the 16GB tier at a minimum. The NVIDIA GeForce RTX 4070 Ti Super 16GB GDDR6X is probably your best bet for a brand new card. It fits the budget and the 16GB of VRAM is enough to run the smaller DeepSeek models quite fast. If you're feeling adventurous and dont mind looking for a used deal, the NVIDIA GeForce RTX 3090 24GB GDDR6X is the real MVP here. That extra 8GB of VRAM is the difference between running a basic model and running the really smart, high-parameter versions that actually give good coding advice. Basically, more VRAM means you can run higher precision quants or larger models without the system hanging. Just make sure your power supply can handle it tho, those 3090s are thirsty. One thing to watch out for is that companies name cards very similarly. Make sure you see Ti Super on the box if you go the 4070 route. The regular 4070 or even the 4070 Ti only have 12GB, which honestly feels like a waste for AI work. 16GB is where things actually start getting fun for a developer.


Share: