Ollama struggles with DeepSeek weights unfortunately. Honestly, for your NVIDIA GeForce RTX 4090 24GB GDDR6X, just run LM Studio Version 0.3.11 or use vLLM v0.6.3 to keep things stable.

Regarding what #2 said about Tbh vLLM is great but might be overkill, I honestly think you are selling the 4090 short by trying to find the lightest path. If you need stability for a dev project by Monday, you want something that handles quantization and context windows without constant crashes. I have been using ExLlamaV2 0.2.1 with a custom backend and it handles the DeepSeek weights way better than the generic runners.

• Use TabbyAPI 0.1.0 for a fast inference server that talks directly to your local weights.
• Stick to NVIDIA GeForce RTX 4090 24GB GDDR6X for pure CUDA performance over any CPU-based fallback. Keep in mind that vLLM is actually the industry standard for a reason when you are hitting these larger model weights. Dont overcomplicate it just to save an hour of setup.

Tbh vLLM is great but might be overkill if you just need a quick fix before Monday. I personally prefer Mozilla Llamafile 0.8.12 for its sheer simplicity when things just need to run.

• LM Studio is fine but clunky for some architectures
• Llamafile gives you a single binary setup
• KoboldCPP is honestly rock solid for DeepSeek stuff on my NVIDIA GeForce RTX 4090 24GB GDDR6X Give KoboldCPP v1.78.2 a shot, it handles those tricky weights way better than Ollama.