So Ive been running LLMs on my own hardware for a while now but these new DeepSeek models are just absolute units and my dual 3090 setup is basically crying at this point. I was thinking of just throwing it on AWS SageMaker because thats what we use at work but the pricing is just insane for what it is and I honestly cant justify those bills for a side project Im trying to get ready for a demo next Friday. My logic was to maybe use something like RunPod or Lambda Labs but every time I check they have zero A100 availability or the spot instances I get just die after twenty minutes which is driving me crazy when Im trying to test long-context stuff.

I looked at Together AI and Groq too but I kinda need more control over the actual instance because Im doing some weird custom stuff with vLLM and a specific quantization. I have about 400 bucks to blow on this for the month but I need something reliable in the EU since my client is in Berlin and the latency from US-East-1 is making the chat interface feel sluggish as hell. Does anyone actually have a good experience hosting the full 67B or the new R1 models without breaking the bank or dealing with constant downtime? I looked at Vast.ai but the security of renting some random persons rig makes me a bit nervous for client data...