> I've been looking at AWS and Azure, but I'm worried about GPU availability

I went through this last year!! Stumbled upon this and honestly, I tried scaling DeepSeek-R1 671B on Azure NDv4-series vs Together AI GPU Clusters. Azure was super stable but sooo pricey. Together AI was literally fantastic for cost-efficiency, though I found there availability kinda tricky during peak hours. its been a wild learning curve for me but amazing stuff overall! 👍

In my experience, trying to host the 671B model on your own infra is basically a nightmare unless you've got a massive budget. I actually tried setting up DeepSeek-R1 on Amazon EC2 P4d Instances a few months back for a client project, and honestly... the costs just spiraled. Managing a cluster of A100s or H100s is a full-time job lol.

For production-grade stuff where you need to scale without breaking the bank, I'd seriously look at Together AI GPU Clusters or Fireworks AI Inference Engine. These serverless providers are way more cost-efficient because they've already optimized the heck out of the weights. I've found Together AI to be super reliable for throughput. Using dedicated instances on AWS or Azure is cool for privacy, but for raw performance-to-price ratio? Serverless is the way to go imo. It kinda depends on your traffic, but for most apps, just use an API and save yourself the devops headache!! gl!

Following

Tbh, I’m gonna have to respectfully disagree with the trend of jumping straight to those managed API wrappers for a 671B beast. In my experience, while the "pay-per-token" model looks attractive on paper, you’re basically trading away control over your inference optimization. I’ve been tracking the market shift recently, and many high-scale apps are hitting walls with those providers because of opaque queuing and shared resource contention. I actually had to migrate my current setup away from a major cloud provider because the overhead was insane. We looked at the numbers and realized that for a model this size, the markup on managed services is HUGE. I ended up moving our workloads to a specialized GPU-focused provider that gives us direct hardware access. It allowed us to implement our own quantization and custom vLLM configurations, which cut our latency significantly. It’s definitely more work to manage the orchestration yourself, but if you want consistent performance without the "noisy neighbor" issues common in serverless environments, owning the stack is the only way to go imo. It’s a bit of a "build vs buy" dilemma, but for DeepSeek-R1 at scale, building your own cluster on specialized infra is much more sustainable long-term.

I totally agree with SouthBankStroll here! The loss of control when you go with those managed wrappers is a massive technical debt trap. Its honestly amazing how much people overlook the nuances of resource contention when they see a flashy dashboard. This whole discussion actually reminds me of a time my old roommate tried to build a custom liquid-cooled rig to compare two different enterprise-grade server brands.

• He was obsessed with the thermal benchmarks and Delta T values of these high-end 4U chassis.
• One brand had these fantastic modular 2200W power supplies that clicked in so satisfyingly.
• The other brand used this weird proprietary rail system that supposedly reduced rack vibration by 15 percent according to their whitepapers.
• We spent basically an entire month just analyzing the CFM flow rates of the intake fans because he was convinced it affected the long-term MOSFET lifespan.
• It turned into this huge ordeal where he actually built a custom soundproof room for the servers, but then he realized he forgot to install a dedicated 30-amp circuit for the power draw. The whole thing ended up being a total disaster because the landlord saw the utility bill and almost evicted us... honestly it was such a legendary mess but we learned a ton about cable management!