Best vector database for a DeepSeek RAG pipeline?

Question

I'm currently building a RAG pipeline using DeepSeek-V3 for a local documentation project and I'm torn on which vector database to pair it with. I've been looking at Milvus and Pinecone, but I'm worried about latency since DeepSeek is so snappy. My dataset is around 500k chunks, mostly technical manuals, so high-dimensional search performance is key. I'm also trying to keep the setup relatively lightweight if possible. Has anyone here tested DeepSeek with specific databases like Weaviate or Qdrant? I’d love to know which one handles the embeddings most efficiently without breaking the bank on infrastructure. What would you recommend for the best balance of speed and scalability?

hxfeprfvnk · Accepted Answer

Curious about one thing: what's the dimension size of the embeddings you're using with DeepSeek-V3? Basically, if you're hitting 1024 or higher, the compute cost for indexing changes a lot. I've used Qdrant Vector Database before and honestly, the performance was solid, but I had some issues with memory overhead when scaling technical manuals. Before I dive into the technical details, are you planning to host this on-prem or go cloud-native?

vxjwpviifx · Answer

Helpful thread 👍

twdskvlmms · Answer

Since you're aiming for a local setup, I'd be careful about jumping into heavy cloud solutions that might add latency. For 500k chunks, you need something that won't eat your entire budget on infra.

You might want to consider LanceDB OSS. It's an embedded vector store, so it runs right in your process. It's incredibly fast for local RAG because it avoids network overhead entirely, which is great when you're already using a snappy model like DeepSeek.

If you need more traditional reliability, make sure to look at pgvector extension for PostgreSQL. Just be careful with the indexing; you definitely want to use HNSW for a dataset of that size. I would suggest avoiding the older IVFFlat index because it won't scale as well once you pass the 100k mark.

Watch out for metadata bloat. Technical manuals often have long paths or heavy headers. If you don't prune that data, your index size will explode and you'll run into memory issues fast. Keep an eye on your RAM usage, especially if you're running the embeddings on the same machine...