Choosing and Populating a Vector Database

A team deploys a specialized vector service for 12,000 chunks. Six weeks later, search has to be restricted to the documents visible to the logged-in user, and the permissions live in PostgreSQL. There is no join between the two systems. The team ends up retrieving 200 results from the vector service, then querying PostgreSQL to find out which ones are authorized, and keeping only three. Latency triples, recall collapses whenever a user has few permissions, and 12,000 vectors would have fit in 50 megabytes of memory.

The choice of a vector database is almost always framed as a question of search performance. For the vast majority of projects, that is not the deciding criterion: below a few million vectors, every option is fast enough. What decides it is your ability to filter by metadata, to keep your vectors consistent with your source of truth, and to cleanly delete a document. Choose on these three axes, not on a speed benchmark.

The Three Approaches and Their Domain of Validity

An in-memory index (NumPy, FAISS, hnswlib) is the right answer up to about 100,000 chunks. An exhaustive search over 100,000 vectors of 768 dimensions takes a few dozen milliseconds in NumPy. There is no service to operate, no version drift, and debugging is trivial. The price: the index is rebuilt at startup, and it cannot be shared across multiple processes without extra work.