Vector Search Is Now a Data Type, but the Cost Math Remains

For several years, it appeared that AI stacks would add a dedicated vector database alongside their primary data store. That phase has largely passed. Approximate nearest-neighbor search is now widely available as a column type in databases teams already operate, including Postgres, Elasticsearch, OpenSearch, ClickHouse, MongoDB, and Redis. For many organizations, the question of whether a new database is necessary now has a straightforward answer: no.

What has not changed is the cost calculation. Treating vector search as a data type means storing dense embeddings in an existing engine and querying them with approximate nearest-neighbor (ANN) indexes alongside the filters, joins, and lexical scoring that engine already supports. This removes an operational surface, but it does not eliminate the underlying resource requirements. At scale, costs depend on the number of bytes occupied by each vector, the level of compression, and how much data is allowed to leave RAM. Those factors are largely independent of the vendor.

The Category Moved Into Existing Database Engines

Dedicated vector databases represented a genuine category with substantial engineering behind it. Pinecone launched its managed service in 2021 and helped define the segment. Milvus, Weaviate, and Qdrant emerged around the same period. The original case was reasonable: general-purpose databases did not offer effective ANN indexes, so semantic search needed a specialized system.

Established database engines closed that gap quickly. Elasticsearch added approximate k-nearest-neighbor (k-NN) search using the Hierarchical Navigable Small World (HNSW) graph algorithm, building on Lucene's HNSW implementation. OpenSearch has included a k-NN plugin since its 1.0 release. pgvector added HNSW in version 0.5.0 in August 2023, making a standard Postgres instance a credible vector store. MongoDB Atlas Vector Search reached general availability in December 2023. Redis exposed vector similarity through RediSearch 2.4 in 2022. ClickHouse made its HNSW vector similarity index generally available in version 25.8.

HNSW appears across these systems because it can deliver more than 95 percent recall without requiring a rebuild step.

The consolidation is not primarily about matching feature lists. Production retrieval is rarely a pure vector lookup. Practical systems combine lexical scoring, structured filters such as tenant, language, access control, and freshness, and aggregations with dense retrieval. They may then rerank the combined result set. Keeping those operations alongside the vectors avoids dual writes, a second consistency model, and another system to operate. When an existing engine can store an embedding as another field, a standalone vector database must provide more than recall to justify its additional operational burden.

Memory Drives the Cost

HNSW is a graph traversed through pointer operations. The traversal remains fast only when the graph and its vectors fit in RAM. Once nodes spill to disk, latency becomes dominated by page faults rather than distance calculations. As a result, memory is the main cost driver for vector search at scale, and memory requirements grow with the product of vector count and dimensionality.

OpenSearch documents the per-vector memory requirement for an in-memory HNSW index as:

1.1 × (4 × dimension + 8 × M) bytes per vector

The 4 × dimension term represents the float32 vector, while 8 × M represents the graph's per-node connection overhead. The factor of 1.1 provides approximately 10 percent slack. This means dimensionality, not document count alone, determines the required hardware tier.

The raw figures are significant. One million 1,536-dimensional float32 vectors, the output size of OpenAI's text-embedding-3-large, require about 6.1 GB before graph overhead. Ten million require roughly 60 GB for the vectors alone, with the data kept in RAM for predictable latency. That capacity requirement turns vector search into a hardware-planning problem.

HNSW also adds graph structure. Because it stores neighbor lists at multiple layers, an HNSW index can reach two to five times the size of the raw vectors, while IVFFlat's flat per-cell storage remains close to 1.1 times the raw-vector size.

This explains why common responses to slow vector search, such as adding shards, increasing recall settings, or raising ef_search, can make performance worse. Each adjustment increases memory pressure, query fan-out, or CPU usage. At scale, the slowest shard determines p99 latency.

The recall-versus-throughput relationship is steep and nonlinear. On standard ANN benchmarks, increasing HNSW recall from 95 percent to 100 percent can reduce throughput by roughly seven times. Pursuing the final few points of recall is often the most expensive part of the pipeline, while producing little visible benefit for users.

Quantization and Tiering Change the Cost Curve

If memory is the main cost, compression can reduce it. Quantization decreases each vector's storage footprint by exchanging a controlled amount of precision for a substantial reduction in bytes. It is often paired with rescoring, which reranks the top candidates using full-precision vectors stored on less expensive media. This is where meaningful cost reductions occur, and the technique is largely independent of the database vendor.

The approaches vary in aggressiveness:

• Lucene's int8 scalar quantization reduces memory by about 75 percent and keeps raw vectors on disk for rescoring.
• Binary quantization reduces each dimension to a single bit, reaching a 32-fold reduction. It works especially well with high-dimensional embeddings when combined with oversampling. Qdrant reports 0.98 recall@100 on 1,536-dimensional OpenAI embeddings with four-times oversampling.
• Elasticsearch's Better Binary Quantization reduced a dataset containing 138 million 1,024-dimensional vectors from roughly 535 GB to about 19 GB.

Tiering is another important lever because not every vector needs to remain in RAM. Disk-based ANN keeps most of the index on SSD and stores only a compressed copy in memory. Microsoft's DiskANN can index one billion points on a single 64 GB machine plus SSD, achieving more than 95 percent recall while serving thousands of queries per second.

Object-storage tiers push the cost tradeoff further. Amazon's S3 Vectors targets up to 90 percent lower costs for large vector datasets by exchanging latency for price. The tradeoff is measurable: OpenSearch's disk-based mode showed p90 latency of 96 milliseconds, compared with 24 milliseconds for its in-memory path.

That difference may be acceptable for agentic retrieval inside a multi-second reasoning loop, but it can be unsuitable for interactive autocomplete. At billion-vector scale, the typical design is tiered: hot vectors remain in RAM-backed HNSW indexes, while cold vectors are quantized and stored on disk or in object storage. AWS S3 Vectors and Turbopuffer are examples of systems designed to combine large-scale vector storage with lower costs and acceptable performance.

Where Managed and Self-Managed Systems Differ

Because database engines have converged on similar algorithms and compression techniques, the choice between managed and self-managed deployment is less about basic capability. Both approaches can support HNSW, quantization, and tiering. The main question is who owns the memory-sizing calculations, rescoring and latency tradeoffs, and routing between hot and cold tiers.

Managed services absorb much of the provisioning work. They size RAM, isolate vector traffic on dedicated nodes so it does not compete with operational queries, and hide cold-tier retrieval behind a single API. MongoDB introduced dedicated Search Nodes alongside the general availability of Vector Search for this purpose. Managed deployments reduce operational responsibility, although they provide less access to some configuration controls and include a service margin.

Self-managed deployments provide direct control over the available settings and the associated costs. Operators choose M, ef_construction, and ef_search; select the quantization approach and oversampling ratio; and determine the rebuild schedule. They also inherit the failure modes, including cold queries against on-disk HNSW indexes that trigger page faults during traversal, recall that changes as data distribution shifts, and fan-out behavior that can make excessive sharding a tail-latency problem.

The decision is usually driven by two questions:

1. Does the vector data already reside in an operational store the team runs? If so, adding a vector column may avoid introducing another system.
2. What latency service-level objective must be met within the available cost ceiling?

A team with a strict sub-20-millisecond interactive target and a small operations group may be better served by a managed hot tier. A team already operating Postgres or OpenSearch at scale, with the staff to handle capacity planning and a need to control quantization and tiering directly, may benefit from self-management. The engine should be selected for the workload rather than the reverse.

Key Takeaways

• The standalone vector database category has largely dissolved. ANN search is now available as a native data type or integrated feature in Postgres through pgvector, the Lucene-based engines, ClickHouse, MongoDB, and Redis because production retrieval benefits from colocating vectors with lexical scoring and filters.
• Memory is the dominant cost. An in-memory HNSW index requires about 1.1 × (4 × dimension + 8 × M) bytes per vector. Ten million 1,536-dimensional vectors require roughly 60 GB before graph overhead.
• The recall-versus-throughput curve is nonlinear. Pursuing the final few points of recall can reduce throughput by multiples, so increasing every search parameter is not always beneficial.
• Quantization and tiering are the main cost levers and are mostly vendor-independent. Int8 quantization can provide a four-fold reduction, Matryoshka truncation can provide a three- to twelve-fold reduction, binary quantization can provide a 32-fold reduction, and disk or object-storage tiers can reduce costs by up to approximately 90 percent at the expense of latency.
• The managed versus self-managed decision concerns ownership more than capability. Managed services absorb sizing and isolation, while self-managed systems provide greater control along with responsibility for the resulting failure modes. The choice should reflect where the data already resides and the balance between latency requirements and cost.