Vector Search

Summary: Vector search replaces brittle keyword matching with nearest‑neighbor queries in high‑dimensional embedding spaces to enable semantic retrieval, recommendation, and multimodal matching at scale, but it isn’t a magic bullet precision depends on strong embeddings, index design, and hybrid ranking. Building reliable, cost‑effective vector search requires careful choices across ANN algorithms and index/data structures, embedding pipelines (model choice, batching, versioning, normalization), and operational telemetry to monitor and maintain production systems.

Executive summary

Vector search is the production-grade technique that underpins semantic retrieval, recommendation, and multimodal matching at scale. It replaces brittle keyword matching with nearest-neighbor queries in high-dimensional embedding spaces. Engineering this well requires decisions across algorithms (ANN families), index design, embedding pipelines, and operational telemetry. Below is a compact, high-signal playbook for building reliable, cost-effective vector search in production.

What vector search is (and isn't)

• Vector search finds items whose embeddings are nearest a query embedding under a chosen similarity metric (cosine, inner product, Euclidean).
• It is not a magic bullet for precision: good embeddings + index design + hybrid ranking are necessary to achieve search-quality parity with classic IR for many applications.

Core components

• Embedding generation: model choice, batching, versioning, and normalization.
• Indexing: data structures and quantization for fast retrieval (HNSW, IVF, PQ, Faiss GPU).
• Retrieval pipeline: first-stage ANN search, optional filter by metadata, second-stage re-ranking with cross-encoders or retrieval-augmented models.
• Ops: metrics (recall@k, P99 latency), sharding/replication, incremental updates, and cost monitoring.

Algorithms and trade-offs

• HNSW (Hierarchical Navigable Small World)

  • Pros: excellent recall/latency trade-off, dynamic inserts, good CPU performance.
  • Key knobs: M (connectivity, typical 16–64), efConstruction (200–2000), efSearch (tuned per latency vs. recall).
  • Use when low-latency and frequent incremental updates are required.

• IVF + PQ (+OPQ)

  • Pros: lower memory footprint, scales to billions of vectors.
  • Cons: needs full re-encoding for OPQ and cluster re-training for large updates.
  • Use when storage costs dominate and the index is relatively static.
  • PQ bits: 8–16 bits/subvector typical; balance bytes-per-vector vs accuracy.

• GPU brute-force (Faiss/GPU)

  • Pros: highest recall, simple to implement for up to tens of millions of vectors; excellent for re-ranking.
  • Cons: cost, especially idle GPU; batch-friendly.

• Hybrid approaches:

  • First-stage ANN for recall, then lexical (BM25) or cross-encoder re-ranking for precision.
  • Apply metadata filters pre-search to reduce false positives and cost.

Metrics that matter

• Recall@k (95%+ often desired for UX-sensitive apps).
• Latency P50/P95/P99 for the full pipeline.
• Index build time and incremental update latency.
• Cost per QPS (CPU/GPU and storage).

Production patterns and actionable tuning

• Embeddings:

  • Normalize for cosine similarity: L2-normalize if using dot product libraries that prefer raw inner product.
  • Freeze and version models; tag vectors with model-version metadata for backward compatibility.

• HNSW tuning (practical defaults to start)

  • M = 32, efConstruction = 500 for balanced build time and quality.
  • Start efSearch = 128; increase for higher recall until latency budget is met.

• IVF+PQ realistic setup

  • IVF clusters = sqrt(N) or empirically 10k–100k; OPQ before PQ improves quality.
  • PQ code length = 8–16 bytes/vector depending on quality vs storage requirements.

• Sharding & availability

  • Horizontal shard by vector-id or vector-hash; replicate shards for HA and read-scaling.
  • Use asynchronous background rebuilds for large re-indexes; swap indices atomically.

• Throughput optimization

  • Batch queries, use SIMD-optimized libraries (Faiss, NMSLIB, Hnswlib).
  • Co-locate nearest-neighbor CPU/GPU nodes with embedding-serving to reduce network cost.

Monitoring, testing, and lifecycle

• Continuous A/B test retrieval quality after embedding model updates.
• Track recall degradation and drift of embedding distribution (cosine histogram).
• Implement offline evaluation harness: synthetic queries + ground-truth sets to tune efSearch, M, PQ bits.

Checklist & next steps

• Instrument recall@k and latency P99 across dev and prod traffic.
• Choose an index family: HNSW for dynamic low-latency, IVF+PQ for cost at scale.
• Establish embedding model versioning and an incremental re-index strategy.
• Prototype a hybrid pipeline: ANN → metadata filter → cross-encoder rerank.

Vector search is an architecture problem as much as an algorithmic one: prioritize clear SLAs (recall, latency, cost), automate evaluation against those SLAs, and treat index configuration and embedding evolution as first-class deployable artifacts.