vexdb is a vector database written from scratch — no faiss bindings,
no black boxes. A hand-rolled HNSW graph index, AVX2 distance kernels, payload
filtering during traversal, binary snapshots, and an HTTP API — finished with an
honest head-to-head against faiss: recall
curves at parity, and the remaining QPS gap measured and explained.
No server, no video — vex-core compiled to WebAssembly (289 KB), searching 5,000 movie-plot embeddings with the same HNSW graph, filters, and snapshot format as the native build. Boot it, describe a movie — then watch the X-ray: every search animates the engine's own traversal trace, and a live benchmark measures recall against brute-force ground truth on your hardware.
One click downloads the 9.5 MB movies.vex
snapshot (5,000 plots × 384-dim MiniLM embeddings) and loads it into the WASM engine.
Exploring by example is instant; typed search additionally fetches the ~25 MB
embedding model so your query is embedded locally too.
The graph drifting behind the headline is the real algorithm. HNSW keeps every vector in a bottom-layer proximity graph, with exponentially sparser layers above it. A query enters at the top, greedily hops toward its target through the sparse layers, then runs a beam search of width ef through the dense bottom layer — visiting a few hundred vectors instead of all of them.
Level sampling, greedy descent, ef-bounded beam search, and the Algorithm 4
neighbor heuristic with keepPrunedConnections — implemented from the
2018 paper, not ported from a library.
Every build is checked against FlatIndex — exact brute force, recall
1.0 by definition. Proptest invariants plus a recall@10 floor catch structural
regressions a unit test never would.
The .vex format: magic bytes, a version field, then the graph arena
dumped as flat little-endian arrays. Atomic writes, validated reads — corrupt
files fail with a typed error, never a panic.
Runtime-detected SIMD for all three metrics, with the scalar versions kept as the correctness oracle in property tests. 2.5× faster index builds, ~2× query throughput.
Payload predicates evaluate during the beam search — non-matching nodes still route the beam but never take result slots, so a 1% filter widens the search instead of starving it.
Collections, upserts, filtered search, snapshots — JSON over HTTP with honest
status codes, input limits, a read-only mode, and a built-in console at
/. The engine stays embeddable; tokio lives in its own crate.
$ vex bench --n 100000 --dim 32 --k 10 # FlatIndex computes ground truth, then HNSW sweeps ef_search dataset: 100000 synthetic vectors, dim 32, 200 queries, k=10, metric L2 build: flat 23.0ms hnsw(M=16, efc=200) 48.4s ef_search recall@k QPS vs flat flat 1.000 504 1.0x 10 0.431 19610 38.9x 40 0.773 6930 13.7x 160 0.973 1989 3.9x 320 0.995 1033 2.0x
| ef | vexdb r@10 / qps | faiss r@10 / qps |
|---|---|---|
| 10 | 0.431 / 19,610 | 0.459 / 41,351 |
| 20 | 0.602 / 11,753 | 0.630 / 29,235 |
| 40 | 0.773 / 6,930 | 0.801 / 17,608 |
| 80 | 0.916 / 4,022 | 0.933 / 9,628 |
| 160 | 0.973 / 1,989 | 0.986 / 5,190 |
| 320 | 0.995 / 1,033 | 0.998 / 2,624 |
Same data, same parameters, single thread. 100k vectors ×
dim 32, M=16, identical splitmix64 streams — vexdb's HNSW vs faiss's
IndexHNSWFlat. Recall within ±0.03 at every ef: the graphs are
equivalent, so the algorithm is right.
The QPS gap is ~2–2.5× — and the flat baselines (504 vs 1,058 QPS, no graph involved) show it is entirely per-distance cost: faiss's hand-scheduled kernels and prefetching vs our AVX2 loop. For a from-scratch build, "graph at parity, kernels 2× behind" is exactly the gap worth understanding — the full analysis is in BENCHMARKS.md.
Core types, three distance metrics under one "smaller is closer" convention, brute-force FlatIndex with bounded-heap top-k, property tests, CI.
The layered graph index, recall tests against exact ground truth, criterion
baselines, and a vex bench harness that prints the recall/QPS curve.
The .vex binary snapshot format — magic bytes, versioning, the
graph arena as flat arrays. Atomic saves, validated loads, corruption tests.
Searches are embarrassingly parallel over a frozen index: RwLock
collections and a rayon-powered search_batch.
vex-server: a Qdrant-shaped JSON API with collections, upserts, deletes,
snapshot lifecycle, input limits, a read-only mode, and a live console at
/. Ships in a Docker image.
JSON payloads per vector and a Qdrant-style filter language (eq / range / in / and / or / not), evaluated during graph traversal so selective filters degrade gracefully.
Runtime-detected AVX2+FMA for all three metrics; scalar versions kept as the property-test oracle. 2.5× faster builds, ~2× query throughput.
The capstone: identical data, identical parameters, single thread. Recall at parity; the 2× QPS gap isolated to distance-kernel throughput and written up honestly in BENCHMARKS.md.
vex-wasm: the engine compiled to 289 KB of WebAssembly. The live demo at the top of this page — 5,000 movie embeddings, filters, snapshot parsing — is the same Rust running in your tab.
Zero-cost trace instrumentation in the engine (search_traced) and
an exact-scan oracle (search_exact): the demo animates the real
traversal — descent, beam, filter rejections — over a PCA projection computed
in-tab, and measures live recall curves against ground truth.