Agentic data science with the Data Agent Kit
cloud.google.com·7h ago
Data Science
Towards Data Science·Jul 27·5 min read
How I Reproduced BM25, Dense Retrieval, and SPLADE on a 16GB MacBook
TL;DR
A software engineer successfully reproduced three retrieval models—BM25, Dense Retrieval, and SPLADE—on a 16GB MacBook, detailing the challenges faced and solutions implemented.
✦ Why It Matters
Experiment with these retrieval models on your own hardware to enhance your understanding of information retrieval techniques.
Key Takeaways
- Reproduce BM25, dense retrieval, and SPLADE for accurate benchmarking.
- Address memory management issues when indexing large datasets.
- Use dot-product checks to validate scoring accuracy.
- Request access for gated models like SPLADE before implementation.
How It Works
BM25 counts shared words between a query and documents, producing a sparse vector based on a fixed formula. Dense retrieval uses a neural network to create embeddings that capture meaning, while SPLADE combines aspects of both, using a trained model to determine word importance.