Agentic data science with the Data Agent Kit
cloud.google.com·7h ago
AI Research
arXiv CS.AI·Aug 3·5 min read
RareSense: Rarity-Aware Similarity Search for Anomaly Retrieval in Transactional Data
TL;DR
RareSense is a new framework designed for anomaly retrieval in sparse transactional data, outperforming traditional similarity measures by focusing on rare itemsets. It achieves superior retrieval performance, particularly in scenarios where anomalies share uncommon structures.
✦ Why It Matters
Implement RareSense to enhance your anomaly detection capabilities in transactional data today.
Key Takeaways
- Utilize RareSense for improved anomaly detection in sparse data.
- Focus on rare itemsets to enhance similarity search outcomes.
- Leverage weighted Jaccard similarity for better data comparisons.
- Apply RareSense in cybersecurity and categorical data domains.
How It Works
RareSense operates by mining minimal rare itemsets from transactional data, which are then used to create association rules. These rules are transformed into sparse profiles that represent the data objects.
The framework employs a weighted Jaccard similarity measure that incorporates various metrics, such as inverse support and structural complexity, to assess the similarity between these profiles. This approach allows RareSense to prioritize shared rare evidence over common features, leading to more accurate anomaly detection.