EmbeddingGemma 2: an open, lightweight multimodal embedding model
deepmind.google·6d ago
AI Engineering
Hacker News·24d ago·5 min read
OpenJev
TL;DR
OpenJev allows users to read and normalize model choice logits for specific options, enabling efficient local model performance on personal GPUs. Users can select between different model sizes based on their device capabilities.
✦ Why It Matters
Experiment with OpenJev today to optimize model performance on your specific hardware.
Key Takeaways
- Test OpenJev with different model sizes on your GPU.
- Normalize choice logits for specific options to improve efficiency.
- Switch to Qwen3 0.6B for better performance on smaller devices.
- Measure performance differences between models to find the best fit.
Full Summary
OpenJev is a tool that facilitates the reading of choice logits from AI models, allowing users to normalize these logits across specified options. This functionality enables local models to either read probabilities for allowed options without decoding or to generate distributions token by token.
The default model is MiniCPM5 2B, but users with smaller devices can switch to Qwen3 0.6B for better performance. While larger models may provide more capabilities, they can also lead to slower loading times and may not fit on low-end devices.
Users are encouraged to measure the performance differences between models on their own GPUs to determine the best fit for their needs.