BerriAI / litellm
github.com·3d ago
Developer Tools
GitHub Trending·Aug 25·5 min read
marin-community / marin
TL;DR
Marin is a comprehensive platform for developing foundation models, focusing on the training of large language models through open development practices. It emphasizes transparency in processes, including data curation and model evaluation, while also supporting diverse applications like audio-text and biological modeling.
✦ Why It Matters
Explore Marin's library to enhance your own model training and experimentation today.
Key Takeaways
- Install Marin to begin training language models.
- Utilize the Delphi scaling suite for large model training.
- Access training checkpoints on Hugging Face for reproducibility.
- Join the Marin community on Discord for collaboration.
Full Summary
Marin serves as a research program and software platform dedicated to the development of foundation models, particularly large language models. It encompasses various processes such as data curation, tokenization, pretraining, and evaluation, while emphasizing transparency by documenting all experiments and decisions.
The platform has been utilized for diverse applications, including audio-text and biological models, and is currently focused on pretraining a large mixture-of-experts model with over 500 billion parameters. Key tools include Delphi, which scales models from 3e18 to 1e23 FLOPs, and a comprehensive set of checkpoints and training pipelines available on Hugging Face.
Recent experiments have shown that Marin can outperform existing models like Llama 3.1, demonstrating its potential for significant advancements in model training efficiency and performance.