Third-party cyber evaluations involving OpenAI models
openai.com·14h ago
AI Research
arXiv CS.AI·May 26·5 min read
Geo-Expert: Towards Expert-Level Geological Reasoning via Parameter-Efficient Fine-Tuning
TL;DR
General-purpose Large Language Models (LLMs) struggle with geological reasoning, particularly regarding subsurface structures. Geo-Expert, a family of parameter-efficient geological LLMs, was developed using Low-Rank Adaptation (LoRA) and fine-tuned on a specialized dataset.
✦ Why It Matters
Engineers and researchers can leverage Geo-Expert for improved geological reasoning in AI applications.
Key Takeaways
- Explore Geo-Expert for specialized geological applications.
- Utilize Low-Rank Adaptation for efficient model fine-tuning.
- Evaluate performance using the Geo-Eval benchmark.
- Consider cost-performance ratios for deploying AI models.
Full Summary
Large Language Models (LLMs) often produce inaccurate results when applied to specialized fields like geology, particularly in understanding subsurface structures and geological history. To address this, Geo-Expert was created, which consists of a series of parameter-efficient geological LLMs fine-tuned using a custom-curated dataset and a unique instruction synthesis pipeline.
The authors fine-tuned three base models—Qwen3-8B, Qwen3-32B, and Gemma-3-27B—using the Low-Rank Adaptation (LoRA) method. Their evaluation on a new benchmark, Geo-Eval, demonstrated that the 8B model could outperform larger models, including a 70B generalist and proprietary GPT-4o, in specialized geological reasoning tasks.
Additionally, the 32B variant approached the performance of leading models in reasoning. The optimized 8B model also provided a favorable cost-performance ratio for practical deployment.
This work lays the groundwork for advancing AI applications in Earth sciences and democratizing access to scientific LLMs.