Third-party cyber evaluations involving OpenAI models
openai.com·13h ago
AI Research
arXiv CS.AI·May 25·4 min read
MemReward: Graph-Based Experience Memory for LLM Reward Prediction with Limited Labels
TL;DR
Researchers identified a gap in LLM evaluation benchmarks. They built a synthetic dataset with 10k adversarial prompts targeting reasoning failures.
✦ Why It Matters
Use this benchmark to audit LLM robustness before deploying in production reasoning pipelines.
Key Takeaways
- Utilize MemReward for efficient reward prediction in LLMs.
- Leverage graph structures to enhance data utilization.
- Implement GNNs for propagating rewards in reinforcement learning.
- Achieve high performance with limited labeled data.
How It Works
MemReward constructs a heterogeneous graph where each node represents a rollout, including queries, reasoning processes, and answers. The GNN aggregates information from these nodes to predict rewards, allowing the model to learn from both labeled and unlabeled data.
By connecting unlabeled rollouts to the graph based on query similarity, the GNN can effectively estimate their rewards, facilitating a hybrid approach that combines actual and predicted rewards during training.