Agentic data science with the Data Agent Kit
cloud.google.com·8h ago
AI Research
arXiv CS.AI·Sep 10·5 min read
Grounded Evaluation and Repair for NL-to-PDDL Problem Generation
TL;DR
A novel pipeline for translating Natural Language (NL) planning descriptions into PDDL problem instances was developed, revealing significant discrepancies between operational success and benchmark reference reconstruction. The approach integrates LLM generation, validation checks, and structured repair feedback to enhance problem fidelity.
✦ Why It Matters
Implement structured feedback mechanisms in your NL-to-PDDL pipelines to enhance problem fidelity today.
Key Takeaways
- Utilize LLMs for translating NL descriptions into PDDL.
- Incorporate validation checks to ensure semantic accuracy.
- Apply structured repair feedback for iterative improvements.
- Benchmark against curated PDDL problems to assess fidelity.
How It Works
The pipeline combines LLM generation with a series of validation checks, including PDDL parsing and planning validation. It also employs a domain-conformance checker and an LLM critic to provide iterative feedback.
This structured approach allows for fine-grained analysis of discrepancies between the generated problems and the original NL descriptions, facilitating targeted repairs.