LLMs as ASP Programmers: Self-Correction Enables Task-Agnostic Nonmonotonic Reasoning

Large language models (LLMs) have made strides in reasoning but struggle with logical inconsistencies and high computational demands, particularly in complex tasks. To address these issues, LLM+ASP was created, which translates natural language into Answer Set Programming (ASP), a formalism that supports nonmonotonic reasoning—allowing for default rules and exceptions.

Unlike previous methods that required manual knowledge modules or were limited to specific tasks, LLM+ASP operates uniformly across various reasoning challenges. The framework incorporates an automated self-correction loop, where feedback from the ASP solver refines the model's outputs iteratively.

Results from six benchmarks demonstrated that LLM+ASP outperformed traditional SMT-based approaches by significant margins, particularly in nonmonotonic tasks. Additionally, the study found that concise in-context reference guides were more effective than verbose documentation, highlighting the importance of context management.

These findings suggest that LLMs can be enhanced for complex reasoning tasks without extensive manual intervention.