NASA’s new dark energy space telescope can also detect killer asteroids
technologyreview.com·3h ago
AI Research
arXiv CS.AI·May 26·5 min read
By Their Fruits You Will Know Them: Comparing Formalizations of Law by the Decisions They Encode
TL;DR
Legal formalizations often involve interpretive choices that can lead to unexpected outcomes. A method was developed to compare different formalizations of legal provisions by analyzing their inferences on specific cases using a SAT solver.
✦ Why It Matters
Engineers and researchers can better assess the reliability of AI-generated legal formalizations and their implications.
Key Takeaways
- Utilize the new method for comparing legal formalizations.
- Explore the implications of interpretive choices in AI-generated law.
- Investigate edge cases to enhance legal reasoning accuracy.
- Consider behavioral divergence when assessing legal AI outputs.
Full Summary
Formalizing legal provisions aims to create machine-readable law and enable automated legal reasoning, but it introduces interpretive choices that can be difficult to predict. A new method was created to systematically compare various formalizations of the same legal provision by examining their inferences on individual cases.
This approach involves matching formalizations at the node level, deriving a shared interface, and using a SAT solver to identify edge cases where formalizations disagree. The method was applied to ten EU legal provisions formalized by nine advanced language models (LLMs).
Results showed that behavioral divergence among formalizations was largely uncorrelated with their structural agreement, and the verbalized edge cases revealed distinct types of disagreement, some reflecting real controversies in legal discussions. These findings suggest that even minor differences in formalization can lead to significant variations in legal interpretation.