Reimagining service delivery in the agentic era with Google Public Sector
cloud.google.com·1d ago
AI Research
arXiv CS.AI·May 27·8 min read
A Universal Cliff and a Design Fingerprint: Cross-Section Defect Detection Under LLM Orchestration
TL;DR
Detecting defects in cross-sectional images (like industrial scans or material inspections) has been difficult because defects vary widely and labeled training data is scarce. Researchers built a system using LLM orchestration—where a large language model coordinates multiple specialized detection tools—to identify and classify defects without extensive manual labeling.
✦ Why It Matters
Engineers can use LLM orchestration to build more robust defect detection systems with less labeled data and better generalization to novel defects.
Key Takeaways
- Test LLMs individually before orchestration to assess defect detection capabilities.
- Be cautious of false alarms when improving model alignment.
- Consider structural issues in LLM orchestration affecting defect detection.
- Utilize findings to refine document processing workflows with LLMs.
How It Works
The study reveals that LLMs operate through a network of worker agents that handle parts of a document request. When these agents work in isolation, they can detect certain defects.
However, when orchestrated, the models lose the ability to identify cross-section defects due to the way information is partitioned and recombined, leading to a significant drop in detection performance.