NASA’s new dark energy space telescope can also detect killer asteroids
technologyreview.com·1h ago
AI Research
arXiv CS.AI·May 26·5 min read
Detecting Unfaithful Chain-of-Thought via Circuit-Guided Internal-External Discrepancy
TL;DR
Large language models (LLMs) often produce reasoning traces that do not accurately reflect their decision-making processes. To address this, the Circuit-guided Internal-External Discrepancy Scorer (CIE-Scorer) was developed to detect unfaithful chain-of-thought (CoT) reasoning by comparing internal and external reasoning graphs.
✦ Why It Matters
Engineers can leverage CIE-Scorer to enhance the reliability of LLM outputs in critical applications.
Key Takeaways
- Utilize CIE-Scorer for detecting unfaithful reasoning in LLMs.
- Combine internal computation evidence with external reasoning signals.
- Reduce circuit construction costs while maintaining accuracy.
- Test CIE-Scorer on various datasets for robust performance.
How It Works
CIE-Scorer operates by tracing compact sentence-level circuits from informative reasoning tokens, allowing it to create internal and external reasoning graphs. It then measures the discrepancies between these graphs using Fused Gromov-Wasserstein distance, which quantifies how different the two reasoning processes are.
This method ensures that faithful reasoning aligns closely with the model's internal computations.