Third-party cyber evaluations involving OpenAI models
openai.com·13h ago
AI Research
arXiv CS.AI·May 25·5 min read
Sutra: Tensor-Op RNNs as a Compilation Target for Vector Symbolic Architectures
TL;DR
Vector Symbolic Architectures (VSAs) lacked a practical implementation for neural networks. Sutra is a functional programming language that compiles to a PyTorch neural network, enabling tensor operations to represent complex logic.
✦ Why It Matters
Engineers can leverage Sutra to create efficient neural networks that also incorporate symbolic reasoning.
Key Takeaways
- Explore Sutra for integrating symbolic logic with neural networks.
- Utilize the compiled tensor operation graph for efficient computation.
- Test models across different embeddings for robust performance.
- Leverage the dual functionality of Sutra for versatile applications.
Full Summary
Vector Symbolic Architectures (VSAs) are theoretical frameworks that combine symbolic reasoning with neural computation, but practical implementations have been limited. Sutra is introduced as a typed, purely functional programming language that compiles into a PyTorch neural network, allowing for the representation of complex operations as tensor operations.
The compiler reduces the entire program, including control flow and input/output, into a single tensor operation graph. Validation was performed by running the same program on multiple frozen embeddings, achieving 100% accuracy across different modalities, including text and protein language models.
Additionally, a fuzzy-rule classifier trained from random initialization to perfect accuracy by backpropagating through the compiled graph. This dual functionality means that the same code serves as both a logic program and a trainable neural network, making it a versatile tool for researchers and engineers.