NASA’s new dark energy space telescope can also detect killer asteroids
technologyreview.com·2h ago
AI Research
arXiv CS.AI·May 25·8 min read
Not Too Generative, Not Too Discriminative: The Human Alignment Sweet Spot
TL;DR
Prior work debated whether human-like vision emerges from discriminative learning (classifying objects) or generative learning (modeling data distribution), but confounded this with architecture and scale differences. Researchers used Joint Energy-Based Models (JEMs) to continuously interpolate between these objectives within identical architectures, isolating the learning objective's effect.
✦ Why It Matters
Engineers building human-aligned vision systems should blend discriminative and generative objectives rather than choosing one exclusively.
Key Takeaways
- Use Joint Energy-Based Models to interpolate between learning objectives within fixed architectures.
- Test vision models across multiple human-alignment benchmarks: perceptual similarity, robustness, and feature attribution.
- Optimize the generative-discriminative mixing coefficient empirically rather than committing to one extreme.
- Expect peak human alignment at intermediate objective blends, not at pure discriminative or generative endpoints.
Full Summary
Computational vision researchers have long questioned whether human visual perception aligns better with discriminative or generative learning objectives. Discriminative models learn to classify inputs into categories; generative models learn to model the underlying data distribution.
Prior comparisons conflated the learning objective with other variables like model architecture, training scale, and dataset composition, making it unclear whether the objective itself drove human alignment. This work uses Joint Energy-Based Models (JEMs)—a framework allowing continuous interpolation between discriminative and generative training via a single mixing coefficient—to isolate the objective's effect while holding architecture constant.
Evaluation across six human-alignment benchmarks (perceptual similarity, gloss perception, uncertainty, robustness, shape-texture conflicts, and feature attribution) consistently showed peak alignment at intermediate points rather than endpoints. Hybrid JEMs combine categorical structure from discriminative learning with input-sensitivity from generative learning, producing more human-like behavior across multiple vision levels.
Results suggest the generative-discriminative dichotomy is a false choice; human-aligned vision emerges from balancing both objectives.