Third-party cyber evaluations involving OpenAI models
openai.com·13h ago
AI Research
arXiv CS.AI·May 26·5 min read
PEDESTRIANQA: A Benchmark for Vision-Language Models on Pedestrian Intention and Trajectory Prediction
TL;DR
Autonomous driving systems struggle with accurately predicting pedestrian intentions and trajectories in complex environments. PedestrianQA, a large-scale video-based dataset, was developed to frame these predictions as question-answering tasks, enhancing learning through natural language annotations.
✦ Why It Matters
Engineers can leverage PedestrianQA to enhance pedestrian behavior prediction in autonomous systems, improving safety and decision-making.
Key Takeaways
- Utilize PedestrianQA for training VLMs in pedestrian behavior prediction.
- Focus on improving intention classification accuracy in autonomous systems.
- Incorporate natural language explanations for better model interpretability.
- Evaluate models on existing benchmarks to measure performance gains.
Full Summary
Predicting pedestrian behavior is crucial for the safety of autonomous vehicles, as it directly impacts navigation decisions in traffic. PedestrianQA introduces a novel dataset that combines video data with natural language annotations, framing pedestrian intention and trajectory prediction as question-answering tasks.
This approach allows vision-language models (VLMs) to leverage visual dynamics and contextual interactions among traffic agents. The dataset includes richly annotated sequences that help VLMs generate concise explanations for their predictions.
Empirical evaluations on existing benchmarks like PIE, JAAD, TITAN, and IDD-PeD demonstrate that fine-tuning VLMs on PedestrianQA leads to significant improvements: intention classification accuracy increased, trajectory forecasting became more precise, and the quality of explanatory rationales improved. These findings suggest that VLMs can serve as a unified framework for modeling pedestrian behavior in safety-critical applications.