Agentic data science with the Data Agent Kit
cloud.google.com·7h ago
AI Research
arXiv CS.AI·10d ago·5 min read
When Does Exercise-Specific Joint Selection Help? An Audit of Evaluation and Control Design
TL;DR
Exercise-specific joint selection can enhance skeleton-based correctness classification, but its benefits are nuanced. An audit of 1,057 repetitions reveals that while some models show gains, the overall impact varies significantly based on evaluation methods.
✦ Why It Matters
Evaluate your joint selection methods carefully to ensure they align with your specific classification goals.
Key Takeaways
- Assess the impact of joint selection on your classification tasks.
- Consider using RBF-SVM for better within-person gains.
- Evaluate your methods against clear estimands.
- Be cautious of claims without robust controls.
Full Summary
Exercise-specific joint selection aims to improve the accuracy of skeleton-based classification tasks in rehabilitation contexts. An audit was conducted on 1,057 repetitions from ten subjects using the REHAB24-6 dataset, focusing on evaluation aggregation, subset structure, and temporal representation.
Results showed that the manual subset k-nearest neighbors (kNN) method had a gain of 0.055 for pooled out-of-fold area under the receiver operating characteristic (AUROC) and 0.020 for equal-weight within-person AUROC, with both intervals including zero. Among 1,000 random maps, only 14 matched or exceeded the manual results, while 145 did so when considering bilateral structure and trunk inclusion.
The radial basis function support vector machine (RBF-SVM) showed a positive gain, contrasting with negative results from logistic regression and a random-convolution comparator. This audit highlights the importance of explicit estimands and appropriate controls in joint-selection claims without proposing a new algorithm or clinical benefit.