NASA’s new dark energy space telescope can also detect killer asteroids
technologyreview.com·3h ago
AI Research
arXiv CS.AI·Jun 10·5 min read
Multi-Level Analyzation of Imbalance to Resolve Non-IID-Ness in Federated Learning
TL;DR
Class imbalance in federated learning (FL) leads to non-identically distributed data, negatively impacting model performance. To address this, FedBB was developed, incorporating a Positive Negative Balanced (PNB) loss function and Client Balanced Reweighting (CBR) to enhance training and aggregation.
✦ Why It Matters
Engineers can implement FedBB to improve model performance in federated learning scenarios with imbalanced data.
Key Takeaways
- Utilize FedBB to address class imbalance in federated learning.
- Implement PNB loss for better local training on skewed datasets.
- Apply CBR to enhance model aggregation based on client data distribution.
- Leverage minimal statistical information for improved privacy in model training.
Full Summary
Class imbalance is a prevalent issue in deep learning, particularly in federated learning (FL), where data is distributed across multiple clients. This paper introduces FedBB, a novel approach that tackles imbalance at three levels: inter-case (within a class), inter-class (between classes), and inter-client (across different clients).
FedBB employs a Positive Negative Balanced (PNB) loss function to improve local training by assigning higher weights to minority cases, thus enhancing generalization. Additionally, Client Balanced Reweighting (CBR) adjusts the influence of client models during aggregation based on the skewness of their datasets.
Experiments conducted on X-ray and natural image datasets demonstrated that FedBB outperformed other algorithms, achieving better accuracy and efficiency. The method also requires limited statistical information, which is advantageous for privacy.
Overall, FedBB serves as a robust baseline for both generic and personalized federated learning applications.