MAT-Agent: Adaptive Multi-Agent Training Optimization

TL;DR

MAT-Agent introduces a multi-agent framework that dynamically optimizes multi-label image classification training by adjusting key parameters in real-time, leading to improved accuracy, stability, and efficiency across multiple datasets.

Contribution

It presents a novel multi-agent system utilizing non-stationary bandit algorithms for adaptive training configuration in complex visual tasks.

Findings

Achieves higher mAP on Pascal VOC, COCO, and VG-256 datasets.

Demonstrates faster convergence and better cross-domain generalization.

Outperforms existing static and adaptive methods in multiple metrics.

Abstract

Multi-label image classification demands adaptive training strategies to navigate complex, evolving visual-semantic landscapes, yet conventional methods rely on static configurations that falter in dynamic settings. We propose MAT-Agent, a novel multi-agent framework that reimagines training as a collaborative, real-time optimization process. By deploying autonomous agents to dynamically tune data augmentation, optimizers, learning rates, and loss functions, MAT-Agent leverages non-stationary multi-armed bandit algorithms to balance exploration and exploitation, guided by a composite reward harmonizing accuracy, rare-class performance, and training stability. Enhanced with dual-rate exponential moving average smoothing and mixed-precision training, it ensures robustness and efficiency. Extensive experiments across Pascal VOC, COCO, and VG-256 demonstrate MAT-Agent's superiority: it achieves an mAP of 97.4 (vs. 96.2 for PAT-T), OF1 of 92.3, and CF1 of 91.4 on Pascal VOC; an mAP of 92.8 (vs. 92.0 for HSQ-CvN), OF1 of 88.2, and CF1 of 87.1 on COCO; and an mAP of 60.9, OF1 of 70.8, and CF1 of 61.1 on VG-256. With accelerated convergence and robust cross-domain generalization, MAT-Agent offers a scalable, intelligent solution for optimizing complex visual models, paving the way for adaptive deep learning advancements.