Diagnosing Ensemble Few-Shot Classifiers

Diagnosing Ensemble Few-Shot Classifiers

Yang W, Ye X, Zhang X, Xiao L, Xia J, Wang Z, Zhu J, Pfister H, and Liu S.

arXiv preprint arXiv:2206.04372, 2022.

The base learners and labeled samples (shots) in an ensemble few-shot classifier greatly affect the model performance. When the performance is not satisfactory, it is usually difficult to understand the underlying causes and make improvements. To tackle this issue, we propose a visual analysis method, FSLDiagnotor. Given a set of base learners and a collection of samples with a few shots, we consider two problems: 1) finding a subset of base learners that well predict the sample collections; and 2) replacing the low-quality shots with more representative ones to adequately represent the sample collections. We formulate both problems as sparse subset selection and develop two selection algorithms to recommend appropriate learners and shots, respectively. A matrix visualization and a scatterplot are combined to explain the recommended learners and shots in context and facilitate users in adjusting them. Based on the adjustment, the algorithm updates the recommendation results for another round of improvement. Two case studies are conducted to demonstrate that FSLDiagnotor helps build a few-shot classifier efficiently and increases the accuracy by 12% and 21%, respectively.

Acknowledgements

This work was supported by the National Key R&D Program of China under Grant 2020YFB2104100, the National Natural Science Foundation of China under grants U21A20469, 61936002, grants from the Institute Guo Qiang, THUIBCS, and BLBCI, and in part by Tsinghua-Kuaishou Institute of Future Media Data.