MSTDKD: a framework of using multiple self-supervised methods for semi-supervised learning

JiaBin Liu; XuanMing Zhang; Jun Hu

doi:10.1117/12.2661030

2 February 2023 MSTDKD: a framework of using multiple self-supervised methods for semi-supervised learning

JiaBin Liu, XuanMing Zhang, Jun Hu

Proceedings Volume 12462, Third International Symposium on Computer Engineering and Intelligent Communications (ISCEIC 2022); 124621T (2023) https://doi.org/10.1117/12.2661030
Event: International Symposium on Computer Engineering and Intelligent Communications (ISCEIC 2022), 2022, Xi'an, China

Abstract

Image classification is a basic task in the field of computer vision, and general image classification task training requires a large amount of labeled data to achieve good generalization performance. However, in practical applications, the cost of obtaining labeled data is expensive. In contrast, unlabeled images are easy to obtain, so semi-supervised image classification is more meaningful for research. This paper pro- poses a framework for semi-supervised classification utilizing multiple self-supervised methods. Our approach is divided into three steps, firstly, pre-train multiple models on unlabeled data using different self-supervised methods. Then use the labeled data to fine-tune these models except the model pre-training by Contrastive learning to obtaining multiple self-supervised teacher models. Finally, the multi-teacher knowledge distillation framework is used to transfer the knowledge of multiple self-supervised teacher models to the model pre-training by contrastive learning to help it achieve further performance. We conducted experiments on cifar10 and miniimagenet60. Our method achieves further results than using only a single self-supervised method, and also achieves superior performance compared to other semi-supervised methods.

Citation Download Citation

JiaBin Liu, XuanMing Zhang, and Jun Hu "MSTDKD: a framework of using multiple self-supervised methods for semi-supervised learning", Proc. SPIE 12462, Third International Symposium on Computer Engineering and Intelligent Communications (ISCEIC 2022), 124621T (2 February 2023); https://doi.org/10.1117/12.2661030

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available