Expand prompt verbalizer by extracting knowledge for Chinese text classification

Di Wu; Shuang Zheng; Quanmin Wang

doi:10.1117/12.2681003

23 May 2023 Expand prompt verbalizer by extracting knowledge for Chinese text classification

Di Wu, Shuang Zheng, Quanmin Wang

Proceedings Volume 12645, International Conference on Computer, Artificial Intelligence, and Control Engineering (CAICE 2023); 126450H (2023) https://doi.org/10.1117/12.2681003
Event: International Conference on Computer, Artificial Intelligence, and Control Engineering (CAICE 2023), 2023, Hangzhou, China

Abstract

Prompt-based Pre-trained Language Models (PLMs) have demonstrated their superior performance on a wide variety of downstream tasks. In particular, the performance of prompt tuning has significantly outperformed traditional fine-tuning in the zero-shot learning and few-shot learning scenarios. The core idea of prompt-tuning is to convert different downstream tasks to mask language modeling problems through prompts, which can bridge the gap between pre-training tasks and downstream tasks for better results. Verbalizer, as an important part of prompt-tuning, can largely determine the final performance of the model, but the design of the Chinese-based verbalizer is yet to be fully explored. In this paper, we propose a method to expand the verbalizer by extracting knowledge from the training set based on a Chinese text classification task. In brief, we first segment the Chinese training set, then filter the words that can express the semantics of the labels by semantic similarity, and finally add them to the verbalizer. Extensive experimental results on multiple text classification datasets show that our approach significantly outperforms ordinary prompt-tuning and outperforms other methods for constructing the verbalizer.

Citation Download Citation

Di Wu, Shuang Zheng, and Quanmin Wang "Expand prompt verbalizer by extracting knowledge for Chinese text classification", Proc. SPIE 12645, International Conference on Computer, Artificial Intelligence, and Control Engineering (CAICE 2023), 126450H (23 May 2023); https://doi.org/10.1117/12.2681003

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available