Research on emotion recognition based on multimodal fusion

Weifa Zheng; LiKai Su

doi:10.1117/12.2680821

23 May 2023 Research on emotion recognition based on multimodal fusion

Weifa Zheng, LiKai Su

Proceedings Volume 12645, International Conference on Computer, Artificial Intelligence, and Control Engineering (CAICE 2023); 1264536 (2023) https://doi.org/10.1117/12.2680821
Event: International Conference on Computer, Artificial Intelligence, and Control Engineering (CAICE 2023), 2023, Hangzhou, China

Abstract

In order to solve the problem that the recognition accuracy of a single mode model depends on the emotion type, this paper proposes a multimodal emotion recognition model based on the Di-directional Gated Recurrent Unit (BiGRU) network and attention mechanism. In this paper, BiGRU neural networks are used to extract high level features from the original features of text, audio and video, and then high-level features are carried out. When high level features are fused, the attention mechanism is used to adjust the weight of emotional features, and finally emotional recognition is carried out to obtain emotional classification. The model is trained and tested on the CH-SIMS dataset, with an accuracy of 75.15%. Experimental results show that the proposed model has lower computational complexity and higher recognition accuracy.

Citation Download Citation

Weifa Zheng and LiKai Su "Research on emotion recognition based on multimodal fusion", Proc. SPIE 12645, International Conference on Computer, Artificial Intelligence, and Control Engineering (CAICE 2023), 1264536 (23 May 2023); https://doi.org/10.1117/12.2680821

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available