STAT-Net: spatiotemporal aggregation transformer network for skeleton-based few-shot action recognition

Dazhi Ren; Shengli Lv; Jinlin Li; Naining Li; Lin Dang

doi:10.1117/12.3058329

3 January 2025 STAT-Net: spatiotemporal aggregation transformer network for skeleton-based few-shot action recognition

Dazhi Ren, Shengli Lv, Jinlin Li, Naining Li, Lin Dang

Author Affiliations +

Proceedings Volume 13519, Third International Conference on Communications, Information System, and Data Science (CISDS 2024); 135190F (2025) https://doi.org/10.1117/12.3058329
Event: Third International Conference on Communications, Information System and Data Science 2024, 2024, Nanjing, China

Abstract

Few-shot action recognition predicts new classes without labels and has received widespread attention for practical systems. The skeleton is a sparse representation of human actions, and existing spatiotemporal based models by training a strong encoder network could make the skeleton graph very dense with edges, which may lead to the over-smoothing problem. To address this issue, we propose the Spatio-Temporal Aggregation Transformer Network (STAT-Net) as a general backbone for skeleton-based few-shot action recognition. In the spatiotemporal aggregation transformer modules, the spatial multi-head self attention for modeling the connection of different joints in the same frame, while the temporal multi-head self attention for modeling the skeleton sequence between two adjacent frames. The extracted features between the three parts are aggregated by Adaptive Fusion technique to obtain a high dimensional embedding. Extensive experiments on two benchmarks demonstrate that our proposed model achieves better recognition results com- pared with other existing methods.

Citation Download Citation

Dazhi Ren, Shengli Lv, Jinlin Li, Naining Li, and Lin Dang "STAT-Net: spatiotemporal aggregation transformer network for skeleton-based few-shot action recognition", Proc. SPIE 13519, Third International Conference on Communications, Information System, and Data Science (CISDS 2024), 135190F (3 January 2025); https://doi.org/10.1117/12.3058329

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

;

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE