3D deformable convolution for action classification in videos

Siew Cheng Lai; Hung Khoon Tan; Phooi Yee Lau

doi:10.1117/12.2591088

13 March 2021 3D deformable convolution for action classification in videos

Siew Cheng Lai, Hung Khoon Tan, Phooi Yee Lau

Proceedings Volume 11766, International Workshop on Advanced Imaging Technology (IWAIT) 2021; 117660R (2021) https://doi.org/10.1117/12.2591088
Event: International Workshop on Advanced Imaging Technology 2021 (IWAIT 2021), 2021, Online Only

Abstract

Action recognition is one of the popular research areas in computer vision because it can be applied to solve many problems especially in security surveillance, behavior analysis, healthcare and so on. Some of the well-known Convolutional Neural Network (CNN) in action classification using 3D convolution are C3D, I3D and R(2 + 1)D. These 3D CNNs assume that the spatial and temporal dimensions of motion are uniform where the 3D filters are uniformly shaped. However, the path in motion can be in any directions and a uniform shape filter might not be able to capture nonuniform spatial motion and this limits the performance of the classification. To address the above problem, we incorporate a 3D deformable filter in a C3D network for action classification. The deformable convolution adds offsets to the regular grid sampling locations in the standard convolution resulting in non-uniform sampling location. We will also investigate the performance of the network when apply the 3D deformable convolution in different layers and the effect of different dilation size of the 3D deformable filter. UCF101 dataset is used in the experiments. From our experiments, it is found that applying the deformable convolution in lower layer yield better result compare to other layers. Our experiment shows that if we put the deformable convolution in Conv1a, the accuracy achieved is 48.50%.

Citation Download Citation

Siew Cheng Lai, Hung Khoon Tan, and Phooi Yee Lau "3D deformable convolution for action classification in videos", Proc. SPIE 11766, International Workshop on Advanced Imaging Technology (IWAIT) 2021, 117660R (13 March 2021); https://doi.org/10.1117/12.2591088

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available