Multimodal social media video classification with deep neural networks

Tomasz Trzcinski

doi:10.1117/12.2501679

1 October 2018 Multimodal social media video classification with deep neural networks

Tomasz Trzcinski

Proceedings Volume 10808, Photonics Applications in Astronomy, Communications, Industry, and High-Energy Physics Experiments 2018; 108082U (2018) https://doi.org/10.1117/12.2501679
Event: Photonics Applications in Astronomy, Communications, Industry, and High-Energy Physics Experiments 2018, 2018, Wilga, Poland

Abstract

Classifying videos according to their content is a common task across various contexts, as it allows effective content tagging, indexing and searching. In this work, we propose a general framework for video classification that is built on top of several neural network architectures. Since we rely on a multimodal approach, we extract both visual and textual features from videos and combine them in a final classification algorithm. When trained on a dataset of 30 000 social media videos and evaluated on 6 000 videos, our multimodal deep learning algorithm outperforms shallow single-modality classification methods by a large margin of up to 95%, achieving overall accuracy of 88%.

Citation Download Citation

Tomasz Trzcinski "Multimodal social media video classification with deep neural networks", Proc. SPIE 10808, Photonics Applications in Astronomy, Communications, Industry, and High-Energy Physics Experiments 2018, 108082U (1 October 2018); https://doi.org/10.1117/12.2501679

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available