A distributed CBIR system based on DCNN on Apache Spark and Alluxio

Chen Li; Linhua Jiang; Xiaodong Chen

doi:10.1117/12.2514230

29 October 2018 A distributed CBIR system based on DCNN on Apache Spark and Alluxio

Chen Li, Linhua Jiang, Xiaodong Chen

Proceedings Volume 10836, 2018 International Conference on Image and Video Processing, and Artificial Intelligence; 108360L (2018) https://doi.org/10.1117/12.2514230
Event: 2018 International Conference on Image, Video Processing and Artificial Intelligence, 2018, Shanghai, China

Abstract

In the face of massive image data, how to improve the speed of image retrieval under the premise of ensuring accuracy is the focus of this research. In this paper, a distribute CBIR system based on the deep convolutional neural networks (DCNN) on Spark and Alluxio is proposed. Spark platform is a distributed memory calculation model to achieve higher computing performance. Alluxio is a high-performance, fault-tolerant, memory-based open source distributed storage system. This article lets Spark focus on computing image features and image matching. The storage of intermediate data in the image matching process is handled by Alluxio, thus breaking the bottleneck of Spark computing. The results of a large number of comparative experiments show that the proposed system shows obvious advantages both in terms of image storage capability and image retrieval speed.

Citation Download Citation

Chen Li, Linhua Jiang, and Xiaodong Chen "A distributed CBIR system based on DCNN on Apache Spark and Alluxio", Proc. SPIE 10836, 2018 International Conference on Image and Video Processing, and Artificial Intelligence, 108360L (29 October 2018); https://doi.org/10.1117/12.2514230

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available