Matrix sketching for big data reduction (Conference Presentation)

Soundararajan Ezekiel; Michael Giansiracusa

doi:10.1117/12.2262937

6 June 2017 Matrix sketching for big data reduction (Conference Presentation)

Soundararajan Ezekiel, Michael Giansiracusa

Proceedings Volume 10199, Geospatial Informatics, Fusion, and Motion Video Analytics VII; 101990F (2017) https://doi.org/10.1117/12.2262937
Event: SPIE Defense + Security, 2017, Anaheim, CA, United States

Abstract

Abstract: In recent years, the concept of Big Data has become a more prominent issue as the volume of data as well as the velocity in which it is produced exponentially increases. By 2020 the amount of data being stored is estimated to be 44 Zettabytes and currently over 31 Terabytes of data is being generated every second. Algorithms and applications must be able to effectively scale to the volume of data being generated. One such application designed to effectively and efficiently work with Big Data is IBM’s Skylark. Part of DARPA’s XDATA program, an open-source catalog of tools to deal with Big Data; Skylark, or Sketching-based Matrix Computations for Machine Learning is a library of functions designed to reduce the complexity of large scale matrix problems that also implements kernel-based machine learning tasks. Sketching reduces the dimensionality of matrices through randomization and compresses matrices while preserving key properties, speeding up computations. Matrix sketches can be used to find accurate solutions to computations in less time, or can summarize data by identifying important rows and columns. In this paper, we investigate the effectiveness of sketched matrix computations using IBM’s Skylark versus non-sketched computations. We judge effectiveness based on several factors: computational complexity and validity of outputs. Initial results from testing with smaller matrices are promising, showing that Skylark has a considerable reduction ratio while still accurately performing matrix computations.

Conference Presentation

Citation Download Citation

Soundararajan Ezekiel and Michael Giansiracusa "Matrix sketching for big data reduction (Conference Presentation)", Proc. SPIE 10199, Geospatial Informatics, Fusion, and Motion Video Analytics VII, 101990F (6 June 2017); https://doi.org/10.1117/12.2262937

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
PRESENTATION

WATCH
PRESENTATION SAVE TO MY LIBRARY

GET CITATION

KEYWORDS

Matrices

Machine learning

Analytics

Current controlled current source

Digital video recorders

Information science

Video

Keywords/Phrases

Search In:

Publication Years