Exploring No-SQL alternatives for ALMA monitoring system

Tzu-Chiang Shen; Ruben Soto; Patricio Merino; Leonel Peña; Marcelo Bartsch; Alvaro Aguirre; Jorge Ibsen

doi:10.1117/12.2055880

18 July 2014 Exploring No-SQL alternatives for ALMA monitoring system

Tzu-Chiang Shen, Ruben Soto, Patricio Merino, Leonel Peña, Marcelo Bartsch, Alvaro Aguirre, Jorge Ibsen

Proceedings Volume 9152, Software and Cyberinfrastructure for Astronomy III; 91520F (2014) https://doi.org/10.1117/12.2055880
Event: SPIE Astronomical Telescopes + Instrumentation, 2014, Montréal, Quebec, Canada

Abstract

The Atacama Large Millimeter /submillimeter Array (ALMA) will be a unique research instrument composed of at least 66 reconfigurable high-precision antennas, located at the Chajnantor plain in the Chilean Andes at an elevation of 5000 m. This paper describes the experience gained after several years working with the monitoring system, which has a strong requirement of collecting and storing up to 150K variables with a highest sampling rate of 20.8 kHz. The original design was built on top of a cluster of relational database server and network attached storage with fiber channel interface. As the number of monitoring points increases with the number of antennas included in the array, the current monitoring system demonstrated to be able to handle the increased data rate in the collection and storage area (only one month of data), but the data query interface showed serious performance degradation. A solution based on no-SQL platform was explored as an alternative to the current long-term storage system. Among several alternatives, mongoDB has been selected. In the data flow, intermediate cache servers based on Redis were introduced to allow faster streaming of the most recently acquired data to web based charts and applications for online data analysis.

Citation Download Citation

Tzu-Chiang Shen, Ruben Soto, Patricio Merino, Leonel Peña, Marcelo Bartsch, Alvaro Aguirre, and Jorge Ibsen "Exploring No-SQL alternatives for ALMA monitoring system", Proc. SPIE 9152, Software and Cyberinfrastructure for Astronomy III, 91520F (18 July 2014); https://doi.org/10.1117/12.2055880

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available