Cloud services on an astronomy data center

Mauricio Solar; Mauricio Araya; Humberto Farias; Diego Mardones; Zhong Wang

doi:10.1117/12.2232705

8 August 2016 Cloud services on an astronomy data center

Mauricio Solar, Mauricio Araya, Humberto Farias, Diego Mardones, Zhong Wang

Proceedings Volume 9913, Software and Cyberinfrastructure for Astronomy IV; 99131G (2016) https://doi.org/10.1117/12.2232705
Event: SPIE Astronomical Telescopes + Instrumentation, 2016, Edinburgh, United Kingdom

Abstract

The research on computational methods for astronomy performed by the first phase of the Chilean Virtual Observatory (ChiVO) led to the development of functional prototypes, implementing state-of-the-art computational methods and proposing new algorithms and techniques. The ChiVO software architecture is based on the use of the IVOA protocols and standards. These protocols and standards are grouped in layers, with emphasis on the application and data layers, because their basic standards define the minimum operation that a VO should conduct. As momentary verification, the current implementation works with a set of data, with 1 TB capacity, which comes from the reduction of the cycle 0 of ALMA. This research was mainly focused on spectroscopic data cubes coming from the cycle 0 ALMA's public data. As the dataset size increases when the cycle 1 ALMA's public data is also increasing every month, data processing is becoming a major bottleneck for scientific research in astronomy. When designing the ChiVO, we focused on improving both computation and I/ O costs, and this led us to configure a data center with 424 high speed cores of 2,6 GHz, 1 PB of storage (distributed in hard disk drives-HDD and solid state drive-SSD) and high speed communication Infiniband. We are developing a cloud based e-infrastructure for ChiVO services, in order to have a coherent framework for developing novel web services for on-line data processing in the ChiVO. We are currently parallelizing these new algorithms and techniques using HPC tools to speed up big data processing, and we will report our results in terms of data size, data distribution, number of cores and response time, in order to compare different processing and storage configurations.

Citation Download Citation

Mauricio Solar, Mauricio Araya, Humberto Farias, Diego Mardones, and Zhong Wang "Cloud services on an astronomy data center", Proc. SPIE 9913, Software and Cyberinfrastructure for Astronomy IV, 99131G (8 August 2016); https://doi.org/10.1117/12.2232705

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
7 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Astronomy

Clouds

Data centers

Data storage

Prototyping

Data processing

Observatories

Show All Keywords

Keywords/Phrases

Search In:

Publication Years