Improving ALMA’s data processing efficiency using a holistic approach

Theodoros Nakos; Harold Francke; Kouichiro Nakanishi; Dirk Petry; Thomas Stanke; Catarina Ubach; Luciano Cerrigone; Erica Keller; Alfonso Trejo; Junko Ueda

doi:10.1117/12.2559324

15 December 2020 Improving ALMA’s data processing efficiency using a holistic approach

Theodoros Nakos, Harold Francke, Kouichiro Nakanishi, Dirk Petry, Thomas Stanke, Catarina Ubach, Luciano Cerrigone, Erica Keller, Alfonso Trejo, Junko Ueda

Author Affiliations +

Proceedings Volume 11449, Observatory Operations: Strategies, Processes, and Systems VIII; 114491T (2020) https://doi.org/10.1117/12.2559324
Event: SPIE Astronomical Telescopes + Instrumentation, 2020, Online Only

Abstract

ALMA (Atacama Large Millimeter/submillimeter Array) is the world's largest ground-based facility for observations in the millimeter/submillimeter regime. One of ALMA's outstanding characteristics is the large effort dedicated to the quality assurance (QA) of the calibrated and imaged data products offered to the astronomical community. The Data Management Group (DMG), in charge of the data processing, review, and delivery of the ALMA data, consists of approximately 60 experts in data reduction, from the ALMA Regional Centers (ARCs) and the Joint ALMA Observatory (JAO), distributed in fourteen countries. With a throughput of more than 3,000 datasets per year, meeting the goal of delivering the pipeline-able data products within 30 days after data acquisition is a huge challenge. This paper presents (a) the history of data processing at ALMA, (b) the challenges our team had and is still facing, (c) the methodology followed to mitigate the operational risks, (d) the ongoing optimization initiatives, (e) the current data processing status, (f) the strategy which is being followed so that, in a few Cycles from now, a team of approximately 10 data reducers (DRs) at JAO can process and review some 80% of the datasets collected during an observing cycle, and, finally, (g) the important role of the ARCs for processing the remaining datasets.

Conference Presentation

Citation Download Citation

Theodoros Nakos, Harold Francke, Kouichiro Nakanishi, Dirk Petry, Thomas Stanke, Catarina Ubach, Luciano Cerrigone, Erica Keller, Alfonso Trejo, and Junko Ueda "Improving ALMA’s data processing efficiency using a holistic approach", Proc. SPIE 11449, Observatory Operations: Strategies, Processes, and Systems VIII, 114491T (15 December 2020); https://doi.org/10.1117/12.2559324

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available