Full Content is available to subscribers

Subscribe/Learn More  >
Proceedings Article

An efficient framework for Java data processing systems in HPC environments

[+] Author Affiliations
Aidan Fries, Javier Castañeda, Yago Isasi, Jordi Portell de Mora

Univ. of Barcelona (Spain) and Institute for Space Studies of Catalonia (Spain) and Institute of Cosmos Sciences (Spain)

Guillermo L. Taboada

Univ. of A Coruña (Spain)

Raül Sirvent

BSC-CNS Barcelona Supercomputing Ctr. (Spain)

Proc. SPIE 8183, High-Performance Computing in Remote Sensing, 818306 (October 11, 2011); doi:10.1117/12.898217
Text Size: A A A
From Conference Volume 8183

  • High-Performance Computing in Remote Sensing
  • Bormin Huang; Antonio J. Plaza
  • Prague, Czech Republic | September 19, 2011

abstract

Java is a commonly used programming language, although its use in High Performance Computing (HPC) remains relatively low. One of the reasons is a lack of libraries offering specific HPC functions to Java applications. In this paper we present a Java-based framework, called DpcbTools, designed to provide a set of functions that fill this gap. It includes a set of efficient data communication functions based on message-passing, thus providing, when a low latency network such as Myrinet is available, higher throughputs and lower latencies than standard solutions used by Java. DpcbTools also includes routines for the launching, monitoring and management of Java applications on several computing nodes by making use of JMX to communicate with remote Java VMs. The Gaia Data Processing and Analysis Consortium (DPAC) is a real case where scientific data from the ESA Gaia astrometric satellite will be entirely processed using Java. In this paper we describe the main elements of DPAC and its usage of the DpcbTools framework. We also assess the usefulness and performance of DpcbTools through its performance evaluation and the analysis of its impact on some DPAC systems deployed in the MareNostrum supercomputer (Barcelona Supercomputing Center).

© (2011) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.
Citation

Aidan Fries ; Javier Castañeda ; Yago Isasi ; Guillermo L. Taboada ; Jordi Portell de Mora, et al.
"An efficient framework for Java data processing systems in HPC environments", Proc. SPIE 8183, High-Performance Computing in Remote Sensing, 818306 (October 11, 2011); doi:10.1117/12.898217; http://dx.doi.org/10.1117/12.898217


Access This Proceeding
Sign in or Create a personal account to Buy this proceeding ($15 for members, $18 for non-members).

Figures

Tables

NOTE:
Citing articles are presented as examples only. In non-demo SCM6 implementation, integration with CrossRef’s "Cited By" API will populate this tab (http://www.crossref.org/citedby.html).

Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging & repositioning the boxes below.

Related Book Chapters

Topic Collections

Advertisement
  • Don't have an account?
  • Subscribe to the SPIE Digital Library
  • Create a FREE account to sign up for Digital Library content alerts and gain access to institutional subscriptions remotely.
Access This Proceeding
Sign in or Create a personal account to Buy this proceeding ($15 for members, $18 for non-members).
Access This Proceeding
Sign in or Create a personal account to Buy this article ($15 for members, $18 for non-members).
Access This Chapter

Access to SPIE eBooks is limited to subscribing institutions and is not available as part of a personal subscription. Print or electronic versions of individual SPIE books may be purchased via SPIE.org.