Araya Mauricio

Abstract Reference: 30759
Identifier: O11.2
Presentation: Oral communication
Key Theme: 6 Python in Astronomy

JOVIAL: Jupyter OVerrIde for Astronomical Libraries

Authors:
Araya Mauricio, HochfarberTeodoro, Valenzuela Camilo, Farías Humberto, Solar Mauricio, 

Jupyter notebooks have proven to be a powerful tool for teaching science, because they handle executable code snippets, context environments, visualization, equations and explanatory text, all within the same web-interface. We propose extending its usage from teaching activities to science activities through the integration of astronomical libraries to the Jupyter environment and its execution as a cloud service. The remote execution of processing tasks through notebooks not only simplifies the usage of libraries, but enables Big Data processing on data centers while hiding the infrastructure and platform-dependent details. This allows moving computations closer to the data archives, executing tasks in a graphical yet non-blocking fashion, using high-performance computing routines transparently and working with very large data files without exhausting local memory. Moreover, a proper use of the notebooks produces self-documented, exportable and reproducible pipelines with graphical support. Our prototype uses Jupyter Hub as the base service, including Astropy, ACALib and CASAC libraries to the Python kernel, and we are currently porting CUPID (Starlink), MPICASA and ADMIT through suitable wrappers. The integration with the VO is two-fold: through the data access layer services via VOTables, and through the application layer via the SAMP protocol. Besides including more libraries, the next steps are exploring the integration with Big Data frameworks such as Apache Spark, and the proper parallelization of the main algorithms of the libraries. To encourage its usage we also plan to assemble a lite distribution of JOVIAL to be used off-line.