Kim Jai Won

Abstract Reference: 30832
Identifier: D6
Presentation: Demo Booth organisational
Key Theme: 6 Python in Astronomy 

SciServer/Compute: Bring Analysis Close to the Data

Authors:
Kim Jai Won, Lemson Gerard

We present a demo that illustrates the capabilities of SciServer/Compute. SciServer/Compute is a component of SciServer, a big-data infrastructure project developed at Johns Hopkins University that will provide a common environment for sharable computational research.

SciServer/Compute uses Jupyter notebooks running within server-s¬ide Docker containers attached to big data collections in relational databases and file storage to bring advanced analysis capabilities close to the data. Apart from interactive notebooks in Python, R and MATLAB, SciServer/Compute offers an API for running asynchronous tasks, also in Docker containers.

SciServer/Compute contains custom libraries for accessing databases available in CasJobs as well as various storage systems. SciServer/MyScratch provides terabytes of scratch storage spaces and SciServer/SciDrive offers a Dropbox like service for long term storage of scientific results. These components are accessible through a single-sign-on mechanism.

SciServer supports a range of scientific disciplines and it integrates large existing databases and file collections in the fields of astronomy, cosmology, turbulence, genomics, oceanography and materials science.

The demo will highlight the data flow between various components of SciServer including file storage, database, and compute with data sets from diverse scientific fields.