The PAU Survey: Operation and orchestration of multi-band survey data

• Published in: Astronomy and computing Vol. 27; pp. 171 - 188
• Main Authors: Tonello, N., Tallada, P., Serrano, S., Carretero, J., Eriksen, M., Folger, M., Neissner, C., Sevilla-Noarbe, I., Castander, F.J., Delfino, M., De Vicente, J., Fernandez, E., Garcia-Bellido, J., Gaztanaga, E., Padilla, C., Sanchez, E., Tortorelli, L.
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01.04.2019
• Subjects: Cosmological survey; Data management; Data modeling; High throughput computing; Relational database; Web application; Data management; High throughput computing; Data modeling; Cosmological survey; Web application; Relational database
• ISSN: 2213-1337; 2213-1345
• DOI: 10.1016/j.ascom.2019.04.002

The Physics of the Accelerating Universe (PAU) Survey is an international project for the study of cosmological parameters associated with Dark Energy. The PAU 18-CCD camera (PAUCam) is installed at the primary focus of the William Herschel Telescope at Observatorio del Roque de los Muchachos (La Palma, Canary Islands). The camera incorporates a unique set of 40 narrow band filters in the optical range from 450 nm to 850 nm and a set of 6 standard broad band filters. It is used to scan part of the northern sky in order to collect low resolution spectral information from millions of galaxies. The PAU data management (PAUdm) team is in charge of treating the data, including data transfer from the observatory to the PAU Survey data center, hosted at Port d’Informació Científica (PIC). PAUdm is also in charge of the storage, data reduction and, finally, of making the results available to the scientific community. We describe the technical solutions adopted to cover different aspects of the PAU Survey data management, from the computing infrastructure to the software tools and web services for the processing orchestration and data exploration. We give special attention to the PAU database, developed to preserve and guarantee the consistency of data and metadata and also used to coordinate the different PAUdm tasks.