Armstrong, E. M., Bourassa, M. A., Cram, T. A., DeBellis, M., Elya, J., Greguska III, F. R., et al. (2019). An Integrated Data Analytics Platform.
Front. Mar. Sci., 6, 354.
Abstract: An Integrated Science Data Analytics Platform is an environment that enables the confluence of resources for scientific investigation. It harmonizes data, tools and computational resources to enable the research community to focus on the investigation rather than spending time on security, data preparation, management, etc. OceanWorks is a NASA technology integration project to establish a cloud-based Integrated Ocean Science Data Analytics Platform for big ocean science at NASA’s Physical Oceanography Distributed Active Archive Center (PO.DAAC) for big ocean science. It focuses on advancement and maturity by bringing together several NASA open-source, big data projects for parallel analytics, anomaly detection, in situ to satellite data matchup, quality-screened data subsetting, search relevancy, and data discovery. Our communities are relying on data available through distributed data centers to conduct their research. In typical investigations, scientists would (1) search for data, (2) evaluate the relevance of that data, (3) download it, and (4) then apply algorithms to identify trends, anomalies, or other attributes of the data. Such a workflow cannot scale if the research involves a massive amount of data or multi-variate measurements. With the upcoming NASA Surface Water and Ocean Topography (SWOT) mission expected to produce over 20PB of observational data during its 3-year nominal mission, the volume of data will challenge all existing Earth Science data archival, distribution and analysis paradigms. This paper discusses how OceanWorks enhances the analysis of physical ocean data where the computation is done on an elastic cloud platform next to the archive to deliver fast, web-accessible services for working with oceanographic measurements.
Freeman, E., Kent, E. C., Brohan, P., Cram, T., Gates, L., Huang, B., et al. (2019). The International Comprehensive Ocean-Atmosphere Data Set – Meeting Users Needs and Future Priorities.
Front. Mar. Sci., 6, 435.
Abstract: The International Comprehensive Ocean-Atmosphere Data Set (ICOADS) is a collection and archive of in situ marine observations, which has been developed over several decades as an international project and recently guided by formal international partnerships and the ICOADS Steering Committee. ICOADS contains observations from many different observing systems encompassing the evolution of measurement technology since the 18th century. ICOADS provides an integrated source of observations for a range of applications including research and climate monitoring, and forms the main marine in situ surface data source, e.g., near-surface ocean observations and lower atmospheric marine-meteorological observations from buoys, ships, coastal stations, and oceanographic sensors, for oceanic and atmospheric research and reanalysis. ICOADS has developed ways to incorporate user and reanalyses feedback information associated with permanent unique identifiers and is also the main repository for data that have been rescued from ships’ logbooks and other marine data digitization activities. ICOADS has been adopted widely because it provides convenient access to a range of observation types, globally, and through the entire marine instrumental record. ICOADS has provided a secure home for such observations for decades. Because of the increased volume of observations, particularly those available in near-real-time, and an expansion of their diversity, the ICOADS processing system now requires extensive modernization. Based on user feedback, we will outline the improvements that are required, the challenges to their implementation, and the benefits of upgrading this important and diverse marine archive and distribution activity.
Freeman, E., Woodruff, S. D., Worley, S. J., Lubker, S. J., Kent, E. C., Angel, W. E., et al. (2017). ICOADS Release 3.0: a major update to the historical marine climate record.
Int. J. Climatol., 37(5), 2211–2232.
Woodruff, S. D., Worley, S. J., Lubker, S. J., Ji, Z., Eric Freeman, J., Berry, D. I., et al. (2011). ICOADS Release 2.5: extensions and enhancements to the surface marine meteorological archive. In
International Journal of Climatology (Vol. 31, pp. 951–967).