TY - GEN
T1 - The Matsu Wheel
T2 - 2nd IEEE International Conference on Big Data Computing Service and Applications, BigDataService 2016
AU - Patterson, Maria T.
AU - Anderson, Nikolas
AU - Bennett, Collin
AU - Bruggemann, Jacob
AU - Grossman, Robert L.
AU - Handy, Matthew
AU - Ly, Vuong
AU - Mandl, Daniel J.
AU - Pederson, Shane
AU - Pivarski, James
AU - Powell, Ray
AU - Spring, Jonathan
AU - Wells, Walt
AU - Xia, John
N1 - Publisher Copyright:
© 2016 IEEE.
PY - 2016/5/19
Y1 - 2016/5/19
N2 - Project Matsu is a collaboration between the Open Commons Consortium and NASA focused on developing open source technology for the cloud-based processing of Earth satellite imagery. A particular focus is the development of applications for detecting fires and floods to help support natural disaster detection and relief. Project Matsu has developed an open source cloud-based infrastructure to process, analyze, and reanalyze large collections of hyperspectral satellite image data using Open-Stack, Hadoop, MapReduce, Storm and related technologies. We describe a framework for efficient analysis of large amounts of data called the Matsu "Wheel." The Matsu Wheel is currently used to process incoming hyperspectral satellite data produced daily by NASA's Earth Observing-1 (EO-1) satellite. The framework is designed to be able to support scanning queries using cloud computing applications, such as Hadoop and Accumulo. A scanning query processes all, or most of the data, in a database or data repository. We also describe our preliminary Wheel analytics, including an anomaly detector for rare spectral signatures or thermal anomalies in hyperspectral data and a land cover classifier that can be used for water and flood detection. Each of these analytics can generate visual reports accessible via the web for the public and interested decision makers. The resultant products of the analytics are also made accessible through an Open Geospatial Compliant (OGC)-compliant Web Map Service (WMS) for further distribution. The Matsu Wheel allows many shared data services to be performed together to efficiently use resources for processing hyperspectral satellite image data and other, e.g., large environmental datasets that may be analyzed for many purposes.
AB - Project Matsu is a collaboration between the Open Commons Consortium and NASA focused on developing open source technology for the cloud-based processing of Earth satellite imagery. A particular focus is the development of applications for detecting fires and floods to help support natural disaster detection and relief. Project Matsu has developed an open source cloud-based infrastructure to process, analyze, and reanalyze large collections of hyperspectral satellite image data using Open-Stack, Hadoop, MapReduce, Storm and related technologies. We describe a framework for efficient analysis of large amounts of data called the Matsu "Wheel." The Matsu Wheel is currently used to process incoming hyperspectral satellite data produced daily by NASA's Earth Observing-1 (EO-1) satellite. The framework is designed to be able to support scanning queries using cloud computing applications, such as Hadoop and Accumulo. A scanning query processes all, or most of the data, in a database or data repository. We also describe our preliminary Wheel analytics, including an anomaly detector for rare spectral signatures or thermal anomalies in hyperspectral data and a land cover classifier that can be used for water and flood detection. Each of these analytics can generate visual reports accessible via the web for the public and interested decision makers. The resultant products of the analytics are also made accessible through an Open Geospatial Compliant (OGC)-compliant Web Map Service (WMS) for further distribution. The Matsu Wheel allows many shared data services to be performed together to efficiently use resources for processing hyperspectral satellite image data and other, e.g., large environmental datasets that may be analyzed for many purposes.
UR - http://www.scopus.com/inward/record.url?scp=84973617913&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84973617913&partnerID=8YFLogxK
U2 - 10.1109/BigDataService.2016.39
DO - 10.1109/BigDataService.2016.39
M3 - Conference contribution
AN - SCOPUS:84973617913
T3 - Proceedings - 2016 IEEE 2nd International Conference on Big Data Computing Service and Applications, BigDataService 2016
SP - 156
EP - 165
BT - Proceedings - 2016 IEEE 2nd International Conference on Big Data Computing Service and Applications, BigDataService 2016
PB - Institute of Electrical and Electronics Engineers Inc.
Y2 - 29 March 2016 through 1 April 2016
ER -