Climate Change Monitoring and Detection
NOAA/JCSDA – The Use of Kernel Methods in Data Selection and Thinning for Satellite Data Assimilation in NWP Models
Funding Type: CIMMS Task III (Program Manager – Dr. Fuzhong Weng)
Objectives
Funded by the Joint Center for Satellite Data Assimilation, the main
objective of the first part of task is to thin the WindSat data; for
the second part, the main goal is ingestion and assimilation of scatterometer
data from the QuikSCAT instrument.
Accomplishments
Data thinning. To accomplish this task, we apply Support vector machines
(SVMs) and then keep only support vectors. To show the quality of those
support vectors as predictors, we reconstruct the wind field and then
compare it to the observed values. SVMs are a family of learning algorithms
used in supervised learning tasks such as statistical classification
and regression analysis. To use a SVM method, we need to solve a quadratic
programming problems with linear constraints. Therefore, the number
of data points used during the training period is critical for the
speed of the algorithm. In this work, given the massive number of satellite
data, we use Voronoi tessellation to make the analysis efficient.
The primary data provided by WindSat are Sea Surface Wind Speed (SSWS) and Sea Surface Wind Direction (SSWD). In addition to ocean surface wind vector, the WindSat system will provide a host of secondary ocean-scene environmental data products. These products include column integrated Cloud Liquid Water (CLW), column integrated Precipitable Water (PW), and Sea Surface Temperature (SST).
Various experiments were performed. Initially, we used data from 1 January 2005 over the domain spanning the region defined by (127W, 145E) and (23N, 42N). After we removed all data points that have some missing information, we were left with 13,540 points. Previous work on imputation of missing data (Richman et al. 2007) suggests that support vector imputation can be applied to these data to improve the results further for reducing the error in the mean and the variance.
The key results to date for data thinning are that:
- In our experiments we used Support Vector Regression is used successfully to thin the WindSat data.
- Unlike normal quadratic programming, which would not be able to handle the massive data sets generated from WindSat, by applying linear constraints and Voronoi tesselation, the SVM approach sales up efficiently. In the largest experiment attempted, 24 hours of satellite data with over 1.6 million data points, the programs made excellent predictions on a desktop computer in minutes.
- The results obtained show that fewer than 15% of the data are needed to reconstruct the wind field with high accuracy (the correlation coefficient is greater than +0.99)
- The MAE of both the U- and V-components is < 0.6 m/s. This is in the range of uncertainty of the wind observations.
- The geographical distribution of the winds match well to the observed winds at various time scales.
This project is ongoing.
Average MSE for six methods for 5 iterations when 20% of the observations have been thinned (left) and bar chart illustrating the difference of variance between the original and imputed data for 5, 10 and 20% data thinning (right).
Assimilation of satellite data. Work is continuing on the ingestion and assimilation of scatterometer data from the QuikSCAT instrument. The initial results had a large positive impact on the forecast skill of the OU-HIRES NWP model when the scatterometer data are thinned and included (Fig. 10). The 48 hour forecast error in the OU-HIRES model with the scatterometer data was 146 km compared with 267 km without the scatterometer data. (Leslie and Buckley, 2006).
Recently, work has focused more on the impact of QuikSCAT data over the data sparse East Indian Ocean. The assimilation has been applied to tropical cyclones, mid-latitude cyclones and strong frontal systems. The QuikSCAT data were necessary to obtain realistic results. In the absence of these data, early detection and issuance of marine weather warnings would not have been possible. When the QuikSCAT data were included, the complete life cycles of the all systems (particularly intensification to severe status) was identifiable and verified with observations available in real-time and when postanalysis data became available (Leslie and Buckley, 2007). In the more recent work, the focus was on the impact of QuikSCAT data for the preparation of accurate operational weather forecasts and the timely issuance of severe marine weather and ocean warnings and advisories for major oceanic weather systems, affecting both coastal areas and the open ocean, and which are major forecasting problems facing the all marine meteorology weather programs and f0recast centers. In Leslie and Buckley (2007), all results were for the Australian Bureau of Meteorology's Regional Forecast Centre (RFC) and its colocated Tropical Cyclone Warning Centre (TCWC) in Perth, Western Australia. The region of responsibility for the Perth RFC is vast, covering a large portion of the southeast Indian and Southern Oceans, both of which are extremely data sparse, especially for near-surface marine wind data. Given that these coastline and open ocean areas are subject to some of the world's most intense tropical cyclones, rapidly intensifying mid-latitude cyclones, and powerful cold fronts (that frequent the southwest corner of Australia), there is now a heavy reliance upon the QuikSCAT data for both routine and severe weather warning forecasts.
The main findings were the need for QuikSCAT data in the Perth RFC for accurate and early detection of maritime severe weather systems, both tropical and extra-tropical. First the role of the QuikSCAT data was described, and then three cases were presented in which the QuikSCAT data were pivotal in providing forecast guidance. The cases are a severe tropical cyclone in its development phase off the northwest coast of Australia, a strong southeast Indian Ocean cold front, and an explosively developing mid-latitude Southern Ocean cyclone. In each case, the Perth RFC would have been unable to provide the early and high quality forecast and warning guidance without the timely availability of the near surface wind data made possible by QuikSCAT.
This project is ongoing.
Operational track of Severe Tropical Cyclone Clare (January 2006) with HIRES 12 hourly forecasts shown as stars out to 1200 UTC 10 January (times correspond to adjacent analysis positions).
Publications
Leslie, L.M. and B.W. Buckley, 2006: Scatterometer-based assessment of 10-m
wind analyses from the operational ECMWF and NCEP numerical weather prediction
models. Mon. Wea. Rev. 134, 737-742.
Leslie, L.M. and B.W. Buckley, 2007: The Operational Impact of QuikSCAT Winds in Perth, Australia: Examples and Limitations. Wea. Forecasting, in press.
Mansouri, H., L. Leslie, R. Gilbert, M.B. Richman and T.B. Trafalis, 2007: Ocean surface wind vector forecasting using support vector regression. Intelligent Engineering Systems Through Artificial Neural Networks, (C.H. Dagli, A.L. Buczak, J. Ghosh, M.J. Embrechts, O. Ersoy, and S.W. Kercel, eds.), ASME Press, submitted.
Richman, M.B., T.T. Trafalis and I. Adrianto, 2007: Multiple imputation through machine learning algorithms. Fifth Conf. on Artificial Intelligence Applications to Environmental Science. Amer. Meteor. Soc., J3.9.