Appeared in Nat. Wea. Dig., 15, No. 3, 29-30
Corresponding author address: Dr. Charles A. Doswell III, National Severe Storms Laboratory, 1313 Halley Circle, Norman, OK 73069 E-mail: email@example.com
There is considerable value in assessing the impact of ASOS on the quality of weather forecasts, and this was the stated aim of the Kansas Pilot Project (KaPP) described in M90. The "rules of engagement" in KaPP were designed to isolate (to the greatest extent possible) forecasts made exclusively with ASOS from those made with "conventional" surface observations. The results show little or no difference between them, which led the authors to conclude there was "... no evidence that the information contained in the remarks appended to surface aviation observations is essential for the issuing of day-to-day forecasts." By obvious implication, the additive remarks have no value, in the opinion of the authors.
I have used the word "implication" because the authors have been careful to avoid any direct statement to this effect; however, the conclusions in the paper leave little doubt about their implications. I certainly commend the authors for conducting a test of the impact of ASOS on forecasting, but I dispute the implied lack of value in additive remarks. I believe I have an obligation to register my concerns for the possible misapplications of the M90 study. My concerns fall into three broad areas.
First, it is not obvious that there must necessarily be a direct connection between the value of additive remarks and the quality of temperature, ceiling, visibility, and sky cover forecasts. It seems to me that if one wished to determine the value of additive remarks, it would make more sense to ask forecasters directly about their use of additive remarks. Had the authors determined somehow (e.g., a survey) what products (if any) were influenced by additive remarks, then a direct test involving those products might have been more conclusive about the real value of additive remarks in the surface observations. I can think of forecast products other than the ones chosen in M90 that might suffer without additive remarks in the surface observations (e.g., convective weather warnings or precipitation forecasts). It also may be that analysis and interpetation of the surface observations suffer without the remarks.
Second, the sample size may have been too small to be certain that additive remarks have no value, even for M90's particular selection of forecast products. It could be that the particular group of forecasters involved in this test were not particularly well-trained in using those remarks. Also, it is obvious from the relatively low frequency of significant weather during KaPP that the sample may not indicate the importance of remarks during important weather events. It is not clear from the "rules of engagement" whether or not all forecasters involved had roughly equal numbers of forecasts with and without the additive remarks. If the "with" and "without" forecasters were kept separate thoughout the KaPP, the sample size is correspondingly reduced further. In either case, while the authors make no truly exaggerated claims about the generality of their results, my experience suggests that there are those in decision-making positions who are all too ready to make that claim, anyway.
Third, I disagree strongly with the implication that if additive remarks are inconsistent, then they have no value. I realize that the authors have not made this claim directly, but again I believe that many might draw this conclusion on the basis of M90's results. Let me suggest a couple of analogies. If all forecasts are not consistent, in the sense that forecasters using the same input should arrive at the same forecast (and they definitely do not!), then should we question the value of forecasts? If our automobile doesn't start every time, can we conclude that it is a piece of junk, worthy only of being cast aside at the earliest opportunity? If consistency is a problem with the additive remarks, then an alternative would be to fix that particular problem rather than to brand the remarks as useless and eliminate them entirely.
I am concerned about the present obsession with objectivity and automation that permeates the National Weather Service today. It seems that subjectivity and thoughtful input is considered unquestionably inferior to objectivity and automation in virtually every instance of their plans for modernization. The dictionary definition of "objective" essentially means something that is done without thought. I hate to think what might happen if the National Weather Service succeeds in removing thought from the processes of observing, analyzing, and forecasting the weather, replacing it with genuine objectivity (i.e., mindlessness). We already have experienced the negative impact of automation with regard to upper-air observations (see Schwartz, 1990). Must we experience it again with surface data?
While I cannot speak about the intentions of the authors of M90, it seems all too obvious to me that there are those involved with National Weather Service modernization planning who will seize upon this study as a general indication of their wisdom in advocating a fully automated observing system. Such an interpretation is not warranted by the results of M90; both the design and execution of the experiment leave too many questions unanswered.
I cannot dispute the value of automated observations, especially as a supplement to the present set of surface observations. There is no intent on my part to slow down or prevent the implementation of ASOS, but I am not convinced by this study that we no longer need additive remarks. The questions I have raised need to be answered. This requires testing along the lines I have indicated, and neither I nor anyone else has done that testing.
McNulty, R.P., J.T. Schaefer, W.E. Sunkel, and T.A. Townsend, 1990: On the need for augmentation in automated surface observations. Nat. Wea. Dig., 15, No. 1, 9-16.
Schwartz, B.E., 1990: Regarding the automation of rawinsonde observations. Wea. Forecasting, 5, 167-171.