Results and Discussion

The BWER algorithm has a rule base consisting of twenty fuzzy sets. Each of these sets, as described in Section a., is parameterized by two numbers $x_1$ and $x_2$ which are subject to tuning by the genetic algorithm. These numbers are determined by $x_{min}$ and $x_{max}$, the extremes that the underlying value can take in the real world, the quantization, $x_{quant}$, we choose and on the values that the GA assigns to $x_1$ and $x_2$. The form of the membership function will never get changed as a result of optimization. The GA can choose whether ``low'' should mean $35~dBZ$ or $25~dBZ$ in the context in which it is used but cannot change the word to ``high''.

By abstracting away the actual numbers that decide the rules, we gain the ability to allow an algorithm to be used without change in different contexts. For example, the BWER algorithm is used both to provide inputs to a neural network for vortex prediction and to produce output that is to be seen by a forecaster. The performance of the algorithm should be different for these two contexts. The neural network expects to see everything, including low-confidence estimates. A human forecaster will be less tolerant. Tuning the BWER algorithm to produce these two estimates was simple - all that needed to be done was to change the fitness function. The fitness measure shown in Eq. 11 is what was used for the neural network. To tune the algorithm for display purposes, we defined the fitness function as:

$$fitness = 0.7 * HSS_r + 0.3 * HSS$$ (13)

where the Heidke Skill Score, HSS, is defined as: Marzban (1998); Heidke (1926)

$$HSS = \frac{2(null*hit-fa*miss)}{(null+fa)(fa+hit)+(null+miss)(miss+hit)}$$ (14)

The values of $null$, $hit$, $miss$ and $fa$ are graduated skill measures for $HSS$ and the numbers obtained after thresholding the detections at a 0.75 confidence level for $HSS_r$. The skill of the algorithm in producing high-confidence detections is given more weight when the end user is a forecaster than when the end user is a neural network.⁸

The BWER algorithm was tuned over five cases containing 200 volumes of radar reflectivity data. There were a total of 131 BWERs in the five cases. The rapid improvement in the average fitness of a chromosome in the first few generations can be clearly seen in Fig. 4. The fitness measure reported is as calculated in Eq. 11.

Figure 4: The performance of an average chromosome in a generation and of the best chromosome in each generation. The improvement in the performance of chromosomes is dramatic in the first few generations as the GA discovers and populates the best regions in the search space.

Are the chromosomes generated by the GA general? To test the generality of the chromosomes output by the tuning process, we split the set of 200 volumes into 170 volumes for tuning and 30 for testing. We then ran the GA on the tuning cases and tested the best chromosome reported by the GA on the test cases. The results obtained when the experiment was performed five times (with the same 170 volumes for tuning and the same 30 volumes for testing) are shown in Table 1. One test of whether the chromosomes produced by the GA are general is whether the performance of a chromosome on the training cases is positively correlated with the performance of the chromosome on the test cases. The correlation coefficient can be computed from Table 1 to be 0.82 for the HSS and 0.85 for the CSI. Since the correlation coefficients are close to 1.0, we can deduce that the chromosomes generated by the GA are general.

Table 1: Performance of the chromosome obtained by tuning the BWER algorithm using the GA. The numbers are traditional CSIs and HSSs obtained by thresholding the detections at a 0.75 confidence level.

GA Runs	Training	Training	Test	Test
	CSI	HSS	CSI	HSS
1	0.435	0.603	0.467	0.633
2	0.473	0.634	0.500	0.663
3	0.382	0.548	0.390	0.557
4	0.404	0.572	0.308	0.467
5	0.364	0.529	0.168	0.483

Are the performances of the best chromosomes in each run of the GA consistent? No. Again from Table 1, we can calculate that the standard deviation in the training HSS is about 7% of its mean while that in the CSI is about 11% of its mean. For the test cases, these go up to 16% and 36% respectively. The Genetic Algorithm doesn't promise optimality and that lack of a guarantee is seen in the high variability of the results that are obtained with successive runs of the GA. Each run of the GA will give you a good chromosome but you can never be sure that the next run will not give you an even better one.

A more detailed table describing the performance of the BWER detection algorithm, including the number of hits, misses and false alarms on individual storm days, can be found in Lakshmanan and Witt (1997).