On Learning Multiple Descriptions of a Concept

In Proceedings of "Tools with Artificial Intelligence", 1994. New
Orleans, LA. 

Kamal Ali, Clifford Brunk and Michael Pazzani
Department of Information and Computer Science,
University of California, Irvine, CA, 92717
{ali,brunk,pazzani}@ics.uci.edu 
714-725-3491, 714-856-5888

In sparse data environments, greater classification accuracy can be achieved
by learning several concept descriptions of the data and combining their
classifications.
Stochastic search is a general tool which can be used 
to generate many good concept descriptions (rule sets) for each
class in the data. 
Bayesian probability theory offers an optimal strategy for combining
classifications of the individual concept descriptions, and here we use an approximation
of that theory.
This strategy is most useful when additional data is difficult to obtain and
every increase in classification accuracy is important.
The primary result of this paper is that multiple concept descriptions are
particularly helpful in ``flat'' hypothesis spaces in which there are many
equally good ways to grow a
rule, each having similar gain.
Another result is experimental evidence that learning
multiple rule sets yields more accurate classifications than learning multiple rules
for some domains.
To demonstrate these behaviors, we learn multiple concept descriptions by adapting
HYDRA, a noise-tolerant relational learning algorithm.