Pazzani, M., Murphy, P., Ali, K. and Schulenburg, D. (1994). Trading off coverage for accuracy in forecasts: Applications to clinical data analysis. AAAI Symposium on AI in Medicine (pp 106-110). Stanford, CA. Machine learning algorithms that perform classification tasks create concept descriptions (e.g., rules, decision trees, or weights on a neural net) guided by a set of classified training examples. The accuracy of the resulting concept description can be evaluated empirically by asking a classifier to use the concept description to classify a set of test examples. One goal of this evaluation is to determine the accuracy of the learning algorithm if it were applied in a "real" application. In such an application, it is assumed that a classifier is produced on a set of training cases and a decision is made automatically on each new case based upon a forecast of the classification of the case. However, in some applications, it is advantageous not to produce a classification on every example. In particular, when a machine learning program is being used to assist a person to perform some task, it might be desirable to have the machine automatically make some decisions while allowing others to be made by the person. The general idea is that in some cases, a user of a machine learning system might be willing to allow the learning system to produce no classification on some examples. This may be acceptable if it means that when the system makes a prediction, the prediction is more accurate.