I have 3 classes and hundreds of attributes each with ~50 values
(continuous, but discretized based on quartiles).  After calculating
information gain for each attribute I sort the information gain in
descending order.  My goal is not to generate a tree, but rather to perform
instance-based learning using the cumulative list of attributes selected.

Something I have not seen in the literature is what to do if a majority of
the attributes with the greatest information gain have less impurity but in
one particular class.  Given this problem, is there a commonly used method
for weighting or selecting attributes which are the purest for a class?  (I
have tried selecting attributes with the greatest gain for each class,
looping through 3 classes each time I select the next best attribute, and
that seemed to work better than just selecting attributes with the greatest
gain).  Do I need to "prune" unwanted attributes?  If so, are there any
papers which show background methods and criteria for pruning unwanted
attributes in instance-based learning?

Last, another remaining question is that because my goal is not really to
build a hierarchical tree, each time I select an attribute I use the
accumulated attribute data and loop through all of the objects (train) in
order to assign each object to the predicted class.  Each time I add an
attribute, a confusion matrix is generated for classification of all the
objects -- from which I obtain accuracy.  So I get a confusion matrix for
the cumulative list of attributes at each step.  In this scenario, when
should I stop selecting attributes?  Recall that I am not building a tree
for which I can assess purity in each node, but rather picking off
attributes to train and generate a confusion matrix in instance-based
learning.