One approach that's often used is to calculate the between to within mean
square ratio (as in a one-way ANOVA), then calculate the F ratio and
p-value for each cut, and then use the cut with the smallest
p-value. This is at best a useful heuristic, however-- probably the best
approach is to choose the clustering that is most interpretable
substantively, or to use a technique such as K-means designed to find a
simple clustering (or partition) in the first place! (K-means explicitly
maximizes the F ratio discussed above for a partitioning into K clusters,
so you may want to use the relative p-values as well as interpretability
and other criteria to choose the appropriate value of K.)
Doug Carroll
At 12:49 PM 5/5/2004 -0500, you wrote:
>Dear all,
>
>I am now working on the hierarchical clustering methods, and
>confused about the following problem.
>
>As you know, to form clustering from the hierarchical tree generated by
>the pairwise distance bw the elements, we have to set a threshold value
>to cut the tree horizonally such that the vertical links intersecting with
>this horizonal critical value will be the final clusters.
>
>However, I do not find a very robust criterion for choosing the
>optimal number of clusters or calculating this threshold value to make the
>clustering results good different pairwise distance(similairty) measure.
>
>So any one has some point on this problem or recommended papers
>or methods?
>
>Thanks for your help.
>
>Fred
>
>
>
######################################################################
# J. Douglas Carroll, Board of Governors Professor of Management and #
#Psychology, Rutgers University, Graduate School of Management, #
#Marketing Dept., MEC125, 111 Washington Street, Newark, New Jersey #
#07102-3027. Tel.: (973) 353-5814, Fax: (973) 353-5376. #
# Home: 14 Forest Drive, Warren, New Jersey 07059-5802. #
# Home Phone: (908) 753-6441 or 753-1620, Home Fax: (908) 757-1086. #
# E-mail: [log in to unmask] #
######################################################################
|
