Category Archives: Data Mining by Ian H. Witten and Eibe Frank

Support Vector Machines versus Neural Networks

“Interest in neural networks appears to have declined since the arrival of support vector machines, perhaps because the latter generally require fewer parameters to be tuned to achieve the same (or greater) accuracy. However, multilayer perceptrons have the advantage that they can learn to ignore irrelevant attributes, and RBF networks trained using k-means can be viewed as a quick-and-dirty method for finding a nonlinear classifier.” (p. 235)

Epicurus, Multiple Explanations, Data Mining Implications

The Greek philosopher Epicurus…expressed almost the opposite sentiment [to Occam’s razor]. His principle of multiple explanations advises “if more than one theory is consistent with the data, keep them all” on the basis that if several explanations are equally in agreement, it may be possible to achieve a higher degree of precision by using them together–and anyway, it would be unscientific to discard some arbitrarily. This brings to mind instance-based learning, in which all the evidence is retained to provide robust predictions, and resonates stronly with decision combination methods such as bagging and boosting that actually do gain predictive power using multiple explanations together.” (p.183)