langreiter.com plain, simple: 2008-07-13-randomForestry

LANGREITER.COM plain, simple

START

INDEX

2008-07-13-randomForestry

CREATED BY chris • LAST EDITED BY chris 6005 days AGO

A Conversation with Leo Breiman (2001)

"The interests and accomplishments of Leo Breiman extend outside the areas of professional statistician and probabilist. He was a waiter in the Catskills, a dishwasher in the Merchant Marine, a trekker into the heart of rainforest Africa, an active father to many children from a small agrarian Mexican village, a member and President of the Santa Monica School Board, the architect of his stunning home and an accomplished sculptor."

If you're only tangentially interested in ensemble methods like Bagging, [create AdaBoost] &c., this is highly recommended reading.

"On most data sets that people have looked at, Adaboost did quite a good deal better than bagging did. This was a startling discovery because you could take a sow's ear and transform it into a silk purse. That is, you could take a classiﬁer like, say, everyday vanilla CART, which was good but not a great classiﬁer, and by using this Adaboost algorithm, which was almost trivial to program, just iterated calls to CART, turn it into a world-class classiﬁcation algorithm that, by almost any standards, had accuracy as good as anything else out there, and better than almost everything else out
there."

I guess the interview was conducted at about the same time as Leo Breiman was working on the first Random Forests publications. Random Forests were recently shown (via) to perform splendidly in high-dimensional supervised learning settings.