Pruning Classification Trees

Pruning intuition

Trees in the real world can grow too large. For example, a tree in someone’s yard might grow over the fence into a neighbor’s property and an expert might be called to prune the tree back (i.e., remove branches). Pruning is the process of removing that which is not needed.

In the case of CART, decision trees that have grown too large are prone to overfitting. CART automatically implements a pruning process after a decision tree has been trained to reduce the decision tree’s complexity.

Take, for example, the following hypothetical decision tree that has grown to full size based on the training data and the minsplit hyperparameter: