DAY 18-100 DAYS MLCODE: DECISION TREES – Part 3

In the previous blog, we learned about the Decision trees capability to perform the classification task, in today’s blog we are going to perform a simple regression task. A decision tree is also capable of performing the regression task.

SciKit-Learn DecisionTreeRegressor class help us to perform the regression task.

we are using the same data set as previous blog and the only change in code is using the DecisionTreeRegressor class of the SciKit-Learn

from sklearn.tree import DecisionTreeRegressor
tree_clf = DecisionTreeRegressor()
tree_clf.fit(X,y)

Output: DecisionTreeRegressor(criterion=’mse’, max_depth=None, max_features=None, max_leaf_nodes=None, min_impurity_decrease=0.0, min_impurity_split=None, min_samples_leaf=1, min_samples_split=2, min_weight_fraction_leaf=0.0, presort=False, random_state=None, splitter=’best’)

Now export the tree and convert into png .

from sklearn.tree import export_graphviz

export_graphviz(
        tree_clf,
        out_file="wine_tree.dot",
        feature_names= wine_data.feature_names,
        class_names= wine_data.target_names,
        rounded=True,
        filled=True
    )

Let’s see review the decision tree.

If you remember from the last blog,  each node has gini impurity with the sample size and class type. But in this case, it looks different here we have the regression value and samples size. And instead of gini impurity, we have MSE .

The CART algorithm works the same way as the Classification but instead of minimizing the impurity, it tries to minimize the MSE .

In conclusion, a Decision trees is a powerful algorithm and can be used to perform both regression and classification task. You can find the code related to today’s here.