{"id":1911,"date":"2023-09-10T20:22:07","date_gmt":"2023-09-10T20:22:07","guid":{"rendered":"https:\/\/mlinsightscentral.com\/?page_id=1911"},"modified":"2023-09-26T21:51:46","modified_gmt":"2023-09-26T21:51:46","slug":"decision-trees","status":"publish","type":"page","link":"https:\/\/mlinsightscentral.com\/index.php\/decision-trees\/","title":{"rendered":"Decision Trees"},"content":{"rendered":"\t\t<div data-elementor-type=\"wp-page\" data-elementor-id=\"1911\" class=\"elementor elementor-1911\">\n\t\t\t\t\t\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-af8cda2 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"af8cda2\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-05050e3\" data-id=\"05050e3\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-ad87aca elementor-widget elementor-widget-heading\" data-id=\"ad87aca\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<style>\/*! elementor - v3.13.3 - 28-05-2023 *\/\n.elementor-heading-title{padding:0;margin:0;line-height:1}.elementor-widget-heading .elementor-heading-title[class*=elementor-size-]>a{color:inherit;font-size:inherit;line-height:inherit}.elementor-widget-heading .elementor-heading-title.elementor-size-small{font-size:15px}.elementor-widget-heading .elementor-heading-title.elementor-size-medium{font-size:19px}.elementor-widget-heading .elementor-heading-title.elementor-size-large{font-size:29px}.elementor-widget-heading .elementor-heading-title.elementor-size-xl{font-size:39px}.elementor-widget-heading .elementor-heading-title.elementor-size-xxl{font-size:59px}<\/style><h4 class=\"elementor-heading-title elementor-size-default\"><span class=\"ez-toc-section\" id=\"Decision_Trees\"><\/span>Decision Trees<span class=\"ez-toc-section-end\"><\/span><\/h4>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-ffa983b elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"ffa983b\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-caea899\" data-id=\"caea899\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-92d4a2f elementor-widget elementor-widget-text-editor\" data-id=\"92d4a2f\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<style>\/*! elementor - v3.13.3 - 28-05-2023 *\/\n.elementor-widget-text-editor.elementor-drop-cap-view-stacked .elementor-drop-cap{background-color:#69727d;color:#fff}.elementor-widget-text-editor.elementor-drop-cap-view-framed .elementor-drop-cap{color:#69727d;border:3px solid;background-color:transparent}.elementor-widget-text-editor:not(.elementor-drop-cap-view-default) .elementor-drop-cap{margin-top:8px}.elementor-widget-text-editor:not(.elementor-drop-cap-view-default) .elementor-drop-cap-letter{width:1em;height:1em}.elementor-widget-text-editor .elementor-drop-cap{float:left;text-align:center;line-height:1;font-size:50px}.elementor-widget-text-editor .elementor-drop-cap-letter{display:inline-block}<\/style>\t\t\t\t<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_53 ez-toc-wrap-left counter-hierarchy ez-toc-counter ez-toc-light-blue ez-toc-container-direction\">\n<div class=\"ez-toc-title-container\">\n<p class=\"ez-toc-title \" >Table of Contents<\/p>\n<span class=\"ez-toc-title-toggle\"><a href=\"#\" class=\"ez-toc-pull-right ez-toc-btn ez-toc-btn-xs ez-toc-btn-default ez-toc-toggle\" aria-label=\"Toggle Table of Content\" role=\"button\"><label for=\"item-69e304c732abe\" ><span class=\"\"><span style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #999;color:#999\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #999;color:#999\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/label><input aria-label=\"Toggle\" aria-label=\"item-69e304c732abe\"  type=\"checkbox\" id=\"item-69e304c732abe\"><\/a><\/span><\/div>\n<nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-4'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/mlinsightscentral.com\/index.php\/decision-trees\/#Decision_Trees\" title=\"Decision Trees\">Decision Trees<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-4'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/mlinsightscentral.com\/index.php\/decision-trees\/#Building_a_decision_tree\" title=\"Building a decision tree\">Building a decision tree<\/a><ul class='ez-toc-list-level-5'><li class='ez-toc-heading-level-5'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/mlinsightscentral.com\/index.php\/decision-trees\/#Pruning_a_decision_tree\" title=\"Pruning a decision tree\">Pruning a decision tree<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-4'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/mlinsightscentral.com\/index.php\/decision-trees\/#Python_Implementation_%E2%80%93_Classification_problem\" title=\"Python Implementation &#8211; Classification problem\">Python Implementation &#8211; Classification problem<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-4'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/mlinsightscentral.com\/index.php\/decision-trees\/#Python_Implementation_%E2%80%93_Regression_problem\" title=\"Python Implementation &#8211; Regression problem\">Python Implementation &#8211; Regression problem<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-4'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/mlinsightscentral.com\/index.php\/decision-trees\/#Conclusion\" title=\"Conclusion\">Conclusion<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-4'><a class=\"ez-toc-link ez-toc-heading-7\" href=\"https:\/\/mlinsightscentral.com\/index.php\/decision-trees\/#References\" title=\"References\">References<\/a><\/li><\/ul><\/nav><\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-a26bd0c elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"a26bd0c\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-0973799\" data-id=\"0973799\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-3dc18da elementor-widget elementor-widget-text-editor\" data-id=\"3dc18da\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>Decision Trees are universal approximators that learn (nonlinear) patterns in the data based on a rule-based inference process that forms by recursive splitting homegenous data subgroups representing the same target class or sharing similar target values. Given a dataset of n features and m records, the decision tree iteratively selects the most informative feature and splits the datasets repeatedly into near-homogenous labelled subcategories until a stopping criterion is met.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-90064be elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"90064be\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-0860dd7\" data-id=\"0860dd7\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-b98d986 elementor-widget elementor-widget-image\" data-id=\"b98d986\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<style>\/*! elementor - v3.13.3 - 28-05-2023 *\/\n.elementor-widget-image{text-align:center}.elementor-widget-image a{display:inline-block}.elementor-widget-image a img[src$=\".svg\"]{width:48px}.elementor-widget-image img{vertical-align:middle;display:inline-block}<\/style>\t\t\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"799\" height=\"493\" src=\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/09\/decision_trees_.png\" class=\"attachment-large size-large wp-image-2144\" alt=\"decision trees\" srcset=\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/09\/decision_trees_.png 799w, https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/09\/decision_trees_-300x185.png 300w, https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/09\/decision_trees_-768x474.png 768w\" sizes=\"auto, (max-width: 799px) 100vw, 799px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-20883a6 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"20883a6\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-eb531e1\" data-id=\"eb531e1\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-1df1ee9 elementor-widget elementor-widget-text-editor\" data-id=\"1df1ee9\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>The top <strong>node<\/strong> is referred to as the <strong>root<\/strong> node and is the starting decision node. (i.e., Gender is Male or Female?). A <strong>branch<\/strong> is a subset of the dataset obtained as an outcome of a test. <strong>Internal nodes<\/strong> are decision nodes based on which subsequent branches are obtained. The <strong>depth<\/strong> of a node is the minimum number of decisions it takes to reach it from the root node. The <strong>leaf nodes<\/strong> are the end of the last branches on the tree which determine the output (class label or regression value).<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-2d28d3c elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"2d28d3c\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-1175280\" data-id=\"1175280\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-197eaac elementor-widget elementor-widget-image\" data-id=\"197eaac\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"601\" height=\"213\" src=\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/09\/decision_tree_example.png\" class=\"attachment-large size-large wp-image-1932\" alt=\"decision_tree_example\" srcset=\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/09\/decision_tree_example.png 601w, https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/09\/decision_tree_example-300x106.png 300w\" sizes=\"auto, (max-width: 601px) 100vw, 601px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Example of a Decision Tree for Classification [EMC, Data Science and Big Data Analytics]<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-5b531cf elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"5b531cf\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-57dd0b1\" data-id=\"57dd0b1\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-b9a1c7d elementor-widget elementor-widget-heading\" data-id=\"b9a1c7d\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h4 class=\"elementor-heading-title elementor-size-default\"><span class=\"ez-toc-section\" id=\"Building_a_decision_tree\"><\/span>Building a decision tree<span class=\"ez-toc-section-end\"><\/span><\/h4>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-093ef73 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"093ef73\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-c036f53\" data-id=\"c036f53\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-7f07da5 elementor-widget elementor-widget-text-editor\" data-id=\"7f07da5\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>Given a dataset of n features and m records, a rule-based graph is formed iteratively by recursive partitioning until the datasets is split in homogenous data groups representing the same target class in a classification problem or sharing close target values in a regression problem . From the root node (i.e. with all the m records), the most informative attribute is identified using some feature important score. The Gini index is the most commonly used feature importance score among others (entropy, information gain):<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-c47ab7f elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"c47ab7f\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-5ac6d32\" data-id=\"5ac6d32\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-2d54764 elementor-widget elementor-widget-text-editor\" data-id=\"2d54764\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>\\[Gini(f) = \\sum_{i=1}^{N_c}P(class=i|f)(1-P(class=i|f))\u00a0 = 1 &#8211; \\sum_{i=1}^{N_c}P(class=i|f)^2\\]<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-3ba5e99 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"3ba5e99\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-2ae8ad5\" data-id=\"2ae8ad5\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-0b64aa5 elementor-widget elementor-widget-text-editor\" data-id=\"0b64aa5\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>which measures the probability of a dataset to purely separated in its distinct classes upon a split on the given feature.\u00a0 It thus gauges the importance of a feature f in distinguishing classes in a given dataset. The lower the Gini index, the more likely the dataset classes are separable by the selected feature split, the higher the Gini index the less likely the dataset classes are separable by the selected feature split. Thus, in any given data subset, <strong>the feature with the lowest gini index is selected<\/strong>.\u00a0 In a practical implementation of a typical binary decision tree, the Gini index upon a given feature split, will be computed as a weighted sum of the gini indexes in each subset:<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-e7ef8d1 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"e7ef8d1\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-4aacdea\" data-id=\"4aacdea\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-6bc7d79 elementor-widget elementor-widget-text-editor\" data-id=\"6bc7d79\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>\\[Gini(f) = \\frac{n_{S_i}}{n_{S_i}+n_{S_j}}Gini(f_{S_i}) + \\frac{n_{S_j}}{n_{S_i}+n_{S_j}}Gini(f_{S_j}) \\]<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-a5ddbea elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"a5ddbea\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-de32525\" data-id=\"de32525\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-fa92fdc elementor-widget elementor-widget-text-editor\" data-id=\"fa92fdc\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>In the same vein, for a regression problem, the quality of the split is measured using some measure within subset target values closeness such as the mean square error or variance.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-161260b elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"161260b\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-167d45d\" data-id=\"167d45d\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-ccb0879 elementor-widget elementor-widget-text-editor\" data-id=\"ccb0879\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>\\[\\bar{y} = \\frac{1}{n_{S_i}}\\sum_{y\\in S_i}^{}y\\]<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-bab6816 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"bab6816\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-161576b\" data-id=\"161576b\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-3455d53 elementor-widget elementor-widget-text-editor\" data-id=\"3455d53\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>\\[MSE(S_i) = \\frac{1}{n_{S_i}}\\sum_{i=1}^{n_{S_i}}(\\bar{y}-y_i)^2\\]<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-33174ea elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"33174ea\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-54dedfc\" data-id=\"54dedfc\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-2bce6b9 elementor-widget elementor-widget-text-editor\" data-id=\"2bce6b9\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>Given an appropriate feature importance selection criterion, the decision tree is thus built as follows by recursive partitioning.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-ee5f592 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"ee5f592\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-3bdd0bf\" data-id=\"3bdd0bf\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-0d883b8 elementor-widget elementor-widget-elementor-syntax-highlighter\" data-id=\"0d883b8\" data-element_type=\"widget\" data-widget_type=\"elementor-syntax-highlighter.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<pre><code class='language-markup-templating'>Step 1: Given M attributes in a dataset N records and a target variable y\nStep 2: Rank features as per the chosen feature importance score\nStep 3: Split the dataset by the feature with the best importance score\nStep 4: Repeat Step 2 to each new subset until a stopping criterion is met <\/code><\/pre><script>\nif (!document.getElementById('syntaxed-prism')) {\n\tvar my_awesome_script = document.createElement('script');\n\tmy_awesome_script.setAttribute('src','https:\/\/mlinsightscentral.com\/wp-content\/plugins\/syntax-highlighter-for-elementor\/assets\/prism2.js');\n\tmy_awesome_script.setAttribute('id','syntaxed-prism');\n\tdocument.body.appendChild(my_awesome_script);\n} else {\n\twindow.Prism && Prism.highlightAll();\n}\n<\/script>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-4dfd9ea elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"4dfd9ea\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-b7e1d8b\" data-id=\"b7e1d8b\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-78f11d7 elementor-widget elementor-widget-text-editor\" data-id=\"78f11d7\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>Recursive partitioning stops when either the minimum number of samples per subset is reached, when a data subset is homogenous or near homogenous by some degree or when the node has a maximum preset depth.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-d0f7cdc elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"d0f7cdc\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-97a4304\" data-id=\"97a4304\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-0465e97 elementor-widget elementor-widget-heading\" data-id=\"0465e97\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h5 class=\"elementor-heading-title elementor-size-default\"><span class=\"ez-toc-section\" id=\"Pruning_a_decision_tree\"><\/span>Pruning a decision tree<span class=\"ez-toc-section-end\"><\/span><\/h5>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-c5b6729 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"c5b6729\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-d0e5b81\" data-id=\"d0e5b81\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-9059258 elementor-widget elementor-widget-text-editor\" data-id=\"9059258\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>The recursive nature of building decision trees renders the algorithm prone to overfitting. A decision tree can reach 100% fitting accuracy on the training set given that it can further split the data until a single data (i.e. guaranteed homogeneity) remains. However, this comes with the risk that the algorithm may lose its generalisation capability on unseen data. A pruning phase may post-process the decision tree, undermine some rules and allow some level of heterogeneity in the data subgroups to secure generalisation on unseen data.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-65bd420 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"65bd420\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-3c83d3a\" data-id=\"3c83d3a\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-aaef620 elementor-widget elementor-widget-heading\" data-id=\"aaef620\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h4 class=\"elementor-heading-title elementor-size-default\">Python Implementation - Classification problem<\/h4>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-b29502b elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"b29502b\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-7ffad15\" data-id=\"7ffad15\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-8853325 elementor-widget elementor-widget-text-editor\" data-id=\"8853325\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>\u00a0A sample dataset for diabetes classification [1] will be used to test the performance of decision trees as a binary classifier.\u00a0 Data were extracted from 2768 patient records with mixed data of healthy and non-healthy patients inclusive of nine attributes:<\/p><ol><li><strong>Id:<\/strong>\u00a0Unique identifier for each data entry.<\/li><li><strong>Pregnancies:<\/strong>\u00a0Number of times pregnant.<\/li><li><strong>Glucose:<\/strong>\u00a0Plasma glucose concentration over 2 hours in an oral glucose tolerance test.<\/li><li><strong>BloodPressure:<\/strong>\u00a0Diastolic blood pressure (mm Hg).<\/li><li><strong>SkinThickness:<\/strong>\u00a0Triceps skinfold thickness (mm).<\/li><li><strong>Insulin:<\/strong>\u00a02-Hour serum insulin (mu U\/ml).<\/li><li><strong>BMI:<\/strong>\u00a0Body mass index (weight in kg \/ height in m^2).<\/li><li><strong>DiabetesPedigreeFunction:<\/strong>\u00a0Diabetes pedigree function, a genetic score of diabetes.<\/li><li><strong>Age:<\/strong>\u00a0Age in years.<\/li><li><strong>Outcome:<\/strong>\u00a0Binary classification indicating the presence (1) or absence (0) of diabetes.<\/li><\/ol>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-e2559e5 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"e2559e5\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-69c3bae\" data-id=\"69c3bae\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-38590c6 elementor-widget elementor-widget-image\" data-id=\"38590c6\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"825\" height=\"179\" src=\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/09\/diabetes_datasets.png\" class=\"attachment-large size-large wp-image-1733\" alt=\"diabetes datasets\" srcset=\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/09\/diabetes_datasets.png 825w, https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/09\/diabetes_datasets-300x65.png 300w, https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/09\/diabetes_datasets-768x167.png 768w\" sizes=\"auto, (max-width: 825px) 100vw, 825px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-b729a71 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"b729a71\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-b5a981c\" data-id=\"b5a981c\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-369c45f elementor-widget elementor-widget-elementor-syntax-highlighter\" data-id=\"369c45f\" data-element_type=\"widget\" data-widget_type=\"elementor-syntax-highlighter.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<pre><code class='language-python'>\n#-- Load the Dataset\nimport pandas as pd\nimport warnings\nwarnings.filterwarnings('ignore') #ignore warnings\ndf = pd.read_csv('https:\/\/raw.githubusercontent.com\/mlinsights\/freemium\/main\/datasets\/classification\/diabetes\/Healthcare-Diabetes.csv')\ndf.head()\n\n\n#-- Summarise the Dataset\ndf.info()\n\n#-- Extract Features and the target variable\nX = df.iloc[:,1:9]\ny = df[['Outcome']]\n\n#-- Split the Dataset in Training and Test Set\nfrom sklearn.model_selection import train_test_split\nX_train, X_test, y_train, y_test = train_test_split(X,y,test_size=0.2,random_state=1234)#fix the random seed (to reproduce the results)\n\n\n#-- Training the model\nfrom sklearn.tree import DecisionTreeClassifier\nimport matplotlib.pyplot as plt\nfrom sklearn import tree\n\ndt_classifier = DecisionTreeClassifier(criterion=&quot;gini&quot;,max_depth=2,min_samples_leaf=1)#Setting the decision tree - settings\ndt_classifier.fit(X_train,y_train)#train the classifier\n\ntree.plot_tree(dt_classifier)\nplt.show()\n\n\n# -- Benchmark the model performance\nfrom sklearn.metrics import confusion_matrix\nfrom sklearn.metrics import classification_report\nimport seaborn as sns\nimport numpy as np\n\ny_pred_train = dt_classifier.predict(X_train)\ny_pred_test = dt_classifier.predict(X_test)\n\n# Create confusion matrix for the training set\ncm_train = confusion_matrix(y_train, y_pred_train)\n\n# Create heatmap - Test set\nplt.figure(figsize=(8, 6))\nsns.set(font_scale=1.2)\nsns.heatmap(cm_train, annot=True, fmt=&quot;d&quot;, cmap=&quot;Blues&quot;, cbar=False, square=True)\nplt.xlabel('Predicted')\nplt.ylabel('Actual')\nplt.title('Confusion Matrix - Train')\nplt.show()\n\ntrain_report = classification_report(y_train, y_pred_train, target_names=['has diabetes','no diabetes'])\nprint(&quot;Classification Report:\\n&quot;, train_report)\n\n# Create confusion matrix test set\ncm_test = confusion_matrix(y_test, y_pred_test)\n# Create heatmap - Test set\nplt.figure(figsize=(8, 6))\nsns.set(font_scale=1.2)\nsns.heatmap(cm_test, annot=True, fmt=&quot;d&quot;, cmap=&quot;Blues&quot;, cbar=False, square=True)\nplt.xlabel('Predicted')\nplt.ylabel('Actual')\nplt.title('Confusion Matrix - Test')\nplt.show()\n\n#tn, fp, fn, tp = cm_test\nclass_acc = (cm_test[0][0]+cm_test[1][1])\/(cm_test[0][0]+cm_test[0][1]+cm_test[1][0]+cm_test[1][1])\ntest_report = classification_report(y_test, y_pred_test, target_names=['has diabetes','no diabetes'])\nprint(&quot;Classification Report:\\n&quot;, test_report)\n\n <\/code><\/pre><script>\nif (!document.getElementById('syntaxed-prism')) {\n\tvar my_awesome_script = document.createElement('script');\n\tmy_awesome_script.setAttribute('src','https:\/\/mlinsightscentral.com\/wp-content\/plugins\/syntax-highlighter-for-elementor\/assets\/prism2.js');\n\tmy_awesome_script.setAttribute('id','syntaxed-prism');\n\tdocument.body.appendChild(my_awesome_script);\n} else {\n\twindow.Prism && Prism.highlightAll();\n}\n<\/script>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-99cbb15 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"99cbb15\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-377bc51\" data-id=\"377bc51\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-dd74132 elementor-widget elementor-widget-text-editor\" data-id=\"dd74132\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>The dataset of records is split into training and test (80\/20) to test the predictive performance of the decision tree in distinguishing between the presence or absence of diabetes. The model will thus be trained on the training set alone but benchmarked on the training and test set. The test performance being the most important has it is an indicator of the model predictive power on unseen data. Having fixed the maximum tree depth to 2, the decision tree rules can be visualised clearly. For performance sake, the maximum depth can be set to None to alow maximum insight discovery. With the current settings, the model classification performances are recorded and presented in the figure below.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-76d9cfd elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"76d9cfd\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-6ad791c\" data-id=\"6ad791c\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-99654bf elementor-widget elementor-widget-image\" data-id=\"99654bf\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"515\" height=\"389\" src=\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/09\/decision_tree_result.png\" class=\"attachment-large size-large wp-image-2217\" alt=\"decision_tree_result\" srcset=\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/09\/decision_tree_result.png 515w, https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/09\/decision_tree_result-300x227.png 300w\" sizes=\"auto, (max-width: 515px) 100vw, 515px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-0d739d2 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"0d739d2\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-50 elementor-top-column elementor-element elementor-element-da963ab\" data-id=\"da963ab\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-c8ac71b elementor-widget elementor-widget-image\" data-id=\"c8ac71b\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"537\" height=\"561\" src=\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/09\/cm_train_cart.png\" class=\"attachment-large size-large wp-image-2219\" alt=\"cm_train_cart\" srcset=\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/09\/cm_train_cart.png 537w, https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/09\/cm_train_cart-287x300.png 287w\" sizes=\"auto, (max-width: 537px) 100vw, 537px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t<div class=\"elementor-column elementor-col-50 elementor-top-column elementor-element elementor-element-00d05a9\" data-id=\"00d05a9\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-fb1fbeb elementor-widget elementor-widget-image\" data-id=\"fb1fbeb\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"537\" height=\"561\" src=\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/09\/cm_test_cart.png\" class=\"attachment-large size-large wp-image-2220\" alt=\"cm_test_cart\" srcset=\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/09\/cm_test_cart.png 537w, https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/09\/cm_test_cart-287x300.png 287w\" sizes=\"auto, (max-width: 537px) 100vw, 537px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-f2bdfe4 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"f2bdfe4\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-50 elementor-top-column elementor-element elementor-element-50f438e\" data-id=\"50f438e\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-41d2177 elementor-widget elementor-widget-image\" data-id=\"41d2177\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"662\" height=\"246\" src=\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/09\/classification_report_cart_train.png\" class=\"attachment-large size-large wp-image-2221\" alt=\"classification_report_cart_train\" srcset=\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/09\/classification_report_cart_train.png 662w, https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/09\/classification_report_cart_train-300x111.png 300w\" sizes=\"auto, (max-width: 662px) 100vw, 662px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t<div class=\"elementor-column elementor-col-50 elementor-top-column elementor-element elementor-element-179a0c2\" data-id=\"179a0c2\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-301a0b5 elementor-widget elementor-widget-image\" data-id=\"301a0b5\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"663\" height=\"244\" src=\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/09\/classification_report_cart_test.png\" class=\"attachment-large size-large wp-image-2222\" alt=\"classification_report_cart_test\" srcset=\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/09\/classification_report_cart_test.png 663w, https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/09\/classification_report_cart_test-300x110.png 300w\" sizes=\"auto, (max-width: 663px) 100vw, 663px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-eee38b3 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"eee38b3\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-0ab1118\" data-id=\"0ab1118\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-ad23b14 elementor-widget elementor-widget-text-editor\" data-id=\"ad23b14\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>After lifting the restriction of the search of the decision tree (i.e. max-depth = None), a fully blown decision tree classifier is obtained with excellent performance illustrating the outstanding predictive capabilities of the algorithm.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-ae03ea6 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"ae03ea6\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-9e62576\" data-id=\"9e62576\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-6c944d2 elementor-widget elementor-widget-elementor-syntax-highlighter\" data-id=\"6c944d2\" data-element_type=\"widget\" data-widget_type=\"elementor-syntax-highlighter.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<pre><code class='language-python'>dt_classifier = DecisionTreeClassifier(criterion=&quot;gini&quot;,max_depth=None,min_samples_leaf=1)#Setting the decision tree - settings\ndt_classifier.fit(X_train,y_train)#train the classifier\n <\/code><\/pre><script>\nif (!document.getElementById('syntaxed-prism')) {\n\tvar my_awesome_script = document.createElement('script');\n\tmy_awesome_script.setAttribute('src','https:\/\/mlinsightscentral.com\/wp-content\/plugins\/syntax-highlighter-for-elementor\/assets\/prism2.js');\n\tmy_awesome_script.setAttribute('id','syntaxed-prism');\n\tdocument.body.appendChild(my_awesome_script);\n} else {\n\twindow.Prism && Prism.highlightAll();\n}\n<\/script>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-c75b923 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"c75b923\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-50 elementor-top-column elementor-element elementor-element-c103645\" data-id=\"c103645\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-bf2deaa elementor-widget elementor-widget-image\" data-id=\"bf2deaa\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"649\" height=\"248\" src=\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/09\/classification_report_cart_test_no_max_depth.png\" class=\"attachment-large size-large wp-image-2228\" alt=\"\" srcset=\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/09\/classification_report_cart_test_no_max_depth.png 649w, https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/09\/classification_report_cart_test_no_max_depth-300x115.png 300w\" sizes=\"auto, (max-width: 649px) 100vw, 649px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t<div class=\"elementor-column elementor-col-50 elementor-top-column elementor-element elementor-element-1760346\" data-id=\"1760346\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-bc63ca7 elementor-widget elementor-widget-image\" data-id=\"bc63ca7\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"661\" height=\"243\" src=\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/09\/classification_report_cart_train_no_max_depth.png\" class=\"attachment-large size-large wp-image-2227\" alt=\"\" srcset=\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/09\/classification_report_cart_train_no_max_depth.png 661w, https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/09\/classification_report_cart_train_no_max_depth-300x110.png 300w\" sizes=\"auto, (max-width: 661px) 100vw, 661px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-4e18402 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"4e18402\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-8b293dc\" data-id=\"8b293dc\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-3f07780 elementor-widget elementor-widget-heading\" data-id=\"3f07780\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h4 class=\"elementor-heading-title elementor-size-default\">Python Implementation - Regression problem<\/h4>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-db41685 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"db41685\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-0147ede\" data-id=\"0147ede\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-fa72ab8 elementor-widget elementor-widget-text-editor\" data-id=\"fa72ab8\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>To illustrate the performance of the decision for regression, a dataset of 414 records of house pricing data in a given region is used. The dataset consists of six attributes: the house&#8217;s distance to the nearest train station, the number of convenience stores surrounding the house as well as the latitude and longitude location of the house. The target variable is the house sold price per square meter. A regression model must thus be built to learn the inherent house pricing strategy used in the area.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-a59aa70 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"a59aa70\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-f65dc96\" data-id=\"f65dc96\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-85641f2 elementor-widget elementor-widget-image\" data-id=\"85641f2\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" src=\"https:\/\/raw.githubusercontent.com\/mlinsights\/freemium\/main\/datasets\/regression-analysis\/real-estate-house-pricing\/house_pricing_data.png\" title=\"\" alt=\"\" loading=\"lazy\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-762d36f elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"762d36f\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-eb9d6ca\" data-id=\"eb9d6ca\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-6bf8e2d elementor-widget elementor-widget-elementor-syntax-highlighter\" data-id=\"6bf8e2d\" data-element_type=\"widget\" data-widget_type=\"elementor-syntax-highlighter.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<pre><code class='language-python'>#--load the datasets\nimport pandas as pd\nimport warnings\nwarnings.filterwarnings('ignore') #ignore warnings\ndf = pd.read_csv('https:\/\/raw.githubusercontent.com\/mlinsights\/freemium\/main\/datasets\/regression-analysis\/real-estate-house-pricing\/real_estate_data.csv')\ndf = df.iloc[:,1:] #retrieve house pricing data\ndf.head()\n\n#get the datasets description\ndf.info()\n\n#--extract features and the target variable\n\nX = df.iloc[:,0:6]#get features\ny = df.iloc[:,[6]]#get target variable\nX.head()\n\n#--Split the the training set from the test set\nfrom sklearn.model_selection import train_test_split\n\nX_train, X_test, y_train, y_test = train_test_split(X,y,test_size=0.2,random_state=1234)#fix the random seed (to reproduce the results)\n\n#--Build the Decision Tree Regression model\nfrom sklearn.tree import DecisionTreeRegressor\nimport matplotlib.pyplot as plt\nfrom sklearn import tree\n\ndt_regressor = DecisionTreeRegressor(criterion=&quot;squared_error&quot;,max_depth=None,min_samples_leaf=1)#Setting the decision tree - settings\ndt_regressor.fit(X_train,y_train)#train the classifier\n\ntree.plot_tree(dt_regressor) #display the model rules\nplt.show()\n\n\n#--Compute the Coefficient of determination and visualise the goodness of fit for both the training and test setfrom sklearn.metrics import r2_score\n\ny_pred_train = dt_regressor.predict(X_train)#get model prediction on training set\ny_pred_test = dt_regressor.predict(X_test)#get model prediction on test\n\nr2_score_train = r2_score(y_train,y_pred_train)\nr2_score_test = r2_score(y_test,y_pred_test)\n\nplt.figure()\nplt.scatter(y_train, y_pred_train, color=&quot;b&quot;)\nplt.xlabel('observed data: y_d')\nplt.ylabel('model prediction: y_p')\nplt.title('Training set - Goodness of fit -  R2: %.2f'%r2_score_train)\nplt.show() \n\nplt.figure()\nplt.scatter(y_test, y_pred_test, color=&quot;b&quot;)\nplt.xlabel('observed data: y_d')\nplt.ylabel('model prediction: y_p')\nplt.title('Test set - Goodness of fit -  R2: %.2f'%r2_score_train)\nplt.show() \n\n <\/code><\/pre><script>\nif (!document.getElementById('syntaxed-prism')) {\n\tvar my_awesome_script = document.createElement('script');\n\tmy_awesome_script.setAttribute('src','https:\/\/mlinsightscentral.com\/wp-content\/plugins\/syntax-highlighter-for-elementor\/assets\/prism2.js');\n\tmy_awesome_script.setAttribute('id','syntaxed-prism');\n\tdocument.body.appendChild(my_awesome_script);\n} else {\n\twindow.Prism && Prism.highlightAll();\n}\n<\/script>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-a1548bb elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"a1548bb\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-50c6be1\" data-id=\"50c6be1\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-69bbbf1 elementor-widget elementor-widget-text-editor\" data-id=\"69bbbf1\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>The mean square error has been to rank features, the decision trees have been trained with limits of node depth and a minimum leaf node size of 1, yielding the goodness of fit recorded in the below graphs.\u00a0 As per theory, the training set performance have tendencies to overfit yield a near perfect which may impact on its generalisation prowesses.\u00a0 The performance on the test set may be due to either overfitting or the complexity of the dataset.\u00a0 To further improve the model performance, a hyperparameter tuning may be use to tune the decision tree parameters to its best performing configuration (i.e. max depth, min leaf node size, etc.)<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-8737a92 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"8737a92\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-50 elementor-top-column elementor-element elementor-element-459bed5\" data-id=\"459bed5\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-a49f277 elementor-widget elementor-widget-image\" data-id=\"a49f277\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"580\" height=\"453\" src=\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/09\/decision-tree-regression-result.png\" class=\"attachment-large size-large wp-image-2251\" alt=\"\" srcset=\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/09\/decision-tree-regression-result.png 580w, https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/09\/decision-tree-regression-result-300x234.png 300w\" sizes=\"auto, (max-width: 580px) 100vw, 580px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t<div class=\"elementor-column elementor-col-50 elementor-top-column elementor-element elementor-element-70c66c7\" data-id=\"70c66c7\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-f1258eb elementor-widget elementor-widget-image\" data-id=\"f1258eb\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"581\" height=\"453\" src=\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/09\/decision-tree-regression-result-testv2.png\" class=\"attachment-large size-large wp-image-2259\" alt=\"\" srcset=\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/09\/decision-tree-regression-result-testv2.png 581w, https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/09\/decision-tree-regression-result-testv2-300x234.png 300w\" sizes=\"auto, (max-width: 581px) 100vw, 581px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-0416ff9 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"0416ff9\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-760e677\" data-id=\"760e677\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-f395300 elementor-widget elementor-widget-text-editor\" data-id=\"f395300\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>Illustratively, reducing the maximum depth of node split to 10 has balanced the tradeoff between training and generalisation further boosting the model performance on the test set.\u00a0 Feature selection may also be considered to further boost the model performance. Nevertheless, as per the No Free Lunch Theorem, other nonlinear models may also be considered given that different models perform differently on different datasets.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-59005ca elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"59005ca\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-35c3993\" data-id=\"35c3993\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-ec97a4a elementor-widget elementor-widget-elementor-syntax-highlighter\" data-id=\"ec97a4a\" data-element_type=\"widget\" data-widget_type=\"elementor-syntax-highlighter.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<pre><code class='language-python'>dt_regressor = DecisionTreeRegressor(criterion=&quot;squared_error&quot;,max_depth=10,min_samples_leaf=1)#Setting the decision tree - settings\r\ndt_regressor.fit(X_train,y_train)#train the regression model\r\n <\/code><\/pre><script>\nif (!document.getElementById('syntaxed-prism')) {\n\tvar my_awesome_script = document.createElement('script');\n\tmy_awesome_script.setAttribute('src','https:\/\/mlinsightscentral.com\/wp-content\/plugins\/syntax-highlighter-for-elementor\/assets\/prism2.js');\n\tmy_awesome_script.setAttribute('id','syntaxed-prism');\n\tdocument.body.appendChild(my_awesome_script);\n} else {\n\twindow.Prism && Prism.highlightAll();\n}\n<\/script>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-3aa8fc0 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"3aa8fc0\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-50 elementor-top-column elementor-element elementor-element-9e50a7e\" data-id=\"9e50a7e\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-4fa10d5 elementor-widget elementor-widget-image\" data-id=\"4fa10d5\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"580\" height=\"453\" src=\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/09\/decision-tree-regression-result-train2.png\" class=\"attachment-large size-large wp-image-2266\" alt=\"\" srcset=\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/09\/decision-tree-regression-result-train2.png 580w, https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/09\/decision-tree-regression-result-train2-300x234.png 300w\" sizes=\"auto, (max-width: 580px) 100vw, 580px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t<div class=\"elementor-column elementor-col-50 elementor-top-column elementor-element elementor-element-7d04979\" data-id=\"7d04979\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-28ea620 elementor-widget elementor-widget-image\" data-id=\"28ea620\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"581\" height=\"453\" src=\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/09\/decision-tree-regression-result-test2.png\" class=\"attachment-large size-large wp-image-2267\" alt=\"\" srcset=\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/09\/decision-tree-regression-result-test2.png 581w, https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/09\/decision-tree-regression-result-test2-300x234.png 300w\" sizes=\"auto, (max-width: 581px) 100vw, 581px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-79e4b24 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"79e4b24\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-9f26857\" data-id=\"9f26857\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-dcf9a2c elementor-widget elementor-widget-heading\" data-id=\"dcf9a2c\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h4 class=\"elementor-heading-title elementor-size-default\"><span class=\"ez-toc-section\" id=\"Conclusion\"><\/span>Conclusion<span class=\"ez-toc-section-end\"><\/span><\/h4>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-9b1e9a3 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"9b1e9a3\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-1dfa2d0\" data-id=\"1dfa2d0\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-cb93202 elementor-widget elementor-widget-text-editor\" data-id=\"cb93202\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>Decision Trees are powerful universal approximators that can be used for both regression and classification. The algorithm generates a rule-based tree of the inherent pattern in the datasets by recursively partitioning the dataset on the informative features per data subset until it separates the original data into homogenous or near-homogeneous groups.\u00a0 It is a nonparametric model that can identify very complex linearities in datasets. Nevertheless, extra care should be taken to control the model training process (i.e. node splitting) and avoid overfitting that may impede on its accuracy on unseen data. Hyperparameter tuning or posterior tree pruning phase may be used to achieve such a fit.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-8e8a77b elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"8e8a77b\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-b155666\" data-id=\"b155666\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-916fdbd elementor-widget elementor-widget-heading\" data-id=\"916fdbd\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<div class=\"elementor-heading-title elementor-size-default\"><b>Author: Yves Matanga, PhD<\/b><\/div>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-8606044 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"8606044\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-fb66898\" data-id=\"fb66898\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-3069930 elementor-widget elementor-widget-heading\" data-id=\"3069930\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h4 class=\"elementor-heading-title elementor-size-default\"><span class=\"ez-toc-section\" id=\"References\"><\/span>References<span class=\"ez-toc-section-end\"><\/span><\/h4>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-0ecf668 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"0ecf668\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-795a5be\" data-id=\"795a5be\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-5f75b3a elementor-widget elementor-widget-text-editor\" data-id=\"5f75b3a\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>[1] Healthcare Diabetes Dataset, Kaggle (<a href=\"https:\/\/www.kaggle.com\/datasets\/nanditapore\/healthcare-diabetes\">https:\/\/www.kaggle.com\/datasets\/nanditapore\/healthcare-diabetes<\/a>)<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"<p>Decision Trees Decision Trees are universal approximators that learn (nonlinear) patterns in the data based on a rule-based inference process that forms by recursive splitting homegenous data subgroups representing the same target class or sharing similar target values. Given a dataset of n features and m records, the decision tree iteratively selects the most informative &hellip;<\/p>\n<p class=\"read-more\"> <a class=\"\" href=\"https:\/\/mlinsightscentral.com\/index.php\/decision-trees\/\"> <span class=\"screen-reader-text\">Decision Trees<\/span> Read More &raquo;<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"open","ping_status":"closed","template":"","meta":{"_uag_custom_page_level_css":"","site-sidebar-layout":"no-sidebar","site-content-layout":"page-builder","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"disabled","ast-breadcrumbs-content":"","ast-featured-img":"disabled","footer-sml-layout":"","theme-transparent-header-meta":"","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","footnotes":""},"wf_page_folders":[26],"class_list":["post-1911","page","type-page","status-publish","hentry"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v20.11 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Decision Trees - MLInsightsCentral<\/title>\n<meta name=\"description\" content=\"Decision Trees are rule based universal approximators that learn patterns in the data based on a feature informed divide conquer approach.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/mlinsightscentral.com\/index.php\/decision-trees\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Decision Trees - MLInsightsCentral\" \/>\n<meta property=\"og:description\" content=\"Decision Trees are rule based universal approximators that learn patterns in the data based on a feature informed divide conquer approach.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/mlinsightscentral.com\/index.php\/decision-trees\/\" \/>\n<meta property=\"og:site_name\" content=\"MLInsightsCentral\" \/>\n<meta property=\"article:modified_time\" content=\"2023-09-26T21:51:46+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/09\/decision_trees_.png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"9 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/mlinsightscentral.com\/index.php\/decision-trees\/\",\"url\":\"https:\/\/mlinsightscentral.com\/index.php\/decision-trees\/\",\"name\":\"Decision Trees - MLInsightsCentral\",\"isPartOf\":{\"@id\":\"https:\/\/mlinsightscentral.com\/#website\"},\"datePublished\":\"2023-09-10T20:22:07+00:00\",\"dateModified\":\"2023-09-26T21:51:46+00:00\",\"description\":\"Decision Trees are rule based universal approximators that learn patterns in the data based on a feature informed divide conquer approach.\",\"breadcrumb\":{\"@id\":\"https:\/\/mlinsightscentral.com\/index.php\/decision-trees\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/mlinsightscentral.com\/index.php\/decision-trees\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/mlinsightscentral.com\/index.php\/decision-trees\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/mlinsightscentral.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Decision Trees\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/mlinsightscentral.com\/#website\",\"url\":\"https:\/\/mlinsightscentral.com\/\",\"name\":\"MLInsightsCentral\",\"description\":\"Learn Machine Learning and AI for engineers, data scientists and AI practionners.\",\"publisher\":{\"@id\":\"https:\/\/mlinsightscentral.com\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/mlinsightscentral.com\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/mlinsightscentral.com\/#organization\",\"name\":\"MLInsightsCentral\",\"url\":\"https:\/\/mlinsightscentral.com\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/mlinsightscentral.com\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/06\/cropped-1290538dccf74accb0ae585ff4e8586c-1.png\",\"contentUrl\":\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/06\/cropped-1290538dccf74accb0ae585ff4e8586c-1.png\",\"width\":200,\"height\":110,\"caption\":\"MLInsightsCentral\"},\"image\":{\"@id\":\"https:\/\/mlinsightscentral.com\/#\/schema\/logo\/image\/\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Decision Trees - MLInsightsCentral","description":"Decision Trees are rule based universal approximators that learn patterns in the data based on a feature informed divide conquer approach.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/mlinsightscentral.com\/index.php\/decision-trees\/","og_locale":"en_US","og_type":"article","og_title":"Decision Trees - MLInsightsCentral","og_description":"Decision Trees are rule based universal approximators that learn patterns in the data based on a feature informed divide conquer approach.","og_url":"https:\/\/mlinsightscentral.com\/index.php\/decision-trees\/","og_site_name":"MLInsightsCentral","article_modified_time":"2023-09-26T21:51:46+00:00","og_image":[{"url":"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/09\/decision_trees_.png"}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"9 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/mlinsightscentral.com\/index.php\/decision-trees\/","url":"https:\/\/mlinsightscentral.com\/index.php\/decision-trees\/","name":"Decision Trees - MLInsightsCentral","isPartOf":{"@id":"https:\/\/mlinsightscentral.com\/#website"},"datePublished":"2023-09-10T20:22:07+00:00","dateModified":"2023-09-26T21:51:46+00:00","description":"Decision Trees are rule based universal approximators that learn patterns in the data based on a feature informed divide conquer approach.","breadcrumb":{"@id":"https:\/\/mlinsightscentral.com\/index.php\/decision-trees\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/mlinsightscentral.com\/index.php\/decision-trees\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/mlinsightscentral.com\/index.php\/decision-trees\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/mlinsightscentral.com\/"},{"@type":"ListItem","position":2,"name":"Decision Trees"}]},{"@type":"WebSite","@id":"https:\/\/mlinsightscentral.com\/#website","url":"https:\/\/mlinsightscentral.com\/","name":"MLInsightsCentral","description":"Learn Machine Learning and AI for engineers, data scientists and AI practionners.","publisher":{"@id":"https:\/\/mlinsightscentral.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/mlinsightscentral.com\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/mlinsightscentral.com\/#organization","name":"MLInsightsCentral","url":"https:\/\/mlinsightscentral.com\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/mlinsightscentral.com\/#\/schema\/logo\/image\/","url":"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/06\/cropped-1290538dccf74accb0ae585ff4e8586c-1.png","contentUrl":"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/06\/cropped-1290538dccf74accb0ae585ff4e8586c-1.png","width":200,"height":110,"caption":"MLInsightsCentral"},"image":{"@id":"https:\/\/mlinsightscentral.com\/#\/schema\/logo\/image\/"}}]}},"uagb_featured_image_src":{"full":false,"thumbnail":false,"medium":false,"medium_large":false,"large":false,"1536x1536":false,"2048x2048":false},"uagb_author_info":{"display_name":"admin","author_link":"https:\/\/mlinsightscentral.com\/index.php\/author\/yvesm\/"},"uagb_comment_info":12,"uagb_excerpt":"Decision Trees Decision Trees are universal approximators that learn (nonlinear) patterns in the data based on a rule-based inference process that forms by recursive splitting homegenous data subgroups representing the same target class or sharing similar target values. Given a dataset of n features and m records, the decision tree iteratively selects the most informative&hellip;","_links":{"self":[{"href":"https:\/\/mlinsightscentral.com\/index.php\/wp-json\/wp\/v2\/pages\/1911","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mlinsightscentral.com\/index.php\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mlinsightscentral.com\/index.php\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mlinsightscentral.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/mlinsightscentral.com\/index.php\/wp-json\/wp\/v2\/comments?post=1911"}],"version-history":[{"count":198,"href":"https:\/\/mlinsightscentral.com\/index.php\/wp-json\/wp\/v2\/pages\/1911\/revisions"}],"predecessor-version":[{"id":2370,"href":"https:\/\/mlinsightscentral.com\/index.php\/wp-json\/wp\/v2\/pages\/1911\/revisions\/2370"}],"wp:attachment":[{"href":"https:\/\/mlinsightscentral.com\/index.php\/wp-json\/wp\/v2\/media?parent=1911"}],"wp:term":[{"taxonomy":"wf_page_folders","embeddable":true,"href":"https:\/\/mlinsightscentral.com\/index.php\/wp-json\/wp\/v2\/wf_page_folders?post=1911"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}