{"id":1468,"date":"2023-08-28T20:15:06","date_gmt":"2023-08-28T20:15:06","guid":{"rendered":"https:\/\/mlinsightscentral.com\/?page_id=1468"},"modified":"2023-09-18T15:11:08","modified_gmt":"2023-09-18T15:11:08","slug":"gradient-descent-algorithm","status":"publish","type":"page","link":"https:\/\/mlinsightscentral.com\/index.php\/gradient-descent-algorithm\/","title":{"rendered":"Gradient Descent Algorithm"},"content":{"rendered":"\t\t<div data-elementor-type=\"wp-page\" data-elementor-id=\"1468\" class=\"elementor elementor-1468\">\n\t\t\t\t\t\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-eb6f5cf elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"eb6f5cf\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-3e0f828\" data-id=\"3e0f828\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap\">\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-ffc13e8 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"ffc13e8\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-28178c2\" data-id=\"28178c2\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-feb9895 elementor-widget elementor-widget-heading\" data-id=\"feb9895\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<style>\/*! elementor - v3.13.3 - 28-05-2023 *\/\n.elementor-heading-title{padding:0;margin:0;line-height:1}.elementor-widget-heading .elementor-heading-title[class*=elementor-size-]>a{color:inherit;font-size:inherit;line-height:inherit}.elementor-widget-heading .elementor-heading-title.elementor-size-small{font-size:15px}.elementor-widget-heading .elementor-heading-title.elementor-size-medium{font-size:19px}.elementor-widget-heading .elementor-heading-title.elementor-size-large{font-size:29px}.elementor-widget-heading .elementor-heading-title.elementor-size-xl{font-size:39px}.elementor-widget-heading .elementor-heading-title.elementor-size-xxl{font-size:59px}<\/style><h4 class=\"elementor-heading-title elementor-size-default\"><span class=\"ez-toc-section\" id=\"Gradient_Descent_Algorithm\"><\/span>Gradient Descent Algorithm<span class=\"ez-toc-section-end\"><\/span><\/h4>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-b0be92c elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"b0be92c\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-f29aac5\" data-id=\"f29aac5\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-fb91605 elementor-widget elementor-widget-text-editor\" data-id=\"fb91605\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<style>\/*! elementor - v3.13.3 - 28-05-2023 *\/\n.elementor-widget-text-editor.elementor-drop-cap-view-stacked .elementor-drop-cap{background-color:#69727d;color:#fff}.elementor-widget-text-editor.elementor-drop-cap-view-framed .elementor-drop-cap{color:#69727d;border:3px solid;background-color:transparent}.elementor-widget-text-editor:not(.elementor-drop-cap-view-default) .elementor-drop-cap{margin-top:8px}.elementor-widget-text-editor:not(.elementor-drop-cap-view-default) .elementor-drop-cap-letter{width:1em;height:1em}.elementor-widget-text-editor .elementor-drop-cap{float:left;text-align:center;line-height:1;font-size:50px}.elementor-widget-text-editor .elementor-drop-cap-letter{display:inline-block}<\/style>\t\t\t\t<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_53 ez-toc-wrap-left counter-hierarchy ez-toc-counter ez-toc-light-blue ez-toc-container-direction\">\n<div class=\"ez-toc-title-container\">\n<p class=\"ez-toc-title \" >Table of Contents<\/p>\n<span class=\"ez-toc-title-toggle\"><a href=\"#\" class=\"ez-toc-pull-right ez-toc-btn ez-toc-btn-xs ez-toc-btn-default ez-toc-toggle\" aria-label=\"Toggle Table of Content\" role=\"button\"><label for=\"item-69e304f05d719\" ><span class=\"\"><span style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #999;color:#999\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #999;color:#999\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/label><input aria-label=\"Toggle\" aria-label=\"item-69e304f05d719\"  type=\"checkbox\" id=\"item-69e304f05d719\"><\/a><\/span><\/div>\n<nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-4'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/mlinsightscentral.com\/index.php\/gradient-descent-algorithm\/#Gradient_Descent_Algorithm\" title=\"Gradient Descent Algorithm\">Gradient Descent Algorithm<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-4'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/mlinsightscentral.com\/index.php\/gradient-descent-algorithm\/#Gradient_descent_%E2%80%93_iterative_search_formula\" title=\"Gradient descent &#8211; iterative search formula\">Gradient descent &#8211; iterative search formula<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-4'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/mlinsightscentral.com\/index.php\/gradient-descent-algorithm\/#Python_Implementation\" title=\"Python Implementation\">Python Implementation<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-4'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/mlinsightscentral.com\/index.php\/gradient-descent-algorithm\/#Gradient_descent_algorithm_in_Machine_learning\" title=\"Gradient descent algorithm in Machine learning\">Gradient descent algorithm in Machine learning<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-4'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/mlinsightscentral.com\/index.php\/gradient-descent-algorithm\/#Limitations_of_the_Gradient_descent_algorithm\" title=\"Limitations of the Gradient descent algorithm\">Limitations of the Gradient descent algorithm<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-4'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/mlinsightscentral.com\/index.php\/gradient-descent-algorithm\/#Conclusion\" title=\"Conclusion\">Conclusion<\/a><\/li><\/ul><\/nav><\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-d3cbce5 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"d3cbce5\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-272b4a4\" data-id=\"272b4a4\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-f3f17a8 elementor-widget elementor-widget-text-editor\" data-id=\"f3f17a8\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>The gradient descent algorithm is an iterative derivative-based local optimisation search algorithm that aims to find the optimum vector value that minimises the objective or cost function. For those not familiar with optimisation theory, the gradient descent algorithm aims to find the coordinate location where a given function (i.e. the objective function) is at its lowest value. Such a point is called the &#8220;optimum&#8221; or &#8220;minimum&#8221; point.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-3b4e728 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"3b4e728\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-07cd9bf\" data-id=\"07cd9bf\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-7e5d5fa elementor-widget elementor-widget-image\" data-id=\"7e5d5fa\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<style>\/*! elementor - v3.13.3 - 28-05-2023 *\/\n.elementor-widget-image{text-align:center}.elementor-widget-image a{display:inline-block}.elementor-widget-image a img[src$=\".svg\"]{width:48px}.elementor-widget-image img{vertical-align:middle;display:inline-block}<\/style>\t\t\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"579\" height=\"453\" src=\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/08\/obj_function2.png\" class=\"attachment-large size-large wp-image-1491\" alt=\"\" srcset=\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/08\/obj_function2.png 579w, https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/08\/obj_function2-300x235.png 300w\" sizes=\"auto, (max-width: 579px) 100vw, 579px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-fa37f40 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"fa37f40\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-d2cee05\" data-id=\"d2cee05\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-0e96047 elementor-widget elementor-widget-image\" data-id=\"0e96047\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"291\" height=\"124\" src=\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/08\/min_func.png\" class=\"attachment-large size-large wp-image-1489\" alt=\"\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-fa5a173 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"fa5a173\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-fd6c4a6\" data-id=\"fd6c4a6\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-761e13d elementor-widget elementor-widget-heading\" data-id=\"761e13d\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h4 class=\"elementor-heading-title elementor-size-default\">Gradient descent - iterative search formula<\/h4>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-5a1e0ca elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"5a1e0ca\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-d638d95\" data-id=\"d638d95\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-874174e elementor-widget elementor-widget-text-editor\" data-id=\"874174e\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>A solution technique must thus be devised to find the value of x that will correspond to the smallest value of f. One such technique for differentiable functions is to use the gradient descent algorithm, that is, given an initial x_0 value, a search direction must be devised to iteratively get closer to the optimum point. Using gradient information can help reach such a point, as the gradient provides information on the direction of the steepest descent or slope:<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-215f054 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"215f054\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-a4ef985\" data-id=\"a4ef985\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-8c1ee01 elementor-widget elementor-widget-image\" data-id=\"8c1ee01\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"636\" height=\"119\" src=\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/08\/gradient_search.png\" class=\"attachment-large size-large wp-image-1492\" alt=\"gradient search\" srcset=\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/08\/gradient_search.png 636w, https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/08\/gradient_search-300x56.png 300w\" sizes=\"auto, (max-width: 636px) 100vw, 636px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-611cc67 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"611cc67\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-73932ee\" data-id=\"73932ee\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-07e6795 elementor-widget elementor-widget-text-editor\" data-id=\"07e6795\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>Thus this iterative update of the initial guess continues repeatedly until a stopping criterion is reached. Typically, the algorithm is stopped when the derivative of the function nears zero and does not significantly change over time.<\/p><p>Although the algorithm is simple, the choice of the learning rate is critical to the success of the search. A too-small learning rate would slow down the search and thus waste time while a too-large learning rate would lead to jumping far away from the optimum point and eventually missing it.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-6619179 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"6619179\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-50 elementor-top-column elementor-element elementor-element-b5680bd\" data-id=\"b5680bd\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-9f11cd5 elementor-widget elementor-widget-image\" data-id=\"9f11cd5\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"559\" height=\"433\" src=\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/08\/gradient_search_too_large.png\" class=\"attachment-large size-large wp-image-1504\" alt=\"\" srcset=\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/08\/gradient_search_too_large.png 559w, https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/08\/gradient_search_too_large-300x232.png 300w\" sizes=\"auto, (max-width: 559px) 100vw, 559px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t<div class=\"elementor-column elementor-col-50 elementor-top-column elementor-element elementor-element-b4fd8e7\" data-id=\"b4fd8e7\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-df00e18 elementor-widget elementor-widget-image\" data-id=\"df00e18\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"559\" height=\"433\" src=\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/08\/gradient_search_too_small.png\" class=\"attachment-large size-large wp-image-1505\" alt=\"Too small learning rate\" srcset=\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/08\/gradient_search_too_small.png 559w, https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/08\/gradient_search_too_small-300x232.png 300w\" sizes=\"auto, (max-width: 559px) 100vw, 559px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-e459312 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"e459312\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-f1a76d9\" data-id=\"f1a76d9\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-9e9e3d6 elementor-widget elementor-widget-heading\" data-id=\"9e9e3d6\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h4 class=\"elementor-heading-title elementor-size-default\"><span class=\"ez-toc-section\" id=\"Python_Implementation\"><\/span>Python Implementation<span class=\"ez-toc-section-end\"><\/span><\/h4>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-30b4862 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"30b4862\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-8a36dcc\" data-id=\"8a36dcc\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap\">\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-b8072d0 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"b8072d0\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-61e6cce\" data-id=\"61e6cce\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-6abff9a elementor-widget elementor-widget-elementor-syntax-highlighter\" data-id=\"6abff9a\" data-element_type=\"widget\" data-widget_type=\"elementor-syntax-highlighter.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<pre><code class='language-python'>import numpy as np\r\nimport matplotlib.pyplot as plt\r\n\r\n#objective function\r\ndef obj_func(x):\r\n    y = np.sin(x) + np.sin((10.0 \/ 3.0) * x)\r\n    return y\r\n#compute the derivative function\r\ndef derivate_func(x):\r\n    dx = np.cos(x) + (10\/3)*np.cos((10.0\/3.0)*x)\r\n    return dx\r\n#search bounds\r\nx_l = -1.3\r\nx_u = 0.5\r\n#function data\r\nx_i = np.arange(x_l,x_u,step=0.01)\r\ny_i = obj_func(x_i)\r\nplt.figure()\r\nplt.xlabel('x')\r\nplt.ylabel('y')\r\nplt.plot(x_i,y_i)\r\nplt.title('Find the optimum point for f(x)=sin(x)+sin(10x\/3)')\r\nplt.show()\r\n\r\n#Gradient descent algorithm\r\nN_search_iters = 100 #number of search steps\r\nx_n = x_l #initial value\r\ngamma = 0.05 #learning rate\r\n\r\nplt.figure()\r\nplt.plot(x_i,y_i,color=&quot;blue&quot;)\r\nfor i in range(N_search_iters):\r\n    x_np = x_n - gamma*derivate_func(x_n) #gradient descent search formula\r\n    x_0 = x_n#gathering data points\r\n    x_1 = x_np #gathering data points\r\n    #update x_n\r\n    x_n = x_np\r\n    #plotting search points\r\n    y_0 = obj_func(x_0)\r\n    y_1 = obj_func(x_1)\r\n    plt.scatter(x_0,y_0,color=&quot;red&quot;)\r\n    plt.scatter(x_1,y_1,color=&quot;red&quot;)\r\n    plt.plot([x_0,x_1],[y_0,y_1],color=&quot;green&quot;)\r\n    \r\nplt.title('Gradient descent search - Learning rate : 0.05')\r\nplt.show()\r\nx_optimum = x_n\r\ny_opt = obj_func(x_optimum)\r\nprint(&quot;x-optimum:%.3f f-optimum:%.3f&quot;%(x_optimum,y_opt))\r\n <\/code><\/pre><script>\nif (!document.getElementById('syntaxed-prism')) {\n\tvar my_awesome_script = document.createElement('script');\n\tmy_awesome_script.setAttribute('src','https:\/\/mlinsightscentral.com\/wp-content\/plugins\/syntax-highlighter-for-elementor\/assets\/prism2.js');\n\tmy_awesome_script.setAttribute('id','syntaxed-prism');\n\tdocument.body.appendChild(my_awesome_script);\n} else {\n\twindow.Prism && Prism.highlightAll();\n}\n<\/script>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-cb65671 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"cb65671\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-1fc758b\" data-id=\"1fc758b\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-a02c001 elementor-widget elementor-widget-image\" data-id=\"a02c001\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"559\" height=\"433\" src=\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/08\/gradient_search_balanced.png\" class=\"attachment-large size-large wp-image-1506\" alt=\"Gradient search balanced\" srcset=\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/08\/gradient_search_balanced.png 559w, https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/08\/gradient_search_balanced-300x232.png 300w\" sizes=\"auto, (max-width: 559px) 100vw, 559px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-35c74ea elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"35c74ea\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-88d8499\" data-id=\"88d8499\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-88d36c6 elementor-widget elementor-widget-image\" data-id=\"88d36c6\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"297\" height=\"34\" src=\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/08\/optimum_point.png\" class=\"attachment-large size-large wp-image-1511\" alt=\"optimum point\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-26f292b elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"26f292b\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-d9d3b42\" data-id=\"d9d3b42\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-561aa62 elementor-widget elementor-widget-text-editor\" data-id=\"561aa62\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>Using a balanced learning rate (i.e. 0.05) and the gradient iterative search the optimum point is obtained x=-0.549, which is the point where the function reaches its smallest value f=-1.488.\u00a0<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-cc240e2 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"cc240e2\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-750cf6e\" data-id=\"750cf6e\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-0325249 elementor-widget elementor-widget-heading\" data-id=\"0325249\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h4 class=\"elementor-heading-title elementor-size-default\"><span class=\"ez-toc-section\" id=\"Gradient_descent_algorithm_in_Machine_learning\"><\/span>Gradient descent algorithm in Machine learning<span class=\"ez-toc-section-end\"><\/span><\/h4>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-2224933 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"2224933\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-57ba9b6\" data-id=\"57ba9b6\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-10951d3 elementor-widget elementor-widget-text-editor\" data-id=\"10951d3\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>The gradient descent algorithm is typically used in machine learning to estimate model parameters. Given, a generic learning model of the type<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-7999271 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"7999271\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-ea0326b\" data-id=\"ea0326b\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-24f1f24 elementor-widget elementor-widget-image\" data-id=\"24f1f24\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"639\" height=\"98\" src=\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/08\/learning_model.png\" class=\"attachment-large size-large wp-image-1547\" alt=\"\" srcset=\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/08\/learning_model.png 639w, https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/08\/learning_model-300x46.png 300w\" sizes=\"auto, (max-width: 639px) 100vw, 639px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-3e556c9 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"3e556c9\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-d3244b1\" data-id=\"d3244b1\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-67f7bb1 elementor-widget elementor-widget-text-editor\" data-id=\"67f7bb1\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>A cost function that aims to minimise the sum of square error between the model prediction (y_p) and the actual observation data (y_d)\u00a0 is often built to estimate the most appropriate model parameters.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-ed1970f elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"ed1970f\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-8685d61\" data-id=\"8685d61\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-8026bdf elementor-widget elementor-widget-image\" data-id=\"8026bdf\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"367\" height=\"182\" src=\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/08\/sse_generic.png\" class=\"attachment-large size-large wp-image-1555\" alt=\"\" srcset=\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/08\/sse_generic.png 367w, https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/08\/sse_generic-300x149.png 300w\" sizes=\"auto, (max-width: 367px) 100vw, 367px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-eebb6d7 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"eebb6d7\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-8a572a9\" data-id=\"8a572a9\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-9c201d3 elementor-widget elementor-widget-text-editor\" data-id=\"9c201d3\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>The optimisation problem below is thus formulated and commonly solved using the gradient descent algorithm.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-511f828 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"511f828\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-005a9f2\" data-id=\"005a9f2\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-21c0776 elementor-widget elementor-widget-image\" data-id=\"21c0776\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"327\" height=\"128\" src=\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/08\/min_sse.png\" class=\"attachment-large size-large wp-image-1556\" alt=\"min_sse\" srcset=\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/08\/min_sse.png 327w, https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/08\/min_sse-300x117.png 300w\" sizes=\"auto, (max-width: 327px) 100vw, 327px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-ddcc8fa elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"ddcc8fa\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-ff6381a\" data-id=\"ff6381a\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-d3da78a elementor-widget elementor-widget-text-editor\" data-id=\"d3da78a\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>leading to a more generic gradient search update formula that takes into account multiple parameter estimates:<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-c5783fb elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"c5783fb\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-6368fbf\" data-id=\"6368fbf\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-fcf333f elementor-widget elementor-widget-image\" data-id=\"fcf333f\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"816\" height=\"147\" src=\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/08\/generic_search_descent.png\" class=\"attachment-large size-large wp-image-1561\" alt=\"multivariate gradient descent formula\" srcset=\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/08\/generic_search_descent.png 816w, https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/08\/generic_search_descent-300x54.png 300w, https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/08\/generic_search_descent-768x138.png 768w\" sizes=\"auto, (max-width: 816px) 100vw, 816px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-e00ade2 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"e00ade2\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-72d0b63\" data-id=\"72d0b63\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-f713dd7 elementor-widget elementor-widget-text-editor\" data-id=\"f713dd7\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>This algorithm is often built into several machine-learning packages and automatically used during the training process.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-34d9505 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"34d9505\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-4b55c66\" data-id=\"4b55c66\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-9f23f27 elementor-widget elementor-widget-heading\" data-id=\"9f23f27\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h4 class=\"elementor-heading-title elementor-size-default\"><span class=\"ez-toc-section\" id=\"Limitations_of_the_Gradient_descent_algorithm\"><\/span>Limitations of the Gradient descent algorithm<span class=\"ez-toc-section-end\"><\/span><\/h4>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-74ba97d elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"74ba97d\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-196dcca\" data-id=\"196dcca\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-80afaec elementor-widget elementor-widget-text-editor\" data-id=\"80afaec\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>One pertinent limitation of the gradient descent algorithm is that it can guarantee the obtention of the true optimum when only the function has one convex lobe.\u00a0 For multimodal functions, with multiple potential optima, the gradient descent algorithm can only obtain the (local) optimum near the initial guess. Nevertheless, this is not a major challenge in machine learning which settles with obtaining a good enough relative minimum as aiming to find the global optimum is unpractical and generally very resource intensive.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-52392ba elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"52392ba\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-6d5a711\" data-id=\"6d5a711\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-607e899 elementor-widget elementor-widget-image\" data-id=\"607e899\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"579\" height=\"453\" src=\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/08\/multimodal_function.png\" class=\"attachment-large size-large wp-image-1527\" alt=\"\" srcset=\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/08\/multimodal_function.png 579w, https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/08\/multimodal_function-300x235.png 300w\" sizes=\"auto, (max-width: 579px) 100vw, 579px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-5d99bee elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"5d99bee\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-55504b8\" data-id=\"55504b8\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-b1801b8 elementor-widget elementor-widget-heading\" data-id=\"b1801b8\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h4 class=\"elementor-heading-title elementor-size-default\"><span class=\"ez-toc-section\" id=\"Conclusion\"><\/span>Conclusion<span class=\"ez-toc-section-end\"><\/span><\/h4>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-3b24ecb elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"3b24ecb\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-429057e\" data-id=\"429057e\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-5e0cd72 elementor-widget elementor-widget-text-editor\" data-id=\"5e0cd72\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p>The gradient descent algorithm is a simple iterative derivative-based local optimisation algorithm commonly used in machine learning for model training (i.e. parameter estimation). The algorithm makes use of gradient information of the objective function to lead the search towards the optimum point of minimisation. The choice of the learning rate is crucial to the speed and success of the search algorithm, often requiring that the model input vectors be normalised in order to set a generic balanced learning rate. More <a href=\"https:\/\/mlinsightscentral.com\/index.php\/ml-reinforcement-learning\/\">advanced parameter search algorithms<\/a> such as particle swarm optimisation and other metaheuristics are sometimes used when it is crucial to find the global optimum to yield some improvement in the model pattern recognition performance.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-f228733 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"f228733\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-d4d57dd\" data-id=\"d4d57dd\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-2f596f5 elementor-widget elementor-widget-heading\" data-id=\"2f596f5\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<p class=\"elementor-heading-title elementor-size-default\">Author: Yves Matanga, PhD<\/p>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-7f2ab7e elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"7f2ab7e\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-b3cffec\" data-id=\"b3cffec\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-503050c5 elementor-widget elementor-widget-heading\" data-id=\"503050c5\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<div class=\"elementor-heading-title elementor-size-default\">Be the first to receive notification, when new content is available!<\/div>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-5a1939b7 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"5a1939b7\" data-element_type=\"section\">\n\t\t\t\t\t\t\t<div class=\"elementor-background-overlay\"><\/div>\n\t\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-440e8442\" data-id=\"440e8442\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-1ab8c415 elementor-widget elementor-widget-wpforms\" data-id=\"1ab8c415\" data-element_type=\"widget\" data-widget_type=\"wpforms.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<div class=\"wpforms-container inline-fields\" id=\"wpforms-10\"><form id=\"wpforms-form-10\" class=\"wpforms-validate wpforms-form\" data-formid=\"10\" method=\"post\" enctype=\"multipart\/form-data\" action=\"\/index.php\/wp-json\/wp\/v2\/pages\/1468\" data-token=\"bbd3c4710160bb8dec80be36f260a4f7\"><noscript class=\"wpforms-error-noscript\">Please enable JavaScript in your browser to complete this form.<\/noscript><div class=\"wpforms-field-container\"><div id=\"wpforms-10-field_1-container\" class=\"wpforms-field wpforms-field-email wpforms-mobile-full\" data-field-id=\"1\"><label class=\"wpforms-field-label wpforms-label-hide\" for=\"wpforms-10-field_1\">Email <span class=\"wpforms-required-label\">*<\/span><\/label><input type=\"email\" id=\"wpforms-10-field_1\" class=\"wpforms-field-large wpforms-field-required\" name=\"wpforms[fields][1]\" placeholder=\"Email address\" spellcheck=\"false\" required><\/div><\/div><!-- .wpforms-field-container --><div class=\"wpforms-submit-container\" ><input type=\"hidden\" name=\"wpforms[id]\" value=\"10\"><input type=\"hidden\" name=\"wpforms[author]\" value=\"1\"><button type=\"submit\" name=\"wpforms[submit]\" id=\"wpforms-submit-10\" class=\"wpforms-submit\" data-alt-text=\"Sending...\" data-submit-text=\"Subscribe\" aria-live=\"assertive\" value=\"wpforms-submit\">Subscribe<\/button><\/div><\/form><\/div>  <!-- .wpforms-container -->\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"<p>Gradient Descent Algorithm The gradient descent algorithm is an iterative derivative-based local optimisation search algorithm that aims to find the optimum vector value that minimises the objective or cost function. For those not familiar with optimisation theory, the gradient descent algorithm aims to find the coordinate location where a given function (i.e. the objective function) &hellip;<\/p>\n<p class=\"read-more\"> <a class=\"\" href=\"https:\/\/mlinsightscentral.com\/index.php\/gradient-descent-algorithm\/\"> <span class=\"screen-reader-text\">Gradient Descent Algorithm<\/span> Read More &raquo;<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"open","ping_status":"closed","template":"","meta":{"_uag_custom_page_level_css":"","site-sidebar-layout":"no-sidebar","site-content-layout":"page-builder","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"disabled","ast-breadcrumbs-content":"","ast-featured-img":"disabled","footer-sml-layout":"","theme-transparent-header-meta":"","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","footnotes":""},"wf_page_folders":[21],"class_list":["post-1468","page","type-page","status-publish","hentry"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v20.11 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Gradient Descent Algorithm - MLInsightsCentral<\/title>\n<meta name=\"description\" content=\"The gradient descent algorithm is an iterative derivative-based local optimisation commonly to train machine learning models.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/mlinsightscentral.com\/index.php\/gradient-descent-algorithm\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Gradient Descent Algorithm - MLInsightsCentral\" \/>\n<meta property=\"og:description\" content=\"The gradient descent algorithm is an iterative derivative-based local optimisation commonly to train machine learning models.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/mlinsightscentral.com\/index.php\/gradient-descent-algorithm\/\" \/>\n<meta property=\"og:site_name\" content=\"MLInsightsCentral\" \/>\n<meta property=\"article:modified_time\" content=\"2023-09-18T15:11:08+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/08\/obj_function2.png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"6 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/mlinsightscentral.com\/index.php\/gradient-descent-algorithm\/\",\"url\":\"https:\/\/mlinsightscentral.com\/index.php\/gradient-descent-algorithm\/\",\"name\":\"Gradient Descent Algorithm - MLInsightsCentral\",\"isPartOf\":{\"@id\":\"https:\/\/mlinsightscentral.com\/#website\"},\"datePublished\":\"2023-08-28T20:15:06+00:00\",\"dateModified\":\"2023-09-18T15:11:08+00:00\",\"description\":\"The gradient descent algorithm is an iterative derivative-based local optimisation commonly to train machine learning models.\",\"breadcrumb\":{\"@id\":\"https:\/\/mlinsightscentral.com\/index.php\/gradient-descent-algorithm\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/mlinsightscentral.com\/index.php\/gradient-descent-algorithm\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/mlinsightscentral.com\/index.php\/gradient-descent-algorithm\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/mlinsightscentral.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Gradient Descent Algorithm\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/mlinsightscentral.com\/#website\",\"url\":\"https:\/\/mlinsightscentral.com\/\",\"name\":\"MLInsightsCentral\",\"description\":\"Learn Machine Learning and AI for engineers, data scientists and AI practionners.\",\"publisher\":{\"@id\":\"https:\/\/mlinsightscentral.com\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/mlinsightscentral.com\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/mlinsightscentral.com\/#organization\",\"name\":\"MLInsightsCentral\",\"url\":\"https:\/\/mlinsightscentral.com\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/mlinsightscentral.com\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/06\/cropped-1290538dccf74accb0ae585ff4e8586c-1.png\",\"contentUrl\":\"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/06\/cropped-1290538dccf74accb0ae585ff4e8586c-1.png\",\"width\":200,\"height\":110,\"caption\":\"MLInsightsCentral\"},\"image\":{\"@id\":\"https:\/\/mlinsightscentral.com\/#\/schema\/logo\/image\/\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Gradient Descent Algorithm - MLInsightsCentral","description":"The gradient descent algorithm is an iterative derivative-based local optimisation commonly to train machine learning models.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/mlinsightscentral.com\/index.php\/gradient-descent-algorithm\/","og_locale":"en_US","og_type":"article","og_title":"Gradient Descent Algorithm - MLInsightsCentral","og_description":"The gradient descent algorithm is an iterative derivative-based local optimisation commonly to train machine learning models.","og_url":"https:\/\/mlinsightscentral.com\/index.php\/gradient-descent-algorithm\/","og_site_name":"MLInsightsCentral","article_modified_time":"2023-09-18T15:11:08+00:00","og_image":[{"url":"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/08\/obj_function2.png"}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"6 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/mlinsightscentral.com\/index.php\/gradient-descent-algorithm\/","url":"https:\/\/mlinsightscentral.com\/index.php\/gradient-descent-algorithm\/","name":"Gradient Descent Algorithm - MLInsightsCentral","isPartOf":{"@id":"https:\/\/mlinsightscentral.com\/#website"},"datePublished":"2023-08-28T20:15:06+00:00","dateModified":"2023-09-18T15:11:08+00:00","description":"The gradient descent algorithm is an iterative derivative-based local optimisation commonly to train machine learning models.","breadcrumb":{"@id":"https:\/\/mlinsightscentral.com\/index.php\/gradient-descent-algorithm\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/mlinsightscentral.com\/index.php\/gradient-descent-algorithm\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/mlinsightscentral.com\/index.php\/gradient-descent-algorithm\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/mlinsightscentral.com\/"},{"@type":"ListItem","position":2,"name":"Gradient Descent Algorithm"}]},{"@type":"WebSite","@id":"https:\/\/mlinsightscentral.com\/#website","url":"https:\/\/mlinsightscentral.com\/","name":"MLInsightsCentral","description":"Learn Machine Learning and AI for engineers, data scientists and AI practionners.","publisher":{"@id":"https:\/\/mlinsightscentral.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/mlinsightscentral.com\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/mlinsightscentral.com\/#organization","name":"MLInsightsCentral","url":"https:\/\/mlinsightscentral.com\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/mlinsightscentral.com\/#\/schema\/logo\/image\/","url":"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/06\/cropped-1290538dccf74accb0ae585ff4e8586c-1.png","contentUrl":"https:\/\/mlinsightscentral.com\/wp-content\/uploads\/2023\/06\/cropped-1290538dccf74accb0ae585ff4e8586c-1.png","width":200,"height":110,"caption":"MLInsightsCentral"},"image":{"@id":"https:\/\/mlinsightscentral.com\/#\/schema\/logo\/image\/"}}]}},"uagb_featured_image_src":{"full":false,"thumbnail":false,"medium":false,"medium_large":false,"large":false,"1536x1536":false,"2048x2048":false},"uagb_author_info":{"display_name":"admin","author_link":"https:\/\/mlinsightscentral.com\/index.php\/author\/yvesm\/"},"uagb_comment_info":2,"uagb_excerpt":"Gradient Descent Algorithm The gradient descent algorithm is an iterative derivative-based local optimisation search algorithm that aims to find the optimum vector value that minimises the objective or cost function. For those not familiar with optimisation theory, the gradient descent algorithm aims to find the coordinate location where a given function (i.e. the objective function)&hellip;","_links":{"self":[{"href":"https:\/\/mlinsightscentral.com\/index.php\/wp-json\/wp\/v2\/pages\/1468","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mlinsightscentral.com\/index.php\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mlinsightscentral.com\/index.php\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mlinsightscentral.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/mlinsightscentral.com\/index.php\/wp-json\/wp\/v2\/comments?post=1468"}],"version-history":[{"count":134,"href":"https:\/\/mlinsightscentral.com\/index.php\/wp-json\/wp\/v2\/pages\/1468\/revisions"}],"predecessor-version":[{"id":2074,"href":"https:\/\/mlinsightscentral.com\/index.php\/wp-json\/wp\/v2\/pages\/1468\/revisions\/2074"}],"wp:attachment":[{"href":"https:\/\/mlinsightscentral.com\/index.php\/wp-json\/wp\/v2\/media?parent=1468"}],"wp:term":[{"taxonomy":"wf_page_folders","embeddable":true,"href":"https:\/\/mlinsightscentral.com\/index.php\/wp-json\/wp\/v2\/wf_page_folders?post=1468"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}