Machine Learning - Simple linear regression（Principle）

2024-03-16 16:12:01
开发
41

Simple linear regression：

Known data sets $D = {(x_1,y_2),(x_2,y_2)....(x_n,y_n)}$ There is only one self-variable x with a variable y. And x and y satisfy a linear relationship.

For these questions，we are trying to fit the following equation:

$f(x_i) = wx_i + b$

Simultaneously make $f(x_i) \doteq y_i$ in the above equation,It can be seen from this that determining w and b is the most important issue among them.

In the regression task, we use mean squared error as a measure of performance, so we need to determine how much the mean squared error can be minimized when w and b are equal to:

$(w^{*},b^{*}) = argmin_{(w,b)}\sum_{i = 1}^{m}(f(x_i) - y_i)^{2}$

$=argmin_{(w,b)}\sum_{i = 1}^{m}(y_i-wx_i - y_i)^{2}$

In the regression task, we use mean square error as a measure of performance, so we need to find the minimum mean square error when w and b are equal. This model based on minimizing mean square error is called the "least squares method", where the formula for mean square error corresponds exactly to the Euclidean distance. The purpose of this algorithm is to try to find a line that minimizes the sum of Euclidean distance lines for the sample points.

The process of solving w and b is actually finding a function

$E(w,b)=argmin_{(w,b)}\sum_{i = 1}^{m}(y_i-wx_i - y_i)^{2}$

The process of minimization is called the least squares Parameter Estimation of minimum linear regression.

Based on the knowledge of advanced mathematics, we can know that for a multivariate function to find its maximum value, it is to take the partial derivative of the parameter and make it 0,then set the partial derivatives to 0 separately, and finally solve for w and b:

$b = \frac{1}{m} \sum_{i = 1}^{m} (y_i - w_i)$

$w = \frac{ \sum_{i = 1}^{m}y_i(x_i - \bar{x}) }{ \sum_{i = 1}^{m}x_i^{2} - \frac{1}{m}(\sum_{i = 1}^{m}x_i)^{2} }$

This is the result we need to seek.

原文地址:https://blog.csdn.net/weixin_74996125/article/details/136689958 本文来自互联网用户投稿，该文观点仅代表作者本人，不代表本站立场。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如若转载，请注明出处：https://www.suanlizi.com/kf/1768913094823776256.html 如若内容造成侵权/违法违规/事实不符，请联系《酸梨子》网邮箱：1419361763@qq.com进行投诉反馈，一经查实，立即删除！

阅读全部