机器学习中的线性回归 (Linear Regression in Machine Learning)

There are two types of supervised machine learning algorithms or task: Regression and classification.

有监督的机器学习算法或任务有两种类型：回归和分类。

• Classification — Classification is a process of categorizing a given set of data into classes, it can be performed on both structured or unstructured data. The process starts with predicting the class of given data points. The classes are often referred to as target, label or categories.

•分类-分类是将给定数据集分类为类的过程，可以在结构化或非结构化数据上执行。 该过程从预测给定数据点的类别开始。 这些类通常称为目标，标签或类别。

Example — Spam, Check defaulter in loan applicant.

示例—垃圾邮件，检查贷款申请人中的违约者。

• Regression — It predicts the continuous value based on historical data. It Predict the future values on the basis of historical data. Example — No of Corona Patients in July, Sales of car in 2021

•回归-根据历史数据预测连续值。 它根据历史数据预测未来价值。 示例— 7月的电晕患者人数，2021年的汽车销量

Linear equation in algebra is the function for this algorithm, we try to find a linear relationship between two or more variables. If we draw this relationship in a two-dimensional space, we get a straight line. (between two variables),

代数中的线性方程是该算法的函数，我们试图找到两个或多个变量之间的线性关系。 如果在二维空间中绘制此关系，则会得到一条直线。 (在两个变量之间)，

It predicts the continuous variable “Y” based on given independent variable “X”. If we plot the independent variable (x) on the x-axis and dependent variable (y) on the y-axis. This algorithm gives us a straight line between y and x axis.

它基于给定的自变量“ X”预测连续变量“ Y”。 如果我们在x轴上绘制自变量(x)，在y轴上绘制因变量(y)。 该算法为我们在y和x轴之间提供了一条直线。

So, our equation looks like below.

因此，我们的等式如下所示。

Y =θ0 +θ1.X (Y = θ0 + θ1.X)

Where, y is predicted value

y是预测值

X is input or Features or Columns

X是输入或特征或列

θ0 and θ1 are the model’s parameters. 1st one is intercept or Bias and 2nd one is slope or Weight.

θ0和θ1是模型的参数。 第一个是拦截或偏差，第二个是斜率或权重。

You can see the equation looks like y = mx + c, which we have studied in our school curriculum's, where m is slope and c is intercept.

您会看到方程看起来像y = mx + c，这是我们在学校课程中研究的方程，其中m是斜率，c是截距。

More generally, a linear model makes a prediction by simply computing a weighted sum of the input features, plus a constant called the bias term (also called the intercept term), We can represent the equation for n independent variables or features or column.

更一般而言，线性模型通过简单地计算输入要素的加权总和加上一个称为偏差项 (也称为截距项 )的常数来进行预测。我们可以表示n个独立变量或要素或列的方程式。

Y =θ0 X 0 +θ1×1 +θ2×2 +⋯+θnxn (y = θ0 x0+ θ1x1 + θ2x2 + ⋯ + θnxn)

• ŷ is the predicted value.

• ŷ是预测值。

• n is the number of features.

• n是要素数量。

• xi is the ith feature value. X0 is always 0

• x i是第i个特征值。 X0始终为0

• θj is the jth model parameter (including the bias term θ0 and the feature weights

•θj的是第j个模型参数(包括偏项θ0和特征权重

θ1, θ2, ⋯, θn).

θ1 ， θ2 ，⋯， θn )。

Same we can write in Vector form.

同样，我们可以用矢量形式写。

• θ is the model’s parameter vector, containing the bias term θ0 and the feature

•θ是该模型的参数向量 ，含有偏项θ0和特征

weights θ1 to θn.

权重为θ1至θn 。

• x is the instance’s feature vector, containing x0 to xn, with x0 always equal to 1.

•x是实例的特征向量 ，从x 0到xn ， x 0始终等于1。

• θ ・ x is the dot product of the vectors θ and x, which is equal to

•θ・ x是向量θ和x的点积，等于

θ0x0 + θ1x1 + θ2x2 + ⋯ + θnxn.

θ0 X 0 +θ1×1 +θ2×2 +⋯+θnxn。

• hθ is the hypothesis function, using the model parameters θ.

• 水平 θ是设定功能，利用该模型参数θ。

In Machine Learning, vectors are often represented as column vectors, which are 2D arrays with a single column. If θ and x are column vectors, then the prediction is: y = θT x, where θT is the transpose of θ (Swap the rows to column or column to rows) and θT x is the matrix multiplication of θT and x. It is of course the same prediction, except it is now represented as a single cell matrix rather than a scalar value.

在机器学习中，向量通常表示为列向量，它们是具有单个列的2D数组。 如果θ和x是列向量，则预测为：y =θTx，其中θT是θ的转置(将行交换到列或将列交换到行)，θTx是θT和x的矩阵乘法。 当然，它是相同的预测，只不过它现在表示为单个单元矩阵而不是标量值。

Transpose — Swapping the rows to column of matrix or vice versa

转置—将行交换到矩阵的列，反之亦然

准备数据进行线性回归 (Preparing Data for Linear Regression)

For an algorithm, data is most crucial part. There is saying “Garbage in, garbage out”. It means if you feed the garbage (Irrelevant data or noisy data), output of your model will be garbage (inaccurate). Hence, we need data to be in best form, before feeding and testing the model.

对于算法而言，数据是最关键的部分。 有人说“垃圾进，垃圾出”。 这意味着如果您输入垃圾(Irelevant数据或嘈杂数据)，则模型的输出将是垃圾(不准确)。 因此，在馈送和测试模型之前，我们需要数据以最佳形式出现。

Below are few techniques to clean scale and modify data for Linear Regression. Linear Regression assumes below points like data is noise less and relationship between independent variable and dependent variables.

以下是一些清理线性比例和修改数据以线性回归的技术。 线性回归假设以下几点，例如数据更少噪音以及自变量和因变量之间的关系。

1-> Linear Assumption: Linear regression assumes that the relationship between your independent variable or X (input) and dependent variable or Y (output) is linear or it tends to be a linear for better performance. If relationship is not linear, Transformation can be applied on data.

1->线性假设：线性回归假设您的自变量或X(输入)与因变量或Y(输出)之间的关系是线性的，或者为了获得更好的性能而倾向于线性。 如果关系不是线性的，则可以将转换应用于数据。

2-> Remove Collinearity: If your independent variables have relation to each other, we should just take the most co related to Y and remove rest related to that X variable. Example If your data has DOB and Age, we can remove one.

2->删除共线性：如果您的自变量相互之间具有关联，则我们应仅取与Y相关的最大co并删除与该X变量相关的其余co。 示例如果您的数据具有DOB和Age，我们可以删除其中一个。

3-> Gaussian Distributions: Linear regression will make more reliable predictions if your input and output variables have a Gaussian distribution or bell-shaped curve. You may get some benefit using transforms on your variables to make their distribution more Gaussian looking.

3->高斯分布：如果您的输入和输出变量具有高斯分布或钟形曲线，则线性回归将提供更可靠的预测。 通过对变量进行变换，使它们的分布更具高斯外观，您可能会获得一些好处。

4-> Remove Noise: Linear regression assumes that your input and output variables are not noisy. We need to clean the data before feeding it to the model. This is most important for the output variable and you want to remove outliers in the dependent variable or Y if possible.

4->消除噪声：线性回归假设您的输入和输出变量没有噪音。 我们需要先清理数据，然后再将其提供给模型。 这对于输出变量最重要，如果可能，您要删除因变量中的异常值或Y。

5-> Rescale Inputs: Linear regression will often make more reliable predictions if you rescale input variables using standardization or normalization y.

5->重新缩放输入：如果您使用标准化或归一化y重新缩放输入变量，则线性回归通常会做出更可靠的预测。

Medium Story on Standardscaler vs MinMax Scaler (Normalization) — https://medium.com/@amitupadhyay6/standardscaler-and-normalization-with-code-and-graph-ba220025c054

关于Standardscaler与MinMax Scaler(规范化)的中级故事— https://medium.com/@amitupadhyay6/standardscaler-and-normalization-with-code-and-graph-ba220025c054

选择绩效指标 (Select a Performance Measure)

Our model is ready to predict the value, but before putting it in production, we need to check the performance of the model. For this purpose, we first need a measure of how well (or poorly) the model fits the training data. Most common performance measure of a regression model is the Root Mean Square Error (RMSE). It gives an idea of how much error the system typically makes in its predictions, with a higher weight for large errors. Therefore, to train a Linear Regression model, you need to find the value of θ that minimizes the RMSE. In practice, it is simpler to minimize the Mean Square Error (MSE) than the RMSE, and it leads to the same result.

我们的模型已经可以预测价值了，但是在将其投入生产之前，我们需要检查模型的性能。 为此，我们首先需要衡量模型拟合训练数据的好坏程度。 回归模型最常见的性能指标是均方根误差(RMSE)。 它给出了系统通常会在预测中产生多少错误的想法，对于较大的错误，权重较高。 因此，要训练线性回归模型，您需要找到使RMSE最小的θ值。 实际上，最小均方误差(MSE)比RMSE更简单，并且得到相同的结果。

Root Mean Square Error:

根均方误差：

The RMSE is the square root of the variance of the residuals. It indicates the absolute fit of the model to the data–how close the observed data points are to the model’s predicted values. Whereas R-squared is a relative measure of fit, RMSE is an absolute measure of fit.

RMSE是残差方差的平方根。 它表示模型与数据的绝对拟合-观察到的数据点与模型的预测值有多接近。 R平方是拟合的相对度量，而RMSE是拟合的绝对度量。

Mean Square Error:

均方误差：

The mean squared error tells you how close a regression line is to a set of points. It does this by taking the distances from the points to the regression line (these distances are the “errors”) and squaring them. It’s called the mean squared error as you’re finding the average of a set of errors.

均方误差告诉您回归线与一组点的接近程度。 它通过获取从点到回归线的距离(这些距离就是“误差”)并对它们进行平方来实现。 当您找到一组误差的平均值时，这称为均方误差。

Mean Absolute Error:

平均绝对误差：

Even though the RMSE is generally the preferred performance measure for regression tasks, in some contexts you may prefer to use another function. For example, suppose that there are many outliers in your data. In that case, you may consider using the Mean Absolute Error .

尽管RMSE通常是回归任务的首选性能指标，但在某些情况下，您可能更喜欢使用其他功能。 例如，假设您的数据中有许多异常值。 在这种情况下，您可以考虑使用平均绝对误差。

Mean Absolute Error (MAE) is another loss function used for regression models. MAE is the sum of absolute differences between our target and predicted variables. So it measures the average magnitude of errors in a set of predictions, without considering their directions

平均绝对误差(MAE)是用于回归模型的另一个损失函数。 MAE是我们的目标变量和预测变量之间的绝对差之和。 因此，它可以测量一组预测中的平均误差幅度，而无需考虑其方向

• Computing the root of a sum of squares (RMSE) corresponds to the Euclidean

•计算平方根(RMSE)的根对应于欧几里得

norm: It is also called the ℓ2 norm, noted ∥ ・ ∥2 (or just ∥ ・ ∥).

规范：也称为ℓ2规范，记为∥・∥2(或仅称为∥・∥)。

• Computing the sum of absolutes (MAE) corresponds to the ℓ1 norm, noted ∥ ・ ∥1.

•计算绝对和(MAE)对应于ℓ1m1范数。

It is also called the Manhattan norm because it measures the distance between two points.

之所以称为曼哈顿范数，是因为它测量了两个点之间的距离。

The most common interpretation of r-squared is how well the regression model fits the observed data. For example, an r-squared of 60% reveals that 60% of the data fit the regression model. Generally, a higher r-squared indicates a better fit for the model.

r平方的最常见解释是回归模型对观测数据的拟合程度。 例如，r-平方为60％表示60％的数据符合回归模型。 通常，较高的r平方表示该模型更合适。

R-squared or R2 explains the degree to which your input variables explain the variation of your output / predicted variable. So, if R-square is 0.8, it means 80% of the variation in the output variable is explained by the input variables. So, in simple terms, higher the R squared, the more variation is explained by your input variables and hence better is your model.

R平方或R2解释输入变量解释输出/预测变量变化的程度。 因此，如果R平方为0.8，则意味着输出变量中80％的变化由输入变量解释。 因此，简单来说，R平方越高，输入变量说明的变化越大，因此您的模型越好。

Where SSres, Residual sum of squared errors of our regression model or error

其中SSres，我们回归模型的平方误差的残差和或

& SStot, is the total sum of squared errors

＆SStot，是平方误差的总和

Adjusted R Square:

调整后的R平方：

However, the problem with R-squared is that it will either stay the same or increase with addition of more variables, even if they do not have any relationship with the output variables. This is where “Adjusted R square” comes to help. Adjusted R-square penalizes you for adding variables which do not improve your existing model.

但是，R平方的问题在于，即使它们与输出变量没有任何关系，它也会保持不变或随着添加更多变量而增加。 这是“调整后的R平方”提供帮助的地方。 调整后的R平方会惩罚您添加不会改善现有模型的变量。

Hence, if you are building Linear regression on multiple variable, it is always suggested that you use Adjusted R-squared to judge goodness of model. In case you only have one input variable, R-square and Adjusted R squared would be exactly same.

因此，如果要在多个变量上建立线性回归，则始终建议您使用调整后的R平方来判断模型的优劣。 如果只有一个输入变量，则R平方和调整后的R平方将完全相同。

Typically, the more non-significant variables you add into the model, the gap in R-squared and Adjusted R-squared increases.

通常，添加到模型中的变量越不重要，R平方和调整R平方的差距就会增加。

As we know, whenever we add new feature or input variable, R2 square value will stay same or get increases, which is good sign, higher the r2 value, higher the performance of our model, but if you are adding the insignificant variable or feature, even though r2 value will remain same or gets increases. which should not happen, hence we use adjusted r2 square, lets see both the cases.

众所周知，每当我们添加新特征或输入变量时，R2平方值将保持不变或增加，这是一个好兆头，r2值越高，模型的性能越高，但是如果添加的变量或特征不重要， ，即使r2值保持不变或增加。 这不应该发生，因此我们使用调整后的r2平方，让我们看一下这两种情况。

1-> Adding relevant feature or input variable in the data set — If you are adding relevant feature, r2 value will gets increase, which means (1-r2) will be small value and if you multiply the small value with big number { (1-r2) * (n-1)/(n-p-1)}, value will be reduced. ex 10 * 2 = 20 and 10 * 0.5 = 5, hence (1 - { (1-r2) * (n-1)/(n-p-1)}) will be bigger number as we are subtracting the small number from 1.

1->在数据集中添加相关特征或输入变量-如果要添加相关特征，则r2值将增加，这意味着(1-r2)将为小值，并且将小值与大数{( 1-r2)*(n-1)/(np-1)}，值将减小。 ex 10 * 2 = 20和10 * 0.5 = 5，因此(1-{(1-r2)*(n-1)/(np-1)})将是较大的数字，因为我们要从1中减去较小的数字。

2-> Adding irrelevant feature or input variable in the data set — If you are adding irrelevant feature, r2 value will remain same or gets increase slightly, which means (1-r2) will be little less, but if we see the denominator (n-p-1), this value will be reduced, as the p is increases. hence nominator {(1-r2)(n-1)} divide small value, will make the bigger value, ex 10/2 = 5, reduce the denominator 10/0.5 = 20. Hence 1 — bigger number the resultant adjusted r2 value will be reduced. Which should suppose to be happen in the case of irrelevant feature addition in the data set.

2->在数据集中添加不相关的要素或输入变量-如果要添加不相关的要素，则r2值将保持不变或略有增加，这意味着(1-r2)会少一点，但是如果看到分母( np-1)，随着p的增加，该值将减小。 因此，分母{(1-r2)(n-1)}除以较小的值，将得到较大的值，例如10/2 = 5，则分母减少10 / 0.5 =20。因此，1 —较大的数字就是调整后的r2值将减少。 如果在数据集中添加了不相关的功能，则应该发生这种情况。

• YouTube Link: Linear Regression → https://www.youtube.com/watch?v=w8S0uTLTaGA

  YouTube链接：线性回归→ https : //www.youtube.com/watch?v=w8S0uTLTaGA

• Performance measure → https://www.youtube.com/watch?v=eNphW-kjT2I&list=PLbbnl6egUbNhJqmLfwX2eN7XmuqRS8rv8&index=15

  绩效指标→ https://www.youtube.com/watch?v=eNphW-kjT2I&list=PLbbnl6egUbNhJqmLfwX2eN7XmuqRS8rv8&index=15

• GitHub Code → https://github.com/amitupadhyay6/My-Python/blob/master/Linear%20Regression%20on%20Boston.ipynb

  GitHub代码→ https://github.com/amitupadhyay6/My-Python/blob/master/Linear%20Regression%20on%20Boston.ipynb