Evaluation metrics;

评估指标；

1. Mean/Median of prediction预测的均值/中位数
2. Standard Deviation of prediction预测标准差
3. Range of prediction预测范围

4. Coefficient of Determination (R2)

   测定系数(R2)

5. Relative Standard Deviation/Coefficient of Variation (RSD)相对标准偏差/变异系数(RSD)
6. Relative Squared Error (RSE)相对平方误差(RSE)

7. Mean Absolute Error (MAE)

   平均绝对误差(MAE)

8. Relative Absolute Error (RAE)相对绝对误差(RAE)
9. Mean Squared Error (MSE)均方误差(MSE)

10. Root Mean Squared Error on Prediction (RMSE/RMSEP)

    预测的均方根误差(RMSE / RMSEP)

11. Normalized Root Mean Squared Error (Norm RMSEP)归一化均方根误差(范数RMSEP)
12. Relative Root Mean Squared Error (RRMSEP)相对均方根误差(RRMSEP)

Let us consider an example of predicting Active Pharmaceutical Ingredients (API) concentration in a tablet. Using absorbance units from NIR spectroscopy we predict the API level in the tablet. The API concentration in a tablet can be 0.0, 0.1, 0.3, 0.5, 1.0, 1.5, 2.0, 2.5, 3.0. We apply PLS (Partial Least Square) and SVR (Support Vector Regressor) for the prediction of API level.

让我们考虑一个预测片剂中活性药物成分(API)浓度的示例。 使用NIR光谱中的吸光度单位，我们可以预测片剂中的API含量。 片剂中的API浓度可以是0.0、0.1、0.3、0.5、1.0、1.5、2.0、2.5、3.0。 我们将PLS(偏最小二乘)和SVR(支持向量回归)用于API级别的预测。

NOTE: The metrics can be used to compare multiple models or one model with different models

注意：指标可用于比较多个模型或一个模型与不同模型

预测的均值/中位数 (Mean/Median of prediction)

We can understand the bias in prediction between two models using the arithmetic mean of the predicted values.

我们可以使用预测值的算术平均值来理解两个模型之间的预测偏差。

For example, The mean of predicted values of 0.5 API is calculated by taking the sum of the predicted values for 0.5 API divided by the total number of samples having 0.5 API.

例如，通过将0.5 API的预测值的总和除以具有0.5 API的样本总数来计算0.5 API的预测值的平均值。

np.mean(predictedArray)

In Fig.1, We can understand how PLS and SVR have performed wrt mean. SVR predicted 0.0 API much better than PLS, whereas, PLS predicted 3.0 API better than SVR. We can choose the models based on the interest of the API level.

在图1中，我们可以了解PLS和SVR如何执行wrt意思。 SVR预测0.0 API优于PLS，而PLS预测3.0 API优于SVR。 我们可以根据API级别的兴趣来选择模型。

Disadvantage: Mean is affected by outliers. Use Median when you have outliers in your predicted values

缺点：中位数受异常值的影响。 当预测值中有异常值时，请使用“中位数”

预测标准差 (Standard Deviation of prediction)

The standard deviation (SD) is a measure of the amount of variation or dispersion of a set of values. A low standard deviation indicates that the values tend to be close to the mean (also called the expected value) of the set,. In contrast, a high standard deviation indicates that the values are spread out over a broader range. The SD of predicted values helps in understanding the dispersion of values in different models.

标准偏差(SD)是一组值的变化或离散量的度量。 低标准偏差表示这些值趋于接近集合的平均值(也称为期望值)。 相反，高标准偏差表示这些值分布在较宽的范围内。 预测值的SD有助于理解不同模型中值的分散性。

np.std(predictedArray)

In Fig.2, The dispersion of predicted values is less in SVR compared to PLS. So, SVR performs better when we consider the SD metrics.

在图2中，与PLS相比，SVR中预测值的离散度较小。 因此，在考虑SD指标时，SVR的性能更好。

预测范围 (Range of prediction)

The range of the prediction is the maximum and minimum value in the predicted values. Even range helps us to understand the dispersion between models.

预测范围是预测值中的最大值和最小值。 偶数范围可以帮助我们了解模型之间的差异。

测定系数(R2) (Coefficient of Determination (R2))

R-squared (R2) is a statistical measure that represents the proportion of the variance for a dependent variable that’s explained by an independent variable or variables in a regression model. Whereas correlation explains the strength of the relationship between an independent and dependent variable, R-squared explains to what extent the variance of one variable explains the variance of the second variable. So, if the R2 of a model is 0.50, then approximately half of the observed variation can be explained by the model’s inputs.

R平方(R2)是一种统计量度，代表因变量的方差比例，该因变量由回归模型中的一个或多个自变量解释。 相关性说明了独立变量和因变量之间关系的强度，而R平方说明了一个变量的方差在多大程度上解释了第二个变量的方差。 因此，如果模型的R2为0.50，则可以通过模型的输入解释观察到的变化的大约一半。

from sklearn.metrics import r2_scorer2_score(Actual, Predicted)

Disadvantage: R2 doesn’t consider overfitting. For more details.

缺点：R2不会考虑过度拟合。 有关更多详细信息 。

相对标准偏差(RSD)/变异系数(CV) (Relative Standard Deviation (RSD) / Coefficient of Variation (CV))

There is a saying that apples shouldn’t be compared with oranges or in other words, don’t compare two items or group of items that are practically incomparable. But the lack of comparability can be overcome if the two items or groups are somehow standardized or brought on the same scale. For instance, when comparing the variances of two groups that are overall very different, such as the variance in the size of bluefin tuna and blue whales, the coefficient of variation (CV) is the method of choice: the CV simply represents the variance of each group standardized by its group mean

有一种说法是，不应将苹果与桔子相提并论，换句话说，不要将实际上无法比拟的两个项目或一组项目进行比较。 但是，如果将这两个项目或组以某种方式标准化或以相同的规模进行，则可以克服可比性不足的问题。 例如，当比较总体差异很大的两组的方差时，例如蓝鳍金枪鱼和蓝鲸的大小方差，变异系数(CV)是选择的方法：CV简单地代表了每个组均以其组均值标准化

The coefficient of variation (CV), also known as relative standard deviation (RSD), is a standardized measure of the dispersion of a probability distribution or frequency distribution. It helps us in understanding how the spread is the data in two different tests

变异系数(CV)，也称为相对标准偏差(RSD)，是概率分布或频率分布的离散度的标准化度量。 它有助于我们了解两种不同测试中数据的传播方式

Standard deviation is the most common measure of variability for a single data set. But why do we need yet another measure, such as the coefficient of variation? Well, comparing the standard deviations of two different data sets is meaningless, but comparing coefficients of variation is not.

标准差是单个数据集变异性的最常见度量。 但是，为什么我们还需要另一种测量方法，例如变异系数？ 好吧，比较两个不同数据集的标准偏差是没有意义的，但是比较变异系数不是。

from scipy.stats import variationvariation(data)

For example, If we consider two different data;

例如，如果我们考虑两个不同的数据；

Data 1: Mean1 = 120000 : SD1 = 2000

数据1：平均值1 = 120000：SD1 = 2000

Data 2: Mean2 = 900000 : SD2 = 10000

数据2：Mean2 = 900000：SD2 = 10000

Let us calculate CV for both datasets

让我们计算两个数据集的简历

CV1 = SD1/Mean1 = 1.6%

CV1 = SD1 /平均值1 = 1.6％

CV2 = SD2/Mean2 = 1.1%

CV2 = SD2 /平均值2 = 1.1％

We can conclude Data 1 is more spread out than Data 2

我们可以得出结论，数据1比数据2更分散

相对平方误差(RSE) (Relative Squared Error (RSE))

The relative squared error (RSE) is relative to what it would have been if a simple predictor had been used. More specifically, this simple predictor is just the average of the actual values. Thus, the relative squared error takes the total squared error and normalizes it by dividing by the total squared error of the simple predictor. It can be compared between models whose errors are measured in the different units.

相对平方误差(RSE)与使用简单的预测变量时的误差有关。 更具体地说，这个简单的预测变量只是实际值的平均值。 因此，相对平方误差取总平方误差并通过除以简单预测变量的总平方误差对其进行归一化。 可以在以不同单位计量误差的模型之间进行比较。

Mathematically, the relative squared error, Ei of an individual model i is evaluated by the equation:

在数学上，单个模型i的相对平方误差Ei由以下公式估算：

where P(ij) is the value predicted by the individual model i for record j (out of n records); Tj is the target value for record j, and Tbar is given by the formula:

其中P ( ij )是单个模型i对记录j (在n条记录中)预测的值； Tj是记录j的目标值， Tbar由以下公式给出：

For a perfect fit, the numerator is equal to 0 and Ei = 0. So, the Ei index ranges from 0 to infinity, with 0 correspondings to the ideal.

为了完美拟合，分子等于0且Ei =0。因此， Ei索引的范围是0到无穷大，其中0对应于理想值。

平均绝对误差(MAE) (Mean Absolute Error (MAE))

In statistics, mean absolute error (MAE) is a measure of errors between paired observations expressing the same phenomenon. Examples of Y versus X include comparisons of predicted versus observed, subsequent time versus initial time, and one technique of measurement versus an alternative technique of measurement. It has the same unit as the original data, and it can only be compared between models whose errors are measured in the same units. It is usually similar in magnitude to RMSE, but slightly smaller. MAE is calculated as:

在统计中，平均绝对误差(MAE)是表示同一现象的成对观测值之间误差的度量。 Y与X的示例包括比较预测值与观察值，后续时间与初始时间，一种测量技术与另一种测量技术的比较。 它具有与原始数据相同的单位，并且只能在以相同单位测量误差的模型之间进行比较。 它的大小通常与RMSE相似，但略小。 MAE的计算公式为：

from sklearn.metrics import mean_absolute_errormean_absolute_error(actual, predicted)

It is thus an arithmetic average of the absolute errors, where yi is the prediction and xi the actual value. Note that alternative formulations may include relative frequencies as weight factors. The mean absolute error uses the same scale as the data being measured. This is known as a scale-dependent accuracy measure and, therefore cannot be used to make comparisons between series using different scales.

因此，它是绝对误差的算术平均值，其中yi是预测值，xi是实际值。 注意，替代公式可以包括相对频率作为权重因子。 平均绝对误差使用与被测数据相同的标度。 这被称为与比例有关的精度量度，因此不能用于在使用不同比例的系列之间进行比较。

Note: As you see, all the statistics compare true values to their estimates, but do it in a slightly different way. They all tell you “how far away” are your estimated values from the true value. Sometimes square roots are used and occasionally absolute values — this is because when using square roots, the extreme values have more influence on the result (see Why to square the difference instead of taking the absolute value in standard deviation? or on Mathoverflow).

注意：如您所见，所有统计信息都将真实值与其估计值进行比较，但是以略有不同的方式进行。 它们都告诉您估计值与真实值有多远。 有时使用平方根，有时使用绝对值-这是因为使用平方根时，极值会对结果产生更大的影响(请参阅 为什么对差值求平方而不是将绝对值 求 标准偏差？ 或对 Mathoverflow )。

In MAE and RMSE, you simply look at the “average difference” between those two values. So you interpret them comparing to the scale of your variable (i.e., MSE of 1 point is a difference of 1 point of actual between predicted and actual).

在MAE和RMSE中，您只需查看这两个值之间的“平均差”即可。 因此，您可以将它们与变量的范围进行比较来进行解释(即，MSE为1分与预测值和实际值之差为1分)。

In RAE and Relative RSE, you divide those differences by the variation of actual, so they have a scale from 0 to 1, and if you multiply this value by 100, you get similarity in 0–100 scale (i.e. percentage).

在RAE和相对RSE中，您将这些差异除以实际的差异，因此它们的标度从0到1，如果将此值乘以100，您将获得0-100的相似度(即百分比)。

The values of ∑(MeanofActual — actual)² or ∑|MeanofActual — actual| tell you how much actual differs from its mean value — so you could tell what it is about how much actual differs from itself (compare to variance). Because of that, the measures are named “relative” — they give you results related to the scale of actual.

∑(MeanofActual —实际)²或∑ | MeanofActual —实际|的值。 告诉您实际值与平均值之间有多少不同-因此您可以知道实际值与自身有多少不同(比较 方差 )。 因此，这些度量被称为“相对” —它们为您提供与实际规模有关的结果。

相对绝对误差(RAE) (Relative Absolute Error (RAE))

Relative Absolute Error (RAE) is a way to measure the performance of a predictive model. RAE is not to be confused with relative error, which is a general measure of precision or accuracy for instruments like clocks, rulers, or scales. It is expressed as a ratio, comparing a mean error (residual) to errors produced by a trivial or naive model. A good forecasting model will produce a ratio close to zero; A poor model (one that’s worse than the naive model) will produce a ratio greater than one.

相对绝对误差(RAE)是一种衡量预测模型性能的方法。 RAE不应与相对误差相混淆，相对误差是相对精度的一种通用度量，用于测量钟表，标尺或刻度尺等仪器。 它表示为比率，将平均误差(残差)与普通模型或幼稚模型产生的误差进行比较。 一个好的预测模型将产生接近于零的比率。 较差的模型(比原始模型差的模型)将产生大于1的比率。

It is very similar to the relative squared error in the sense that it is also relative to a simple predictor, which is just the average of the actual values. In this case, though, the error is just the total absolute error instead of the total squared error. Thus, the relative absolute error takes the total absolute error and normalizes it by dividing by the total absolute error of the simple predictor.

它与相对平方误差非常相似，因为它也与简单的预测变量有关，后者只是实际值的平均值。 但是，在这种情况下，误差只是总的绝对误差，而不是总的平方误差。 因此，相对绝对误差取总绝对误差并通过除以简单预测变量的总绝对误差对其进行归一化。

Mathematically, the relative absolute error, Ei of an individual model i is evaluated by the equation:

在数学上，单个模型i的相对绝对误差Ei由以下公式估算：

where P(ij) is the value predicted by the individual model i for record j (out of n records); Tj is the target value for record j, and Tbar is given by the formula:

其中P ( ij )是单个模型i对记录j (在n条记录中)预测的值； Tj是记录j的目标值， Tbar由以下公式给出：

For a perfect fit, the numerator is equal to 0 and Ei = 0. So, the Ei index ranges from 0 to infinity, with 0 correspondings to the ideal.

对于完美拟合，分子等于0且Ei =0。因此， Ei索引的范围是0到无穷大，其中0对应于理想值。

均方误差(MSE) (Mean Squared Error (MSE))

Mean Squared Error (MSE) or Mean Squared Deviation (MSD) of an estimator (of a procedure for estimating an unobserved quantity) measures the average of the squares of the errors — that is, the average squared difference between the estimated values and the actual value. MSE is a risk function, corresponding to the expected value of the squared error loss. The fact that MSE is almost always strictly positive (and not zero) is because of randomness or because the estimator does not account for information that could produce a more accurate estimate.

估计器的均方误差(MSE)或均方偏差(MSD)(用于估计未观察到的量的过程)测量误差平方的平均值-即，估算值与实际值之间的均方差值。 MSE是一个风险函数，对应于平方误差损失的期望值。 MSE几乎始终严格为正数(而不是零)的事实是由于随机性，或者是因为估算器没有考虑可能产生更准确估算的信息。

The MSE assesses the quality of a predictor (i.e., a function mapping arbitrary inputs to a sample of values of some random variable), or an estimator (i.e., a mathematical function mapping a sample of data to an estimate of a parameter of the population from which the data is sampled). The definition of an MSE differs according to whether one is describing a predictor or an estimator.

MSE评估预测器(即，将任意输入映射到某个随机变量的值的样本的函数)或估计器(即，将数据的样本映射到总体参数的估计的数学函数)的质量从中采样数据)。 MSE的定义因描述的是预测变量还是估计变量而异。

The MSE is a measure of the quality of an estimator — it is always non-negative, and values closer to zero are better.

MSE是对估算器质量的度量，它始终是非负的，并且接近零的值更好。

from sklearn.metrics import mean_squared_errormean_squared_error(actual, predicted)

Let’s analyze what this equation actually means.

让我们分析一下这个方程的实际含义。

• In mathematics, the character that looks like weird E is called summation (Greek sigma). It is the sum of a sequence of numbers, from i=1 to n. Let’s imagine this like an array of points, where we go through all the points, from the first (i=1) to the last (i=n).在数学中，看起来像奇怪的E的字符称为求和(Greek sigma)。 它是从i = 1到n的数字序列的总和。 让我们想象这就像一个点数组，我们从头一个点(i = 1)到最后一个点(i = n)遍历所有点。
• For each point, we take the y-coordinate of the point, and the y’-coordinate. We subtract the y-coordinate value from the y’-coordinate value and calculate the square of the result.对于每个点，我们采用该点的y坐标和y'坐标。 我们从y'坐标值中减去y坐标值，然后计算结果的平方。
• The third part is to take the sum of all the (y-y’)² values and divide it by n, which will give the mean.第三部分是取所有(y-y')²值的总和，然后除以n，得到平均值。

Our goal is to minimize this mean, which will provide us with the best line that goes through all the points. For more information.

我们的目标是尽量减少这种均值，这将为我们提供贯穿所有要点的最佳路线。 有关更多信息 。

预测的均方根误差(RMSE / RMSEP) (Root Mean Squared Error on Prediction (RMSE / RMSEP))

In statistical modeling and particularly regression analyses, a common way of measuring the quality of the fit of the model is the RMSE (also called Root Mean Square Deviation), given by

在统计建模(尤其是回归分析)中，衡量模型拟合质量的常用方法是RMSE(也称为均方根偏差)，由

from sklearn.metrics import mean_squared_errormse = mean_squared_error(actual, predicted)rmse = sqrt(mse)

where yi is the ith observation of y and ŷ the predicted y value given the model. If the predicted responses are very close to the true responses the RMSE will be small. If the predicted and true responses differ substantially — at least for some observations — the RMSE will be large. A value of zero would indicate a perfect fit to the data. Since the RMSE is measured on the same scale, with the same units as y, one can expect 68% of the y values to be within 1 RMSE — given the data is normally distributed.

其中yi是y的第i个观测值， ŷ是给定模型的y预测值。 如果预测的响应非常接近真实响应，则RMSE将很小。 如果预测的响应与真实的响应大不相同(至少对于某些观察而言)，则RMSE将很大。 零值表示对数据的完美拟合。 由于RMSE是在与y相同的比例尺和相同的单位下进行测量的，因此，假设数据是正态分布的，则可以预期y值的68％在1 RMSE之内。

NOTE: RMSE is concerned with deviations from the true value whereas S is concerned with deviations from the mean.

注意：RMSE与实际值的偏差有关，而S与平均值的偏差有关。

So calculating the MSE helps to compare different models that are based on the same y observations. But what if

因此，计算MSE有助于比较基于相同y观测值的不同模型。 但是如果

1. one wants to compare model fits of different response variables?是否想比较不同响应变量的模型拟合？

2. the response variable y is modified in some models, e.g. standardized or sqrt- or log-transformed?

   响应变量y是否在某些模型中进行了修改，例如标准化或sqrt转换或对数转换？

3. And does the splitting of data into a training and test dataset (after the modification) and the RMSE calculation based on the test data an effect on point 1. and 2.?并将数据拆分为训练和测试数据集(修改后)以及基于测试数据的RMSE计算是否会对点1和点2产生影响？

The first two points are typical issues when comparing ecological indicator performances and the latter, so-called validation set approach, is pretty common in statistical and machine learning. One solution to overcome these barriers is to calculate the Normalized RMSE.

前两点是比较生态指标性能时的典型问题，而后者，即所谓的验证集方法 ，在统计和机器学习中非常普遍。 克服这些障碍的一种方法是计算归一化的 RMSE。

归一化均方根误差(范数RMSEP) (Normalized Root Mean Squared Error (Norm RMSEP))

Normalizing the RMSE facilitates the comparison between datasets or models with different scales. You will find, however, various different methods of RMSE normalizations in the literature:

标准化RMSE有助于比较不同比例的数据集或模型。 但是，您会在文献中找到各种不同的RMSE归一化方法：

You can normalize by

您可以通过

If the response variables have few extreme values, choosing the interquartile range is a good option as it is less sensitive to outliers.

如果响应变量的极值很少，则选择四分位数范围是一个不错的选择，因为它对异常值不太敏感。

RMSEP/standard deviation is called Relative Root Mean Squared Error (RRMSEP)

RMSEP /标准偏差称为相对均方根误差(RRMSEP)

1/RRMSEP is also a metric. A value greater than 2 is considered to be a good.

1 / RRMSEP也是一个度量。 大于2的值被认为是好的。

There are also terms like, Standard Error of Prediction(SEP) and Ratio of the Standard Error of Prediction to Standard Deviation (RPD) which are mainly used in chemometrics.

还有一些术语，例如预测标准误差(SEP)和预测标准误差与标准偏差(RPD)的比率，它们主要用于化学计量学。

I hope this blog helped you to understand different metrics to evaluate your regression model. I have used multiple sources to understand and write this article. Thank you for your time.

我希望这个博客可以帮助您了解不同的指标，以评估您的回归模型。 我使用了多种资源来理解和撰写本文。 感谢您的时间。