So yesterday I launched a new package for python: asgl (the name comes from Adaptive Sparse Group Lasso) that adds a lot of features that were already available in R packages but not in python, like solving sparse group lasso models, and goes beyond that, adding extra features that improve the results that sparse group lasso can provide.

所以昨天我为python启动了一个新软件包： asgl (名称来自Adaptive Sparse Group Lasso)，它添加了许多R包中已经可用但python中没有的功能，例如解决稀疏组lasso模型，并且超出了此范围，添加了额外的功能来改善稀疏组的结果套索可以提供。

This is going to be the first article on a series about state-of-the-art regularization techniques in regression, and I would like to start talking about the sparse group lasso: what is it and how to use it. Specifically, here we will see:

这将是有关回归的最新正则化技术系列的第一篇文章，我想开始谈论稀疏组套索：它是什么以及如何使用它。 具体来说，在这里我们将看到：

• What is sparse group lasso

  什么是稀疏组套索

• How to use sparse group lasso in python

  如何在python中使用稀疏组套索

• How to perform k-fold cross validation

  如何执行K折交叉验证

• How to use grid search in order to find the optimal solution.

  如何使用网格搜索以找到最佳解决方案。

什么是稀疏组套索 (What is sparse group lasso)

To understand what is sparse group lasso we need to talk (briefly) about two techniques: lasso and group lasso. Given a risk function, for example the linear regression risk,

要了解什么是稀疏组套索，我们需要(简要地)谈论两种技术： 套索和套索套索 。 给定风险函数，例如线性回归风险，

Lasso: is defined by adding a penalization on the absolute value of the β coefficients,

套索：通过对β系数的绝对值加罚分来定义，

This definition provides sparse solutions, because it will send to zero some of the β coefficients (the least related with the response variable). The effect of this penalization can be controlled using the λ parameter. A large λ value provides solutions where the penalization has a greater importance, and thus there are more zeros among the β coefficients. This is mainly useful in high dimensional datasets, where there are more variables than observations but we only expect a small fragment of the variables to be truly meaningful.

该定义提供了稀疏解 ，因为它将某些β系数(与响应变量最小相关)发送到零。 惩罚的效果可以使用λ参数来控制 。 较大的λ值提供了惩罚更重要的解决方案，因此β系数中有更多零。 这主要在高维数据集中有用，在高数据集中 ，变量多于观测值，但我们只希望变量的一小部分真正有意义。

However, there are situations in which the predictor variables in X have a natural grouped structure. For example, in biostatistics it is common to deal with genetic datasets in which predictors are grouped into genetical pathways. In stock market analysis one can group companies from the same business segment. In climate data one can group different regions… And lasso provides individual sparse solutions, not group sparse.

但是，在某些情况下，X中的预测变量具有自然的分组结构。 例如，在生物统计学中，通常要处理将预测因子归为遗传途径的遗传数据集。 在股票市场分析中，可以将同一业务部门的公司分组。 在气候数据中，可以将不同的区域分组...套索提供的是稀疏解决方案，而不是稀疏解决方案。

Group lasso: So here comes group lasso to the rescue. Group lasso is built as the sum of squares of coefficients belonging to the same group.

套索组：来了 套索救援。 组套索被构建为属于同一组的系数的平方和。

This way it takes into account the possible grouped structure of predictors, and it sends to zero whole groups of variables. If all the groups are of size 1 (only one predictor per group) we will be solving a lasso model. Lets see lasso and group lasso graphically,

这样，它考虑了预测变量的可能分组结构，并将整组变量发送到零。 如果所有组的大小均为1(每组只有一个预测变量)，我们将求解套索模型。 让我们以图形方式查看套索并分组套索，

In the image above we have a simple problem with three coefficients, β₁β₁₁ and β₁₂. The last two coefficients form a group, and as we can see, lasso (left image) does not take this grouping information into account, but group lasso does. So group lasso can be seen as lasso between groups and ridge within groups. If a group is meaningful, we select the whole group. If it is not, we send it to zero.

在上图中，我们有一个简单的问题，其中三个系数分别为β₁β₁₁和β₁2。 最后两个系数组成一个组，正如我们所看到的，套索(左图)没有考虑该分组信息，但是套索却在考虑。 因此，组套索可以看作是组之间和组内脊之间的套索。 如果一个组有意义，我们选择整个组。 如果不是，我们将其发送为零。

Sparse group lasso: and finally here it is,

稀疏组套索：终于到了，

Sparse group lasso is a linear combination between lasso and group lasso, so it provides solutions that are both between and within group sparse.

稀疏组套索是套索和套索之间的线性组合，因此它提供的稀疏介于组内和组内。

This technique selects the most meaningful predictors from the most meaningful groups, and is one of the best variable selection alternatives of recent years. However, there was no implementation of sparse group lasso for python… until now.

该技术从最有意义的组中选择最有意义的预测变量，并且是近年来最好的变量选择替代方法之一。 但是，到目前为止，还没有针对python的稀疏组套索的实现。

移至python：安装asgl (Moving to python: install asgl)

Lets start by installing asgl. This can be easily done using pip

让我们从安装asgl开始。 可以使用pip轻松完成

pip install asgl

Or alternatively, one can pull the github repository and run the setup.py

或者，可以拉github存储库并运行setup.py

git clone https://github.com/alvaromc317/asgl.gitcd asglpython setup.py

导入库 (Import libraries)

Once we have the package installed, we can start using it. First lets import the libraries and the BostonHousing dataset that we will use in this example.

一旦安装了软件包，就可以开始使用它了。 首先，让我们导入在本示例中将使用的库和BostonHousing数据集。

Here, in addition to importing the dataset, we have created a variable called group_index . This variable describes the group structure of the data, so if we have 13 predictors, group_index should be a variable of length 13, and if the first three predictors form a group, they should have the same group_index value. However, BostonHousing dataset does not have a natural grouped structure, so here we define a fake one just for the sake of this article.

在这里，除了导入数据集，我们还创建了一个名为group_index的变量。 此变量描述了数据的组结构，因此，如果我们有13个预测变量，则group_index应该是长度为13的变量，并且如果前三个预测变量形成一个组，则它们应该具有相同的group_index值。 但是， BostonHousing数据集没有自然的分组结构，因此，仅出于本文的目的，在此我们定义一个伪造的结构。

sql模型的参数 (Parameters for the sgl model)

If we have a look to the sparse group lasso equation above, we can see that there are two parameters, α and λ, that can be optimized. λ controlls how much weight we want to give to the penalization, so larger λ values produce more sparse solutions. And α controls the tradeoff between lasso and group lasso. α equal to 1 provides a lasso, and α equal to 0 provides a group lasso. Now, usually, we can define a grid of possible values for both parameters and try to find the combination that minimizes the error.

如果我们看一下上面的稀疏组套索方程，我们可以看到有两个参数可以优化。 λ控制着我们希望对惩罚给予多少权重，因此较大的λ值会产生更多的稀疏解。 α控制套索和组套索之间的权衡。 等于1的α提供套索，等于0的α提供组套索。 现在，通常我们可以为两个参数定义一个可能值的网格，并尝试找到使误差最小的组合。

Additionally, we specify the type of model to solve (lm, because we are solving a linear model), the penalization (sgl, because we want the sparse group lasso), the error type to use (MSE, because we will use the mean squared error) and finally, we can indicate if we want to run this code sequentially or in parallel. Parallel execution makes use of the multiprocess python library and can solve the problem much faster than sequentially.

此外，我们指定要求解的模型的类型(lm，因为我们正在求解线性模型)，惩罚(sgl，因为我们想要稀疏组套索)，要使用的错误类型(MSE，因为我们将使用均值平方误差)，最后，我们可以指出是要顺序运行还是并行运行此代码。 并行执行利用了multiprocess 进程 python库，并且比顺序执行要快得多。

交叉验证 (Cross validation)

We have defined a grid of 23 possible λ values and 20 possible α values. A total number of 460 combinations of parameters. And we are going to find the optimal combination using cross validation (that will run in parallel)

我们定义了一个由23个可能的λ值和20个可能的α值组成的网格。 总共460个参数组合。 我们将使用交叉验证(将并行运行)找到最佳组合

So first, we define a CV object, and we plug in all the parameter information defined above. Then, we run the cross_validation() function, that will return the error that each of the 460 models achieve on each of the k folds.

因此，首先，我们定义一个CV对象，并插入上面定义的所有参数信息。 然后，我们运行cross_validation()函数，该函数将返回460个模型中的每一个在k折上实现的错误。

With that, we search for the parameter values that minimize the mean squared error, and store their values in optimal_lambda and optimal_alpha .

这样，我们将搜索使均方误差最小的参数值 ，并将其值存储在optimal_lambda和optimal_alpha 。

As simple as that, we have found our optimal model.

如此简单，我们就找到了最佳模型。

获得最终错误 (Obtain the final error)

However, one should remember that cross validation errors are a little bit biased, so in order to obtain the final prediction error we are going to run a final train / test split just using the optimal parameters,

但是，请记住，交叉验证错误有些偏颇，因此，为了获得最终的预测错误，我们将仅使用最佳参数进行最终的训练/测试拆分，

First, we define an ASGL object, that we will use for fitting a simple sparse group lasso model (no cross validation). We then split the data into 70% train 30% test using the function train_test_split() , and we fit the model so that we obtain the final estimates of the β coefficients (stored in final_beta_solution )

首先，我们定义一个ASGL对象，将其用于拟合简单的稀疏组套索模型(无交叉验证)。 然后，使用函数train_test_split()将数据拆分为70％的训练，30％的测试，并拟合模型，以便获得β系数的最终估计值(存储在final_beta_solution )

Given a new dataset, we can predict the value of the response variable using the function ASGL.predict() , and compute the final prediction error using the test set with the function error_calculator .

给定一个新的数据集，我们可以预测使用该函数的响应变量的值ASGL.predict()并使用所述测试组与该功能计算最终的预测误差error_calculator 。

And that’s it on how to implement sparse group lasso in python. I hope you enjoyed this post and found it useful, so stay tuned for future posts on this series and please do not hesitate on contacting me if you have any question / suggestion.

这就是如何在python中实现稀疏组套索。 我希望您喜欢这篇文章并觉得它有用，所以请继续关注本系列的后续文章，如果您有任何问题/建议，请随时与我联系。

For a deeper review on what the asgl package has to offer, I recommend reading the jupyter notebook provided in the github repository.

为了更深入地了解asgl软件包所提供的内容，我建议阅读github存储库中提供的jupyter笔记本 。

Have a good day!

祝你有美好的一天！