In the previous article, we created a logistic regression model to predict user enrollment using app behavior data. Hopefully, you had good learning there. This post aims to improve your model building skills with new techniques and tricks based on a larger mobile app behavior data. It is split into 7 parts.

在上一篇文章中 ，我们创建了一个逻辑回归模型，以使用应用程序行为数据来预测用户注册。 希望您在那里学得很好。 这篇文章旨在通过基于更大的移动应用行为数据的新技术和技巧来提高您的模型构建技能 。 它分为7部分。

1. Business challenge

1.业务挑战

2. Data processing

2.数据处理

3. Model building

3.模型制作

4. Model validation

4.模型验证

5. Feature analysis

5.特征分析

6. Feature selection

6.功能选择

7. Conclusion

7.结论

Now let’s begin the journey.

现在开始旅程。

1. Business challenge

   业务挑战

We are tasked by a Fintech firm to analyze mobile app behavior data to identify potential churn customers. The goal is to predict which users are likely to churn, so the firm can focus on re-engaging these users with better products.

金融科技公司的任务是分析移动应用行为数据，以识别潜在的客户流失。 目标是预测哪些用户可能流失，以便公司可以专注于通过更好的产品重新吸引这些用户。

2. Data processing

2. 数据处理

EDA should be performed before data processing. Detailed steps are introduced in this article. The video below shows the final data after EDA.

EDA应该在数据处理之前执行。 本文介绍了详细步骤。 以下视频显示了EDA之后的最终数据。

2.1 One-hot encoding

2.1一键编码

One-hot encoding is a technique to convert categorical variables into numerical variables. It is needed as the model we are to build cannot read categorical data. One-hot encoding simply creates additional features based on the number of unique categories. Here, specifically,

一键式编码是一种将分类变量转换为数值变量的技术。 之所以需要它，是因为我们要建立的模型无法读取分类数据。 一键式编码仅根据唯一类别的数量创建其他功能。 具体来说，

dataset = pd.get_dummies(dataset)

Above automatically convert all categorical variables to numerical variables. But one drawback of one-hot encoding is the dummy variable trap. It is a scenario in which variables are highly correlated to each other. To avoid the trap, one of the dummy variables has to be dropped. Specifically,

以上自动将所有类别变量转换为数值变量。 但是，一键编码的一个缺点是伪变量陷阱 。 在这种情况下，变量之间高度相关。 为了避免陷阱，必须删除其中一个虚拟变量。 特别，

dataset = dataset.drop(columns = [‘housing_na’, ‘zodiac_sign_na’, ‘payment_type_na’])

2.2 Data split

2.2数据分割

This is to split the data into train and test sets. Specifically,

这是将数据分为训练集和测试集。 特别，

X_train, X_test, y_train, y_test = train_test_split(dataset.drop(columns = ‘churn’), dataset[‘churn’], test_size = 0.2,random_state = 0)

2.3 Data balancing

2.3数据平衡

类别不平衡是分类中的常见问题，其中每个类别中观察值的比例不成比例。 但是，如果我们在不平衡的数据集上训练模型会怎样？ 该模型将明智地决定最好的事情是始终预测1类，因为1类占用了90％的数据，因此该模型可以达到90％的准确性。 (Imbalanced classes are a common problem in a classification where the disproportionate ratio of observations in each class occurs. But what would happen if we train a model on an imbalanced dataset? The model would cleverly decide the best thing is to always predict class 1, because class 1 takes 90% of the data, so the model can achieve 90% accuracy.)

There are many ways to combat imbalanced classes, such as changing performance metrics, collecting more data, over-sampling or down-sampling data, etc. Here we use the down-sampling method.

这里有许多解决不平衡类的方法，例如更改性能指标，收集更多数据，过采样或下采样数据等。在这里，我们使用下采样方法。

First, let’s investigate the imbalance level of the dependent variable in y_train.

首先，让我们研究y_train中因变量的不平衡水平。

As shown in Fig.1, the dependent variable is slightly imbalanced. To down-sample the data, we take the index of each class, and randomly choose the index of the majority class at a number of minority class in y_train. Then concatenate the index of both classes and down-sample x_train and y_train.

如图1所示，因变量略有失衡。 为了对数据进行降采样，我们获取每个类别的索引，并在y_train中的少数少数类别中随机选择多数类别的索引。 然后连接两个类的索引以及下采样x_train和y_train 。

pos_index = y_train[y_train.values == 1].indexneg_index = y_train[y_train.values == 0].indexif len(pos_index) > len(neg_index):    higher = pos_index    lower = neg_indexelse:    higher = neg_index    lower = pos_indexrandom.seed(0)higher = np.random.choice(higher, size=len(lower))lower = np.asarray(lower)new_indexes = np.concatenate((lower, higher))X_train = X_train.loc[new_indexes,]y_train = y_train[new_indexes]

2.4. Feature scaling

2.4。 功能缩放

Fundamentally, feature scaling is to normalize the range of the variables. This is to avoid any variable having a dominant impact on the model. For a neural network, feature scaling helps gradient descent converge faster than without it.

从根本上说，特征缩放是为了规范变量的范围。 这是为了避免任何变量对模型产生主要影响。 对于神经网络，特征缩放有助于梯度下降收敛，而没有梯度下降则更快。

Here we use standardization to normalize the variables. Specifically,

这里我们使用的标准化规范化的变量。 特别，

from sklearn.preprocessing import StandardScalersc_X = StandardScaler()X_train2 = pd.DataFrame(sc_X.fit_transform(X_train))X_test2 = pd.DataFrame(sc_X.transform(X_test))

3. Model building

3. 建立模型

在这里，我们建立了流失预测的逻辑回归分类器。 本质上，逻辑回归使用独立变量的线性组合来预测一类概率的对数。 如果您想了解有关逻辑回归的更多详细信息，请访问此Wikipedia页面。 (Here we build a logistic regression classifier for churn prediction. Essentially, logistic regression predicts the logarithm of the probability of a class using a linear combination of independent variables. If you like to dive into more details on logistic regression, visit this Wikipedia page.)

Specifically,

特别，

from sklearn.linear_model import LogisticRegressionclassifier = LogisticRegression(random_state = 0)classifier.fit(X_train, y_train)

Now, let’s test and evaluate the model. Specifically,

现在，让我们测试和评估模型。 特别，

y_pred = classifier.predict(X_test)from sklearn.metrics import confusion_matrix, accuracy_score, f1_scorecm = confusion_matrix(y_test, y_pred)accuracy_score(y_test, y_pred)f1_score(y_test, y_pred)

Finally, we got an accuracy of 0.61 and F1 of 0. 61. Not too bad performance.

最终，我们得到0.61的精度和0的F1。61 。 性能还不错。

4. Model validation

4. 模型验证

With the model trained and tested, one question is how good the model is to generalize to an unknown dataset. We use cross-validation to measure the size of the performance difference between known datasets and unknown datasets. Specifically,

在对模型进行训练和测试之后，一个问题是该模型将泛化到未知数据集的质量如何。 我们使用交叉验证来衡量已知数据集和未知数据集之间的性能差异的大小。 特别，

from sklearn.model_selection import cross_val_scoreaccuracies = cross_val_score(estimator = classifier, X = X_train, y = y_train, cv = 10)

With the above, we found 10-fold cross-validation produces an average accuracy of 0.64.5 with a standard deviation of 0.023. This indicates the model can generalize well on an unknown dataset.

综上所述，我们发现10倍交叉验证产生的平均准确度为0.64.5 ，标准偏差为0.023 。 这表明该模型可以很好地概括未知数据集。

5. Feature analysis

5. 特征分析

With 41 features, we built a logistic regression model. But how to know which feature is more important in predicting the dependent variable? Specifically,

具有41个功能，我们建立了逻辑回归模型。 但是如何知道哪个特征在预测因变量中更重要？ 特别，

pd.concat([pd.DataFrame(X_train.columns, columns = [“features”]), pd.DataFrame(np.transpose(classifier.coef_), columns = [“coef”])],axis = 1)

As shown in Figure 2, we found two features that are very important: purchase_partners and purchase. This indicates a user’s purchase history plays a great role when deciding churn or not. Meanwhile, this occurs to you that not all variables are relevant for prediction.

如图2所示，我们发现了两个非常重要的功能： purchase_partners和purchase 。 这表明在决定是否流失时，用户的购买历史起着重要作用。 同时，您会想到并非所有变量都与预测相关。

6. Feature selection

6. 功能选择

Feature selection is a technique to select a subset of the most relevant features for modeling training.

特征选择是一种选择最相关特征的子集进行建模训练的技术。

In this application, x_train contains 41 features, but as seen in Figure 2, not all features play important roles. Using feature selection helps to reduce the number of unimportant features and achieve similar performance with less training data. A more detailed explanation of feature selection can be found here.

在此应用程序中， x_train包含41个功能，但是如图2所示，并非所有功能都起着重要的作用。 使用特征选择有助于减少不重要特征的数量，并以较少的训练数据获得相似的性能 。 有关功能选择的详细说明，请参见此处 。

Here, we use the Recursive Feature Elimination (RFE). It works by fitting the given algorithm, ranking the feature by importance, discarding the least important features, and refitting until a specified number of features is achieved. Specifically,

在这里，我们使用递归特征消除 (RFE)。 它通过拟合给定算法，按重要性对特征进行排序，丢弃最不重要的特征并重新拟合直到达到指定数量的特征来工作。 特别，

from sklearn.feature_selection import RFEfrom sklearn.linear_model import LogisticRegressionclassifier = LogisticRegression()rfe = RFE(classifier, 20)rfe = rfe.fit(X_train, y_train)

Note above, we set to select 20 features. Figure 3 shows all selected features.

请注意，我们设置为选择20个功能 。 图3显示了所有选定的功能。

Great. with the RFE selected features, let’s retrain and test the model.

大。 使用RFE选定的功能，让我们重新训练和测试模型。

classifier.fit(X_train[X_train.columns[rfe.support_]], y_train)y_pred = classifier.predict(X_test[X_train.columns[rfe.support_]])

In the end, we got an accuracy of 0.61 and F1 of 0.61. The same performance as the model trained on 41 features!

最后，我们得到了0.61的0.61的精度和F1。 与使用41个功能训练的模型具有相同的性能！

If we apply cross-validation again, we got an average accuracy of 0.647 with a standard deviation of 0.014. Again, very much the same as the previous model.

如果再次应用交叉验证，则平均精度为0.647 ，标准偏差为0.014 。 同样，与以前的模型非常相似。

7. Conclusion

7. 结论

Initially, we trained a logistic regression model with 41 features, achieving an accuracy of 0.645. But using feature selection to reduce features, we created a light version of the model, with an accuracy of 0.647. We found that half of the features are of no relevance in deciding the customer’s churn. Great. Well done!

最初，我们训练了具有41个特征的逻辑回归模型，实现了0.645的准确性。 但是使用特征选择来减少特征，我们创建了模型的精简版，精度为0.647。 我们发现，一半功能与决定客户流失无关。 大。 做得好！

Huge congratulations for making it to the end. If you need the source code, feel free to visit my Github page.

巨大的祝贺，使它走到了尽头。 如果您需要源代码，请随时访问我的Github页面。