随机森林、LGBM基于贝叶斯优化调参
前言
- 本文基于孕妇吸烟与胎儿问题中数据集与前期处理
- 针对随机森林与LGBM模型网格搜索效率低,使用贝叶斯调参提高效率
- 有关于贝叶斯优化包相关参数说明详解可以看GitHub地址
- 将处理好的数据用
dill
包进行封装,大家在尝试运行时,需要安装dill
包 - 数据文件下载地址
数据导入
- 基于
jupyter notebook
的魔术命令,如果不在jupyter notebook
中运行,请将%
号去掉
# 如果已经安装过dill包可以不要下面的魔术命令
%pip install dill
import dill
# 如果执行这步出错,请检查错误信息
# 是否有包未安装,比如bayes_opt包或lightgbm包
dill.load_session('C:/Users/lenovo/Desktop/data.pkl')
# 若未报错,则数据导入成功
安装贝叶斯优化包
- 基于
jupyter notebook
的魔术命令,如果不在jupyter notebook
中运行,请将%
号去掉
%pip install bayesian-optimization
随机森林贝叶斯调参
导入包
# 贝叶斯调参优化随机森林
from sklearn.model_selection import cross_val_score
from bayes_opt import BayesianOptimization
构造黑盒函数
- 构造黑盒函数,即需要优化的目标,这里我选取的是10折交叉检验的分数值。当然你也可以在
RandomForestRegressor()
后使用.fit(train_x, train_y).score(test_x, test_y)
将模型在测试集上表现作为最优化目标 - 因为
bayes_opt
库只支持最大值,所以最后的输出如果是求最小值,那么需要在前面加上负号,以转为最大值。这里使用neg_mean_squared_error
作为最大化目标 - 由于
bayes
优化只能优化连续超参数,因此要加上int()
转为离散超参数
# 函数中包含需要调整的参数
def rf_cv(n_estimators, min_samples_split, max_features, max_depth, min_samples_leaf, max_leaf_nodes):# 交叉检验,得到的评分为贝叶斯调参优化目标val = cross_val_score(# 由于bayes优化只能优化连续超参数,因此要加上int()转为离散超参数RandomForestRegressor(n_estimators=int(n_estimators),min_samples_split=int(min_samples_split),min_samples_leaf = int(min_samples_leaf),max_leaf_nodes = int(max_leaf_nodes),max_features=int(max_features)max_depth=int(max_depth),random_state=42,oob_score = 'True'),X_train, y_train, scoring='neg_mean_squared_error', cv=10).mean()return val
确定域空间
- 确定参数搜索的范围,并打印迭代过程
# 规定各参数搜索范围
rf_bo = BayesianOptimization(rf_cv,{'n_estimators': (800, 1500),'min_samples_split': (2, 20),'max_features': (1, 6),'max_depth': (3, 10),'min_samples_leaf' : (2,20),'max_leaf_nodes':(10,40)})
rf_bo.maximize()
输出:
| iter | target | max_depth | max_fe... | max_le... | min_sa... | min_sa... | n_esti... |
-------------------------------------------------------------------------------------------------
| 1 | -246.0 | 4.341 | 3.478 | 27.05 | 2.973 | 16.09 | 882.3 |
| 2 | -264.1 | 4.023 | 1.761 | 31.94 | 19.03 | 5.529 | 1.338e+0 |
| 3 | -245.4 | 5.505 | 4.8 | 23.88 | 4.304 | 12.89 | 1.147e+0 |
| 4 | -247.3 | 9.276 | 2.907 | 33.51 | 7.78 | 6.09 | 1.492e+0 |
| 5 | -257.1 | 6.966 | 1.406 | 25.3 | 14.24 | 11.24 | 925.4 |
| 6 | -245.3 | 5.946 | 4.281 | 26.73 | 5.724 | 13.45 | 1.147e+0 |
| 7 | -250.0 | 3.0 | 6.0 | 29.3 | 2.0 | 20.0 | 826.7 |
| 8 | -272.6 | 3.0 | 1.0 | 38.17 | 20.0 | 2.0 | 1.097e+0 |
| 9 | -248.6 | 8.698 | 5.645 | 23.75 | 2.0 | 20.0 | 1.182e+0 |
| 10 | -264.9 | 10.0 | 1.0 | 10.0 | 20.0 | 2.0 | 858.3 |
| 11 | -272.4 | 3.0 | 1.0 | 17.23 | 20.0 | 2.0 | 1.165e+0 |
| 12 | -247.0 | 4.871 | 5.969 | 31.59 | 2.493 | 16.71 | 1.134e+0 |
| 13 | -253.0 | 10.0 | 6.0 | 40.0 | 2.0 | 20.0 | 893.9 |
| 14 | -247.9 | 8.938 | 5.776 | 10.13 | 3.715 | 19.55 | 1.129e+0 |
| 15 | -246.0 | 8.643 | 4.938 | 16.8 | 12.07 | 17.0 | 1.48e+03 |
| 16 | -272.3 | 3.046 | 1.13 | 30.6 | 20.0 | 4.148 | 1.47e+03 |
| 17 | -247.3 | 10.0 | 5.957 | 19.54 | 4.089 | 17.66 | 1.495e+0 |
| 18 | -248.7 | 6.204 | 3.098 | 10.2 | 18.7 | 11.91 | 1.495e+0 |
| 19 | -249.2 | 3.0 | 5.068 | 39.19 | 2.0 | 20.0 | 865.9 |
| 20 | -248.0 | 6.247 | 5.639 | 30.41 | 16.99 | 19.59 | 1.499e+0 |
| 21 | -252.9 | 10.0 | 6.0 | 40.0 | 2.0 | 20.0 | 1.161e+0 |
| 22 | -249.2 | 10.0 | 6.0 | 22.95 | 19.86 | 20.0 | 1.136e+0 |
| 23 | -251.1 | 10.0 | 6.0 | 29.33 | 2.0 | 20.0 | 1.207e+0 |
| 24 | -265.7 | 10.0 | 1.0 | 10.0 | 2.0 | 2.0 | 1.487e+0 |
| 25 | -248.4 | 9.63 | 5.778 | 22.09 | 2.579 | 2.131 | 1.132e+0 |
| 26 | -250.4 | 3.485 | 5.665 | 34.65 | 19.73 | 17.61 | 881.4 |
| 27 | -248.1 | 3.569 | 4.336 | 11.09 | 7.701 | 17.22 | 892.6 |
| 28 | -272.5 | 3.0 | 1.0 | 22.71 | 2.0 | 2.0 | 892.6 |
| 29 | -248.5 | 6.826 | 2.856 | 17.2 | 10.92 | 19.96 | 879.0 |
| 30 | -247.6 | 6.9 | 4.121 | 39.17 | 3.153 | 19.32 | 1.49e+03 |
=================================================================================================
- 可以看到当迭代到30次时,就已经找到了较为理想的参数
最优参
# 使用max得到最优参
rf_bo.max
输出:
{'target': -245.31420890294567,'params': {'max_depth': 5.946464811467048,'max_features': 4.280789636350939,'max_leaf_nodes': 26.726081689294052,'min_samples_leaf': 5.724298965332711,'min_samples_split': 13.447191575489041,'n_estimators': 1147.3188404305388}}
- 使用最优参(记得该取整的要取整)再次训练模型,得到相对理想的模型。
使用最优参训练模型
# 基于sklearn封装的随机森林
from sklearn.ensemble import RandomForestRegressor
# 在RandomForestRegressor()括号中填入最优参
# 这里以上面我得到的最优参为例
rf_reg = RandomForestRegressor(max_depth = 5,max_features = 4,max_leaf_nodes = 26,min_samples_leaf = 5,min_samples_split = 13,n_estimators = 1147)
# 在训练集上训练模型
rf_reg.fit(X_train, y_train)
# 查看模型在训练集上均方误差
print('train_mes : {:.3f}'.format(mean_squared_error(y_train, rf_reg.predict(X_train))))
LGBM贝叶斯调参
- 同样的,先构造黑箱函数,然后确定域空间,唯一不同的这次我们规定随机搜索的步数和贝叶斯优化的步数。这样会增加搜索的时间,但也许能比默认步数搜索到更好的参数
LGBM
模型中L1
、L2
正则化等参数允许为浮点类型,所以不需要使用int()
进行类型转换。
def lgbm_cv(n_estimators,max_depth,num_leaves,min_child_samples,reg_alpha,reg_lambda,subsample,colsample_bytree):val = cross_val_score(LGBMRegressor(learning_rate = 0.001,n_estimators=int(n_estimators),max_depth=int(max_depth),num_leaves = int(num_leaves),min_child_samples = int(min_child_samples),reg_alpha = reg_alpha, # floatreg_lambda = reg_lambda,subsample = subsample,colsample_bytree = colsample_bytree),X_train, y_train, scoring='neg_mean_squared_error', cv=10).mean()return vallgbm_bo = BayesianOptimization(lgbm_cv,{'n_estimators': (1500, 3000),'max_depth': (2, 10),'num_leaves': (5, 20),'min_child_samples': (3, 100),'reg_alpha' : (0.1,1),'reg_lambda':(0.001,1),'subsample':(0.8,1),'colsample_bytree':(0.8,1)})
确定迭代次数
- 迭代次数由两部分组成,随机搜索的步数和贝叶斯优化的步数,贝叶斯优化的步数要多一点,步骤越多,就越有可能找到一个好的最大值。随机探索可以通过扩大探索空间而有所帮助。
# n_iter贝叶斯优化次数,init_points随机优化次数
lgbm_bo.maximize(n_iter = 100,init_points = 50)
输出:
| iter | target | colsam... | max_depth | min_ch... | n_esti... | num_le... | reg_alpha | reg_la... | subsample |
-------------------------------------------------------------------------------------------------------------------------
| 1 | -251.5 | 0.9299 | 6.878 | 82.45 | 2.147e+0 | 11.21 | 0.3167 | 0.2107 | 0.8781 |
| 2 | -252.4 | 0.8555 | 4.824 | 58.81 | 2.055e+0 | 11.71 | 0.7608 | 0.2855 | 0.8864 |
| 3 | -249.1 | 0.96 | 7.997 | 14.63 | 2.232e+0 | 19.55 | 0.1661 | 0.6375 | 0.8603 |
| 4 | -249.4 | 0.9018 | 8.336 | 20.44 | 2.155e+0 | 18.32 | 0.6761 | 0.1784 | 0.9618 |
| 5 | -249.9 | 0.8464 | 8.894 | 76.6 | 2.907e+0 | 15.3 | 0.1962 | 0.9202 | 0.9374 |
| 6 | -256.1 | 0.9289 | 9.386 | 70.46 | 1.546e+0 | 7.014 | 0.4163 | 0.9269 | 0.8498 |
| 7 | -250.1 | 0.8198 | 5.917 | 22.09 | 1.997e+0 | 8.863 | 0.2419 | 0.08677 | 0.8001 |
| 8 | -250.1 | 0.84 | 4.637 | 25.22 | 2.121e+0 | 13.66 | 0.7642 | 0.04577 | 0.9546 |
| 9 | -248.0 | 0.8454 | 8.498 | 15.58 | 2.894e+0 | 18.63 | 0.1802 | 0.4565 | 0.8858 |
| 10 | -255.2 | 0.8839 | 7.783 | 53.08 | 1.979e+0 | 5.731 | 0.4185 | 0.5306 | 0.9554 |
| 11 | -249.2 | 0.944 | 4.216 | 88.81 | 2.929e+0 | 10.36 | 0.5145 | 0.1131 | 0.9041 |
| 12 | -247.2 | 0.8847 | 5.88 | 12.0 | 2.059e+0 | 13.3 | 0.6635 | 0.09008 | 0.9531 |
| 13 | -250.2 | 0.8651 | 3.733 | 50.73 | 2.875e+0 | 16.17 | 0.542 | 0.847 | 0.9391 |
| 14 | -251.2 | 0.8654 | 8.812 | 92.12 | 2.522e+0 | 12.46 | 0.3778 | 0.6421 | 0.9782 |
| 15 | -251.3 | 0.9663 | 6.259 | 45.49 | 2.908e+0 | 17.26 | 0.3127 | 0.04984 | 0.9103 |
| 16 | -248.5 | 0.8887 | 6.157 | 79.92 | 2.83e+03 | 7.47 | 0.7573 | 0.1109 | 0.8294 |
| 17 | -254.0 | 0.9811 | 4.113 | 83.44 | 1.704e+0 | 8.561 | 0.787 | 0.1622 | 0.9373 |
| 18 | -250.7 | 0.9525 | 6.545 | 94.2 | 2.758e+0 | 8.907 | 0.6938 | 0.3916 | 0.9476 |
| 19 | -251.0 | 0.9972 | 6.941 | 46.37 | 2.577e+0 | 12.63 | 0.7774 | 0.9983 | 0.8211 |
| 20 | -246.4 | 0.8041 | 3.084 | 3.125 | 2.505e+0 | 17.59 | 0.4361 | 0.7433 | 0.9699 |
| 21 | -249.1 | 0.8621 | 8.701 | 65.99 | 2.938e+0 | 19.25 | 0.9019 | 0.2082 | 0.8993 |
| 22 | -249.5 | 0.9085 | 3.629 | 23.46 | 2.138e+0 | 12.78 | 0.99 | 0.6435 | 0.9545 |
| 23 | -259.9 | 0.8006 | 2.359 | 33.99 | 1.649e+0 | 16.87 | 0.9538 | 0.7214 | 0.8083 |
| 24 | -253.8 | 0.9543 | 9.966 | 76.27 | 1.817e+0 | 15.48 | 0.953 | 0.4465 | 0.8373 |
| 25 | -250.0 | 0.9024 | 9.932 | 87.61 | 2.563e+0 | 17.52 | 0.6365 | 0.3728 | 0.9358 |
| 26 | -247.7 | 0.9368 | 9.499 | 26.12 | 2.618e+0 | 6.258 | 0.5139 | 0.01136 | 0.8822 |
| 27 | -256.5 | 0.8403 | 2.212 | 72.82 | 1.975e+0 | 9.344 | 0.7336 | 0.9867 | 0.986 |
| 28 | -250.4 | 0.8356 | 6.36 | 25.71 | 2.362e+0 | 12.92 | 0.9792 | 0.1116 | 0.9806 |
| 29 | -250.4 | 0.967 | 6.722 | 72.78 | 2.598e+0 | 11.43 | 0.3715 | 0.7805 | 0.9167 |
| 30 | -253.8 | 0.9365 | 2.325 | 42.29 | 2.333e+0 | 6.774 | 0.7346 | 0.5393 | 0.9785 |
| 31 | -252.4 | 0.984 | 7.318 | 44.95 | 1.856e+0 | 11.53 | 0.364 | 0.06497 | 0.9884 |
| 32 | -255.7 | 0.895 | 4.489 | 41.63 | 1.858e+0 | 5.682 | 0.1547 | 0.7007 | 0.9882 |
| 33 | -248.9 | 0.9449 | 7.396 | 32.28 | 2.355e+0 | 14.05 | 0.8676 | 0.9371 | 0.8904 |
| 34 | -251.9 | 0.8728 | 7.131 | 58.31 | 2.399e+0 | 13.46 | 0.8022 | 0.9378 | 0.9732 |
| 35 | -252.0 | 0.859 | 5.741 | 43.33 | 1.914e+0 | 11.67 | 0.4837 | 0.1606 | 0.9066 |
| 36 | -256.2 | 0.9248 | 3.282 | 44.33 | 1.549e+0 | 17.83 | 0.2039 | 0.1553 | 0.8587 |
| 37 | -250.0 | 0.8103 | 5.894 | 43.89 | 2.872e+0 | 11.2 | 0.9715 | 0.1911 | 0.8768 |
| 38 | -247.4 | 0.9543 | 4.029 | 29.43 | 2.807e+0 | 18.67 | 0.6966 | 0.3294 | 0.8734 |
| 39 | -251.7 | 0.8035 | 9.248 | 49.61 | 2.935e+0 | 19.69 | 0.4769 | 0.7963 | 0.8527 |
| 40 | -253.8 | 0.8426 | 7.232 | 73.47 | 1.953e+0 | 9.829 | 0.4052 | 0.6666 | 0.8147 |
| 41 | -253.7 | 0.8878 | 9.484 | 77.37 | 1.981e+0 | 10.31 | 0.9634 | 0.37 | 0.8724 |
| 42 | -248.7 | 0.837 | 5.001 | 5.432 | 1.895e+0 | 10.73 | 0.4941 | 0.07088 | 0.9292 |
| 43 | -252.1 | 0.8235 | 2.448 | 9.134 | 2.106e+0 | 16.9 | 0.8737 | 0.3368 | 0.8999 |
| 44 | -252.3 | 0.927 | 7.326 | 91.65 | 2.058e+0 | 11.56 | 0.7419 | 0.4703 | 0.8103 |
| 45 | -250.4 | 0.9256 | 7.025 | 64.44 | 2.401e+0 | 16.9 | 0.107 | 0.7307 | 0.8434 |
| 46 | -248.8 | 0.9423 | 3.798 | 29.89 | 2.524e+0 | 12.89 | 0.7705 | 0.741 | 0.9162 |
| 47 | -258.6 | 0.9142 | 5.767 | 76.21 | 1.505e+0 | 6.561 | 0.2224 | 0.2147 | 0.9764 |
| 48 | -252.4 | 0.9271 | 8.811 | 6.559 | 1.796e+0 | 5.845 | 0.542 | 0.8878 | 0.9614 |
| 49 | -258.8 | 0.8216 | 7.332 | 98.04 | 1.61e+03 | 5.039 | 0.4765 | 0.09902 | 0.8323 |
| 50 | -251.2 | 0.8809 | 3.795 | 97.97 | 2.588e+0 | 5.977 | 0.9903 | 0.1333 | 0.8626 |
| 51 | -252.7 | 0.8713 | 3.811 | 55.85 | 2.138e+0 | 17.5 | 0.527 | 0.4825 | 0.8298 |
| 52 | -255.0 | 0.8587 | 9.695 | 56.51 | 1.657e+0 | 14.53 | 0.6904 | 0.6378 | 0.9473 |
| 53 | -254.0 | 0.971 | 2.733 | 91.54 | 2.319e+0 | 5.299 | 0.6559 | 0.9105 | 0.9984 |
| 54 | -252.5 | 0.915 | 9.869 | 92.64 | 2.222e+0 | 10.35 | 0.9431 | 0.4694 | 0.821 |
| 55 | -251.9 | 0.9004 | 7.716 | 73.5 | 2.288e+0 | 11.8 | 0.4452 | 0.4667 | 0.9693 |
| 56 | -251.1 | 0.9678 | 4.192 | 96.36 | 2.536e+0 | 14.14 | 0.1397 | 0.04324 | 0.8681 |
| 57 | -253.8 | 0.8579 | 4.479 | 72.29 | 1.916e+0 | 17.29 | 0.8099 | 0.4709 | 0.9209 |
| 58 | -251.7 | 0.8303 | 5.407 | 92.42 | 2.351e+0 | 9.32 | 0.1488 | 0.5252 | 0.8027 |
| 59 | -250.2 | 0.8626 | 9.889 | 37.49 | 2.543e+0 | 11.68 | 0.3817 | 0.1709 | 0.8627 |
| 60 | -250.7 | 0.9972 | 8.553 | 32.94 | 1.672e+0 | 15.26 | 0.3957 | 0.1086 | 0.8383 |
| 61 | -251.9 | 0.8949 | 5.338 | 26.2 | 1.942e+0 | 7.799 | 0.8794 | 0.1337 | 0.8964 |
| 62 | -246.9 | 0.8566 | 3.851 | 16.47 | 2.076e+0 | 18.73 | 0.5352 | 0.4207 | 0.9956 |
| 63 | -245.1 | 0.9937 | 4.547 | 3.305 | 2.867e+0 | 7.185 | 0.8233 | 0.4205 | 0.9417 |
| 64 | -250.5 | 0.9768 | 8.349 | 85.95 | 2.332e+0 | 14.78 | 0.3782 | 0.8485 | 0.9663 |
| 65 | -250.4 | 0.8401 | 9.422 | 32.77 | 2.428e+0 | 18.25 | 0.7231 | 0.7265 | 0.8794 |
| 66 | -247.4 | 0.9794 | 4.493 | 8.738 | 2.042e+0 | 16.57 | 0.6579 | 0.6725 | 0.844 |
| 67 | -248.6 | 0.946 | 4.873 | 34.24 | 2.044e+0 | 16.74 | 0.3182 | 0.2692 | 0.8474 |
| 68 | -254.3 | 0.8538 | 5.687 | 79.42 | 1.824e+0 | 18.33 | 0.7764 | 0.6676 | 0.8872 |
| 69 | -255.9 | 0.8359 | 8.635 | 76.64 | 1.688e+0 | 9.01 | 0.6837 | 0.9395 | 0.9813 |
| 70 | -252.5 | 0.9696 | 9.777 | 42.32 | 2.846e+0 | 14.44 | 0.4389 | 0.1839 | 0.9697 |
| 71 | -251.0 | 0.8293 | 7.864 | 97.97 | 2.745e+0 | 15.07 | 0.4306 | 0.3661 | 0.8145 |
| 72 | -254.7 | 0.844 | 2.044 | 82.31 | 2.124e+0 | 12.8 | 0.9643 | 0.9277 | 0.8329 |
| 73 | -248.6 | 0.9647 | 4.654 | 79.34 | 2.879e+0 | 14.25 | 0.3899 | 0.4517 | 0.8848 |
| 74 | -253.5 | 0.9759 | 9.409 | 34.56 | 1.831e+0 | 6.315 | 0.2175 | 0.9694 | 0.9046 |
| 75 | -249.8 | 0.9993 | 3.751 | 53.76 | 2.705e+0 | 18.52 | 0.9932 | 0.3771 | 0.9974 |
| 76 | -248.2 | 0.8568 | 7.918 | 26.47 | 2.832e+0 | 9.567 | 0.3081 | 0.2001 | 0.9637 |
| 77 | -250.6 | 0.9161 | 9.572 | 21.78 | 2.08e+03 | 17.8 | 0.659 | 0.2496 | 0.8105 |
| 78 | -256.7 | 0.8722 | 5.862 | 96.38 | 1.661e+0 | 17.3 | 0.8781 | 0.3713 | 0.8406 |
| 79 | -249.6 | 0.9668 | 3.886 | 67.11 | 2.591e+0 | 6.384 | 0.2177 | 0.1065 | 0.9837 |
| 80 | -249.0 | 0.8911 | 5.993 | 29.05 | 2.946e+0 | 13.13 | 0.902 | 0.1512 | 0.9985 |
| 81 | -249.2 | 0.8635 | 3.71 | 43.96 | 2.767e+0 | 18.02 | 0.9057 | 0.6716 | 0.8505 |
| 82 | -253.0 | 0.9688 | 3.429 | 79.2 | 2.066e+0 | 6.202 | 0.9512 | 0.6868 | 0.8608 |
| 83 | -250.9 | 0.952 | 4.57 | 75.44 | 2.293e+0 | 17.13 | 0.6878 | 0.9675 | 0.8407 |
| 84 | -254.5 | 0.9119 | 3.122 | 82.4 | 1.808e+0 | 12.66 | 0.9125 | 0.5803 | 0.9666 |
| 85 | -257.6 | 0.9048 | 3.93 | 95.5 | 1.624e+0 | 10.92 | 0.6581 | 0.2187 | 0.9817 |
| 86 | -257.4 | 0.8564 | 2.41 | 22.95 | 1.807e+0 | 8.449 | 0.3729 | 0.761 | 0.8282 |
| 87 | -248.5 | 0.8595 | 4.704 | 5.026 | 1.914e+0 | 17.77 | 0.5784 | 0.7281 | 0.8922 |
| 88 | -248.6 | 0.9925 | 2.062 | 13.15 | 2.681e+0 | 12.09 | 0.1668 | 0.6771 | 0.8767 |
| 89 | -247.2 | 0.9495 | 8.553 | 6.885 | 2.643e+0 | 14.56 | 0.8508 | 0.2082 | 0.9777 |
| 90 | -249.9 | 0.8889 | 5.049 | 75.7 | 2.95e+03 | 19.67 | 0.5517 | 0.6659 | 0.9488 |
| 91 | -252.1 | 0.8089 | 5.886 | 56.19 | 2.138e+0 | 7.085 | 0.1107 | 0.08426 | 0.8332 |
| 92 | -250.7 | 0.8242 | 8.274 | 80.49 | 2.32e+03 | 6.095 | 0.5902 | 0.6058 | 0.9501 |
| 93 | -257.2 | 0.9934 | 2.024 | 65.59 | 1.808e+0 | 8.934 | 0.1617 | 0.9746 | 0.8435 |
| 94 | -250.5 | 0.8985 | 2.545 | 84.7 | 2.795e+0 | 19.0 | 0.1595 | 0.3708 | 0.8454 |
| 95 | -250.3 | 0.8932 | 8.764 | 47.66 | 2.538e+0 | 19.69 | 0.5193 | 0.4982 | 0.8129 |
| 96 | -251.0 | 0.9709 | 7.095 | 48.09 | 2.861e+0 | 11.9 | 0.3469 | 0.7833 | 0.9071 |
| 97 | -252.4 | 0.9348 | 8.922 | 77.9 | 2.059e+0 | 15.69 | 0.8596 | 0.5381 | 0.9972 |
| 98 | -253.6 | 0.9101 | 2.922 | 11.67 | 1.967e+0 | 15.83 | 0.9068 | 0.7492 | 0.8623 |
| 99 | -250.2 | 0.8583 | 8.64 | 89.44 | 2.598e+0 | 18.2 | 0.1822 | 0.4276 | 0.8993 |
| 100 | -246.6 | 0.8264 | 5.264 | 10.81 | 2.47e+03 | 7.567 | 0.6361 | 0.3497 | 0.8002 |
| 101 | -250.1 | 0.8995 | 3.704 | 52.65 | 2.985e+0 | 19.72 | 0.11 | 0.1371 | 0.8817 |
| 102 | -248.8 | 0.8594 | 5.012 | 4.972 | 2.131e+0 | 8.508 | 0.3815 | 0.945 | 0.9192 |
| 103 | -247.6 | 0.9669 | 3.601 | 29.7 | 2.932e+0 | 18.49 | 0.3248 | 0.7928 | 0.8384 |
| 104 | -256.3 | 0.9432 | 9.293 | 54.1 | 1.761e+0 | 5.085 | 0.3883 | 0.5722 | 0.9298 |
| 105 | -257.2 | 0.9938 | 2.443 | 84.29 | 1.823e+0 | 18.0 | 0.3708 | 0.7673 | 0.9651 |
| 106 | -252.1 | 0.9587 | 8.62 | 42.49 | 2.116e+0 | 19.23 | 0.7527 | 0.4659 | 0.9647 |
| 107 | -253.4 | 0.8556 | 9.788 | 62.2 | 1.972e+0 | 19.38 | 0.9473 | 0.1454 | 0.8905 |
| 108 | -253.2 | 0.8099 | 7.696 | 50.18 | 2.162e+0 | 15.46 | 0.6554 | 0.827 | 0.8181 |
| 109 | -251.1 | 0.8175 | 8.761 | 13.57 | 1.69e+03 | 8.422 | 0.8813 | 0.9929 | 0.9977 |
| 110 | -251.1 | 0.8626 | 6.849 | 92.65 | 2.571e+0 | 8.158 | 0.758 | 0.8469 | 0.9977 |
| 111 | -252.6 | 0.8674 | 7.393 | 27.39 | 1.724e+0 | 18.88 | 0.7886 | 0.4438 | 0.8879 |
| 112 | -251.1 | 0.9263 | 4.146 | 78.4 | 2.257e+0 | 14.98 | 0.2221 | 0.145 | 0.9518 |
| 113 | -247.5 | 0.8534 | 6.048 | 10.58 | 2.222e+0 | 8.573 | 0.4608 | 0.3796 | 0.99 |
| 114 | -246.1 | 0.8489 | 3.597 | 6.778 | 2.394e+0 | 9.335 | 0.5718 | 0.1133 | 0.8493 |
| 115 | -250.9 | 0.9191 | 7.406 | 27.09 | 2.777e+0 | 19.52 | 0.6142 | 0.7267 | 0.8041 |
| 116 | -248.9 | 0.8929 | 7.28 | 38.2 | 2.463e+0 | 8.195 | 0.4663 | 0.3756 | 0.8455 |
| 117 | -252.0 | 0.9764 | 8.82 | 12.13 | 1.569e+0 | 7.788 | 0.7776 | 0.3066 | 0.8528 |
| 118 | -254.7 | 0.9845 | 6.606 | 88.3 | 1.692e+0 | 15.52 | 0.1237 | 0.8686 | 0.9593 |
| 119 | -249.9 | 0.9242 | 4.519 | 52.74 | 2.537e+0 | 19.79 | 0.4742 | 0.5048 | 0.9595 |
| 120 | -254.5 | 0.8294 | 4.186 | 32.16 | 1.826e+0 | 6.485 | 0.4382 | 0.6719 | 0.9082 |
| 121 | -257.4 | 0.8519 | 7.839 | 87.22 | 1.519e+0 | 13.29 | 0.3876 | 0.01006 | 0.9211 |
| 122 | -253.1 | 0.8557 | 4.863 | 99.88 | 2.196e+0 | 6.027 | 0.2864 | 0.08695 | 0.8477 |
| 123 | -255.3 | 0.9123 | 3.487 | 58.82 | 1.743e+0 | 16.01 | 0.981 | 0.2148 | 0.9191 |
| 124 | -252.9 | 0.8671 | 6.105 | 34.59 | 1.822e+0 | 7.959 | 0.5157 | 0.4601 | 0.9288 |
| 125 | -251.6 | 0.823 | 4.232 | 4.69 | 1.63e+03 | 17.58 | 0.3137 | 0.896 | 0.8394 |
| 126 | -251.1 | 0.9605 | 9.27 | 3.885 | 1.594e+0 | 9.155 | 0.3988 | 0.04495 | 0.8383 |
| 127 | -251.1 | 0.8553 | 8.43 | 92.19 | 2.549e+0 | 15.68 | 0.4312 | 0.3145 | 0.8269 |
| 128 | -253.7 | 0.8167 | 6.511 | 63.97 | 1.761e+0 | 18.79 | 0.4067 | 0.4545 | 0.8913 |
| 129 | -250.5 | 0.9509 | 3.603 | 86.18 | 2.351e+0 | 6.903 | 0.858 | 0.1375 | 0.887 |
| 130 | -251.3 | 0.8577 | 7.442 | 58.83 | 2.718e+0 | 18.86 | 0.3148 | 0.3542 | 0.8545 |
| 131 | -252.5 | 0.8522 | 5.962 | 44.5 | 1.682e+0 | 17.12 | 0.1616 | 0.242 | 0.8918 |
| 132 | -256.4 | 0.9798 | 4.611 | 95.35 | 1.608e+0 | 9.765 | 0.6692 | 0.7236 | 0.8428 |
| 133 | -251.3 | 0.9747 | 8.909 | 39.93 | 2.322e+0 | 13.09 | 0.7278 | 0.2057 | 0.9541 |
| 134 | -258.2 | 0.898 | 2.558 | 77.46 | 1.824e+0 | 17.6 | 0.184 | 0.882 | 0.8166 |
| 135 | -250.7 | 0.9765 | 8.294 | 36.04 | 1.986e+0 | 15.15 | 0.1793 | 0.0547 | 0.8536 |
| 136 | -256.1 | 0.8067 | 8.756 | 78.74 | 1.653e+0 | 9.253 | 0.4187 | 0.356 | 0.8482 |
| 137 | -255.0 | 0.9928 | 6.42 | 93.78 | 1.784e+0 | 18.29 | 0.3528 | 0.7969 | 0.9533 |
| 138 | -250.7 | 0.9377 | 9.27 | 93.17 | 2.997e+0 | 13.43 | 0.2546 | 0.3083 | 0.8694 |
| 139 | -259.6 | 0.8726 | 8.204 | 78.49 | 1.566e+0 | 5.512 | 0.6164 | 0.7091 | 0.8871 |
| 140 | -253.0 | 0.9226 | 2.858 | 49.18 | 2.565e+0 | 14.22 | 0.7601 | 0.5902 | 0.8829 |
| 141 | -255.2 | 0.8894 | 2.576 | 54.63 | 2.112e+0 | 16.73 | 0.9314 | 0.9692 | 0.9501 |
| 142 | -251.9 | 0.8662 | 2.44 | 42.66 | 2.677e+0 | 7.367 | 0.5093 | 0.6031 | 0.8568 |
| 143 | -246.4 | 0.8408 | 5.624 | 9.557 | 2.655e+0 | 17.48 | 0.6699 | 0.01308 | 0.9373 |
| 144 | -256.3 | 0.8831 | 4.472 | 72.16 | 1.638e+0 | 8.795 | 0.748 | 0.7101 | 0.8074 |
| 145 | -254.3 | 0.9842 | 5.52 | 54.41 | 1.66e+03 | 10.72 | 0.8502 | 0.954 | 0.8083 |
| 146 | -250.6 | 0.977 | 5.445 | 60.33 | 2.612e+0 | 18.05 | 0.5285 | 0.458 | 0.9856 |
| 147 | -249.9 | 0.9103 | 5.405 | 91.39 | 2.967e+0 | 12.09 | 0.6206 | 0.8223 | 0.812 |
| 148 | -252.6 | 0.9774 | 4.257 | 42.9 | 1.731e+0 | 15.42 | 0.9746 | 0.8842 | 0.8383 |
| 149 | -254.0 | 0.858 | 2.464 | 43.46 | 2.257e+0 | 13.9 | 0.3589 | 0.9985 | 0.8966 |
| 150 | -253.4 | 0.9733 | 9.152 | 55.79 | 2.118e+0 | 5.46 | 0.8561 | 0.062 | 0.8039 |
输出最优参
lgbm_bo.max
输出:
{'target': -243.40837561015314,'params': {'colsample_bytree': 0.9035439141128341,'max_depth': 3.167260637404743,'min_child_samples': 9.324254670380586,'n_estimators': 2816.120214269747,'num_leaves': 17.035319354824765,'reg_alpha': 0.33257959049618924,'reg_lambda': 0.5556596408409464,'subsample': 0.8898942099929366}}
- 使用最优参(记得该取整的要取整)再次训练模型,得到相对理想的模型。
写在最后
- 如果大家想更好的使用贝叶斯优化包,可以读一读优化包的GitHub说明,里面有基于经验范围的精密搜索、经验函数等一些参数的调整等,或许能提升优化器性能。
随机森林、LGBM基于贝叶斯优化调参相关推荐
- 随机森林算法及贝叶斯优化调参Python实践
1. 随机森林算法 1.1. 集成模型简介 集成学习模型使用一系列弱学习器(也称为基础模型或基模型)进行学习,并将各个弱学习器的结果进行整合,从而获得比单个学习器更好的学习效果. 集成学习模型的常见算 ...
- 贝叶斯优化调参-Bayesian optimiazation原理加实践
随着机器学习用来处理大量数据被广泛使用,超参数调优所需要的空间和过程越来越复杂.传统的网格搜索和随即搜索已经不能满足用户的需求,因此方便快捷的贝叶斯优化调参越来越受程序员青睐. 1.贝叶斯优化原理介绍 ...
- sklearn模型使用贝叶斯优化调参(以随机森林为例)
文章目录 1. 构造数据源 2. 构造黑盒目标函数 3. 确定取值空间 4. 构造贝叶斯优化器 5. 运行,导出结果与最优参数 全部代码 贝叶斯优化github地址:https://github.co ...
- 基于贝叶斯优化的离散组合序列问题调研
基于贝叶斯优化的离散组合序列问题调研 标题 发表时间 作者 期刊 引用量 摘要 问题 心得体会 Applying Bayesian Approach to Combinatorial Problem ...
- 数据分享|Python决策树、随机森林、朴素贝叶斯、KNN(K-最近邻居)分类分析银行拉新活动挖掘潜在贷款客户...
原文链接:http://tecdat.cn/?p=23518 项目背景:银行的主要盈利业务靠的是贷款,这些客户中的大多数是存款大小不等的责任客户(存款人).银行拥有不断增长的客户(点击文末" ...
- 让机器学习手把手指导你的下一步实验-基于贝叶斯优化的序贯实验
在企业的日常研发工作中, 工程师可能会面对众多潜在影响因子及这些因子间的复杂组合而无所适从,不知道如何通过实验构建这些因子与最终响应间的数学模型.类似困扰常出现在锂电池行业对于电解液配方的研究.医药行 ...
- 【2 - 随机森林 - 案例部分:随机森林在乳腺癌数据上的调参】菜菜sklearn机器学习
课程地址:<菜菜的机器学习sklearn课堂>_哔哩哔哩_bilibili 第一期:sklearn入门 & 决策树在sklearn中的实现 第二期:随机森林在sklearn中的实现 ...
- 基于6种监督学习(逻辑回归+决策树+随机森林+SVM+朴素贝叶斯+神经网络)的毒蘑菇分类
公众号:尤而小屋 作者:Peter 编辑:Peter 大家好,我是Peter~ 本文是kaggle案例分享的第3篇,赛题的名称是:Mushroom Classification,Safe to eat ...
- 基于贝叶斯优化方法的自动调参实现
文章目录 1. 贝叶斯优化方法 2. Python中的选择 3. 优化问题的四个部分 4. 代码演示 1. 贝叶斯优化方法 (注意是方法,是一种思想) 贝叶斯优化通过基于目标函数的过去评估结果建立替代 ...
最新文章
- 关于C#调用非托管DLL,报“内存已损坏的”坑,坑,坑
- Java 的这 100 个坑,我发誓不会再踩了
- mysql事务嵌套 php_使用以下代码,MySQL中的PHP“嵌套”事务是否...
- Flutter基础—定位对齐之填充
- ubuntu16.04 gnome环境,怎么样下载使用monaco字体
- bin音频文件转MP3
- Dart语言入门(一)
- Dell台式计算机BIOS放电,戴尔BIOS设置电池维修笔记本电脑电池BIOS设置图形方法...
- python 压缩 解压文件
- docker学习(二)docker入门
- 使用关键词快速搜索商品代码
- “省份、城市、区县” 三级联动数据
- 好的博客(给自己翻译用)
- pytorch网络知识点四_thop计算Parameter和FLOPs,__all__的作用, GFLOPs =? GMacs
- 阿里CTO王坚:商业进化的四个关键词
- BitComet 0.90 ED下载功能的插件下载地址
- Linux搭建Web网站
- LHS 和 RHS---你所不知道的JavaScript
- Day 11. Evidence for a mental health crisis in graduate education
- 【渝粤教育】国家开放大学2018年秋季 8181-21T (1)老年保健按摩 参考试题
热门文章
- Inferring Motion Direction using Commodity Wi-Fi for Interactive Exergames (WiDrancd)论文翻译
- 八百元八核的服务器?二手服务器搭建指南
- 今日头条运营秘诀分享:让你的文章阅读量和收入都“飞”起来
- 记录第一次使用python模拟鼠标点击
- developer 开启双重验证 your appid needs to be updated
- matlab 分类学习工具箱 Classification Learner的使用及导出其生成的图,混淆矩阵confusion matrix的画法
- 关于阿里云服务器购买和域名购买的相关流程
- 中职学校计算机教学探讨,中职学校计算机课程教学方法的探讨.doc
- Allegro172版本DFM规则之DFT outline
- 知晓云 php,2020 知晓云小程序年度评选获奖名单发布