集成算法-Xgboost

Xgboost其实是将弱分类器组合起来的一种算法
核心在于加入新分类器后提升预测能力

惩罚项：欧米伽ft
其中γ是惩罚力度，T是树的个数，w是权重

Xgboost Python实例：
数据集展示：

import xgboost
# First XGBoost model for Pima Indians dataset
from numpy import loadtxt
from xgboost import XGBClassifier
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score
# load data
dataset = loadtxt('pima-indians-diabetes.csv', delimiter=",")
# split data into X and y
X = dataset[:,0:8]
Y = dataset[:,8]
# split data into train and test sets
seed = 7
test_size = 0.33
X_train, X_test, y_train, y_test = train_test_split(X, Y, test_size=test_size, random_state=seed)
# fit model no training data
model = XGBClassifier()
model.fit(X_train, y_train)
# make predictions for test data
y_pred = model.predict(X_test)
predictions = [round(value) for value in y_pred]
# evaluate predictions
accuracy = accuracy_score(y_test, predictions)
print("Accuracy: %.2f%%" % (accuracy * 100.0))

需要用到xgboost模块
from xgboost import XGBClassifier
还用到了准确率测算模块
from sklearn.metrics import accuracy_score

from numpy import loadtxt
from xgboost import XGBClassifier
from xgboost import plot_importance
from matplotlib import pyplot
# load data
dataset = loadtxt('pima-indians-diabetes.csv', delimiter=",")
# split data into X and y
X = dataset[:,0:8]
y = dataset[:,8]
# fit model no training data
model = XGBClassifier()
model.fit(X, y)
# plot feature importance
plot_importance(model)
pyplot.show()

# Tune learning_rate
from numpy import loadtxt
from xgboost import XGBClassifier
from sklearn.model_selection import GridSearchCV
from sklearn.model_selection import StratifiedKFold
# load data
dataset = loadtxt('pima-indians-diabetes.csv', delimiter=",")
# split data into X and y
X = dataset[:,0:8]
Y = dataset[:,8]
# grid search
model = XGBClassifier()
learning_rate = [0.0001, 0.001, 0.01, 0.1, 0.2, 0.3]
param_grid = dict(learning_rate=learning_rate)
kfold = StratifiedKFold(n_splits=10, shuffle=True, random_state=7)
grid_search = GridSearchCV(model, param_grid, scoring="neg_log_loss", n_jobs=-1, cv=kfold)
grid_result = grid_search.fit(X, Y)
# summarize results
print("Best: %f using %s" % (grid_result.best_score_, grid_result.best_params_))
means = grid_result.cv_results_['mean_test_score']
params = grid_result.cv_results_['params']
for mean, param in zip(means, params):print("%f  with: %r" % (mean, param))

1.learning rate
2.tree
max_depth
min_child_weight
subsample, colsample_bytree
gamma
3.正则化参数
lambda
alpha

xgb1 = XGBClassifier(learning_rate =0.1,n_estimators=1000,max_depth=5,min_child_weight=1,gamma=0,subsample=0.8,colsample_bytree=0.8,objective= 'binary:logistic',nthread=4,scale_pos_weight=1,seed=27)

集成算法-Xgboost相关推荐

xgboost简单介绍_好文干货|全面理解项目中最主流的集成算法XGBoost 和 LightGBM
点击上方"智能与算法之路",选择"星标"公众号第一时间获取价值内容本文主要介绍基于 Boosting 框架的主流集成算法,包括 XGBoost 和 Ligh ...
【机器学习】XGBoost集成算法——（理论+图解+python代码比较其他算法使用天池蒸汽数据）
一.集成算法思想二.XGBoost基本思想三.用python实现XGBoost算法在竞赛题中经常会用到XGBoost算法,用这个算法通常会使我们模型的准确率有一个较大的提升.既然它效果这么好,那 ...
以XGBoost为代表的集成算法体现的哲学思想与数学技巧
目录哲学思想一:抓住主要矛盾为什么AdaBoost要增加前一次错分样本的权重? 为什么lightGBM可以忽略梯度小的样本? 哲学思想二: 矛盾在一定条件下是可以相互转化的. 为什么随机森林比单一 ...
集成学习-Boosting集成学习算法XGBoost
XGBoost全名叫(eXtreme Gradient Boosting)极端梯度提升,经常被用在一些项目中,其效果显著.它是大规模并行boosted tree的工具,它是目前最快最好的开源boost ...
halcon机器视觉算法原理与编程实战_快速弄懂机器学习里的集成算法：原理、框架与实战...
作者: 博观厚积简书专栏:https://www.jianshu.com/u/2f376f777ef1 1. 关于集成学习算法集成学习算法,通俗地讲就是:三个臭皮匠,顶个诸葛亮,这在很多地方都有 ...
数据挖掘：模型选择——集成算法与树模型
之前介绍的树模型属于弱学习器,本身的算法比较简单,但是与集成算法合并后,会产生更好的效果.比如:决策树+bagging=随机森林:决策树+boosting=提升树这里先简单介绍下集成算法,然后对随机 ...
10- 天猫用户复购预测 (机器学习集成算法) (项目十) *
项目难点 merchant: 商人重命名列名: user_log.rename(columns={'seller_id':'merchant_id'}, inplace=True) 数据类型转换 ...
2022国赛数学建模思路 - 案例：集成算法AdaBoost
2022 高教社杯(国赛数学建模)思路解析 2022高教社杯ABCD赛题思路解析: https://blog.csdn.net/dc_sinor/article/details/126211983 集 ...
集成算法终极模型之《神器LightGBM》—最后的高山
关注+星标,听说他有点东西全文共 6481 字,阅读全文需 16 分钟写在前面的话大家好,我是小一 2021 年第一篇文章,还是决定用 6000 多字的技术文来开篇. 今天的文章是机器学习算法里 ...

集成算法-Xgboost

集成算法-Xgboost相关推荐

最新文章

热门文章