1SE rule details in CCP pruning of CART

several ways to choose the best tree in the pruned tree list
besttreefrom={cross−validation,smalldatasetsk−SE,largedatasetschoosethewiththelowesterror,largedatasetsbest\ tree from=\left\{ \begin{aligned} cross-validation ,small\ datasets \\ k-SE \ ,large\ datasets\\ choose\ the\ with\ the\ lowest\ error,large\ datasets \end{aligned} \right.best treefrom=⎩⎪⎨⎪⎧cross−validation,small datasetsk−SE ,large datasetschoose the with the lowest error,large datasets

At first,"pruned tree"is an ambiguous word which has several meanings,Let’s
make a decision :
1.call the following bigger circle as “pruned tree” who contains the original root node.
1.call the following smaller circle as “pruned parts” who don’t contains the original root node.

CCP:cost complexity pruning
according to [1]:

the 1-SE Rule advises us to find the complexity parameter
with an associated cross-validated error less than the
minimum cross-validated standard error + 1 standard deviation
of that error.

according to[2]:

In summary:
when we get the pruned sequence after CCP Algorithm,the pruned tree sequence is:
[T1,T2,...TkT_1,T_2,...T_kT1,T2,...Tk]
We do NOT select the candidate in the above sequence with the minimum MSE where MSE is computed with an independent validation datasets based on the pruned tree-Tkmodel,k∈[1,K]T_k model ,k\in[1,K]Tkmodel,k∈[1,K]

We make a compromise:
we choose the subtree in above sequence whose MSE(computed with independent validatation datasets)lower than the"minimum MSE + 1·Standard Error of the pruned tree who owns the minimum MSE".
The number of pruned trees in the above sequence who satisfied the above requirement may be more than one.We just select the subtree who has the biggest k among these pruned trees who satisfy the requirement..

Such behavior loses some precision in terms of validation,but it improve the simplicity of the final tree from the above sequence.

Reference:
[1]Statistical Consulting Group
[2]Cost-Complexity Pruning Process - IBM

1SE rule details in CCP pruning of CART相关推荐

History of pruning algorithm development and python implementation(finished)
All the python-implementation for 7 post-pruning Algorithms are here. Table of Decision Trees: name ...
决策树（三）--完整总结（ID3，C4.5，CART,剪枝，替代）
参考: http://blog.csdn.net/zhaocj/article/details/50503450 总结很完整一.原理决策树是一种非参数的监督学习方法,它主要用于分类和回归.决策树的 ...
CART分类与回归树的原理与实现
算法概述 CART(Classification And Regression Tree)算法是一种决策树分类方法. 它采用一种二分递归分割的技术,分割方法采用基于最小距离的基尼指数估计函数,将当前的 ...
1. 分类与回归树原理（CART）
1. 简介分类与回归树(Classification And Regression Tree),采用二分递归分割技术,将当前样本集划分成两个子集,即其结构为二叉树,每个内部节点均只有两个分支.左分支 ...
模型通道剪枝汇总（channel pruning）
目前所看到的通道剪枝的方法不是特别多,先总结一下现有的,持续更新 Pruning Filters for Efficient ConvNets 论文:https://arxiv.org/pdf/160 ...
notes from《classification and regression trees》
page 11th R(d)=1N∑n=1NX(d(xn)≠jn)−−−−−−−(1.8)R(d)=\frac{1}{N}\sum_{n=1}^NX(d(x_n)≠j_n)-------(1.8)R( ...
别以为if slse很简单——决策树
怎么分--熵与Gini指数熵,表示信息量的期望,含义是混乱程度,也是对随机变量编码所需的最小比特数.请参考之前的文章. 信息增益建立在熵之上,是选择某特征之后熵减少的多少(熵减少即信息增加),等于信 ...
如何启用SAP Business by design里的Correction Invoice功能
Sent: Wednesday, May 18, 2011 11:42 AM Subject: [Tip] How to enable the function Correction Invoice ...
【ASP.NET Web API教程】2.3 与实体框架一起使用Web API
2.3 Using Web API with Entity Framework 2.3 与实体框架一起使用Web API 本小节是ASP.NET Web API第2章的第3小节,原文共分为7个部分,分 ...

1SE rule details in CCP pruning of CART

1SE rule details in CCP pruning of CART相关推荐

最新文章

热门文章