机器学习知识点(七)决策树学习算法Java实现

为理解机器学习第四章节决策树学习算法，通过网上找到的一份现成代码，主要实现了最优划分属性选择和决策树构造，其中最优划分属性选择采用信息增益准则、决策树构造采用递归实现，代码如下：

package sk.ml;import java.util.HashMap;
import java.util.LinkedList;
import java.util.List;
import java.util.Map;
import java.util.Map.Entry;
import java.util.Set;public class DicisionTree {public static void main(String[] args) throws Exception {String[] attrNames = new String[] { "AGE", "INCOME", "STUDENT","CREDIT_RATING" }; // 读取样本集Map<Object, List<Sample>> samples = readSamples(attrNames);// 生成决策树Object decisionTree = generateDecisionTree(samples, attrNames);// 输出决策树outputDecisionTree(decisionTree, 0, null);}/*** 读取已分类的样本集，返回Map：分类 -> 属于该分类的样本的列表*/static Map<Object, List<Sample>> readSamples(String[] attrNames) {// 样本属性及其所属分类（数组中的最后一个元素为样本所属分类）Object[][] rawData = new Object[][] {{ "<30  ", "High  ", "No ", "Fair     ", "0" },{ "<30  ", "High  ", "No ", "Excellent", "0" },{ "30-40", "High  ", "No ", "Fair     ", "1" },{ ">40  ", "Medium", "No ", "Fair     ", "1" },{ ">40  ", "Low   ", "Yes", "Fair     ", "1" },{ ">40  ", "Low   ", "Yes", "Excellent", "0" },{ "30-40", "Low   ", "Yes", "Excellent", "1" },{ "<30  ", "Medium", "No ", "Fair     ", "0" },{ "<30  ", "Low   ", "Yes", "Fair     ", "1" },{ ">40  ", "Medium", "Yes", "Fair     ", "1" },{ "<30  ", "Medium", "Yes", "Excellent", "1" },{ "30-40", "Medium", "No ", "Excellent", "1" },{ "30-40", "High  ", "Yes", "Fair     ", "1" },{ ">40  ", "Medium", "No ", "Excellent", "0" } };// 读取样本属性及其所属分类，构造表示样本的Sample对象，并按分类划分样本集Map<Object, List<Sample>> ret = new HashMap<Object, List<Sample>>();for (Object[] row : rawData) {Sample sample = new Sample();int i = 0;for (int n = row.length - 1; i < n; i++)sample.setAttribute(attrNames[i], row[i]);sample.setCategory(row[i]);List<Sample> samples = ret.get(row[i]);if (samples == null) {samples = new LinkedList<Sample>();ret.put(row[i], samples);}samples.add(sample);}return ret;}/*** 构造决策树*/static Object generateDecisionTree(Map<Object, List<Sample>> categoryToSamples, String[] attrNames) {// 如果只有一个样本，将该样本所属分类作为新样本的分类if (categoryToSamples.size() == 1)return categoryToSamples.keySet().iterator().next();// 如果没有供决策的属性，则将样本集中具有最多样本的分类作为新样本的分类，即投票选举出分类if (attrNames.length == 0) {int max = 0;Object maxCategory = null;for (Entry<Object, List<Sample>> entry : categoryToSamples.entrySet()) {int cur = entry.getValue().size();if (cur > max) {max = cur;maxCategory = entry.getKey();}}return maxCategory;}// 选取测试属性Object[] rst = chooseBestTestAttribute(categoryToSamples, attrNames);// 决策树根结点，分支属性为选取的测试属性Tree tree = new Tree(attrNames[(Integer) rst[0]]);// 已用过的测试属性不应再次被选为测试属性String[] subA = new String[attrNames.length - 1];for (int i = 0, j = 0; i < attrNames.length; i++)if (i != (Integer) rst[0])subA[j++] = attrNames[i];// 根据分支属性生成分支@SuppressWarnings("unchecked")Map<Object, Map<Object, List<Sample>>> splits =/* NEW LINE */(Map<Object, Map<Object, List<Sample>>>) rst[2];for (Entry<Object, Map<Object, List<Sample>>> entry : splits.entrySet()) {Object attrValue = entry.getKey();Map<Object, List<Sample>> split = entry.getValue();Object child = generateDecisionTree(split, subA);tree.setChild(attrValue, child);}return tree;}/*** 选取最优测试属性。最优是指如果根据选取的测试属性分支，则从各分支确定新样本* 的分类需要的信息量之和最小，这等价于确定新样本的测试属性获得的信息增益最大* 返回数组：选取的属性下标、信息量之和、Map(属性值->(分类->样本列表))*/static Object[] chooseBestTestAttribute(Map<Object, List<Sample>> categoryToSamples, String[] attrNames) {int minIndex = -1; // 最优属性下标double minValue = Double.MAX_VALUE; // 最小信息量Map<Object, Map<Object, List<Sample>>> minSplits = null; // 最优分支方案// 对每一个属性，计算将其作为测试属性的情况下在各分支确定新样本的分类需要的信息量之和，选取最小为最优for (int attrIndex = 0; attrIndex < attrNames.length; attrIndex++) {int allCount = 0; // 统计样本总数的计数器// 按当前属性构建Map：属性值->(分类->样本列表)Map<Object, Map<Object, List<Sample>>> curSplits =/* NEW LINE */new HashMap<Object, Map<Object, List<Sample>>>();for (Entry<Object, List<Sample>> entry : categoryToSamples.entrySet()) {Object category = entry.getKey();List<Sample> samples = entry.getValue();for (Sample sample : samples) {Object attrValue = sample.getAttribute(attrNames[attrIndex]);Map<Object, List<Sample>> split = curSplits.get(attrValue);if (split == null) {split = new HashMap<Object, List<Sample>>();curSplits.put(attrValue, split);}List<Sample> splitSamples = split.get(category);if (splitSamples == null) {splitSamples = new LinkedList<Sample>();split.put(category, splitSamples);}splitSamples.add(sample);}allCount += samples.size();}// 计算将当前属性作为测试属性的情况下在各分支确定新样本的分类需要的信息量之和double curValue = 0.0; // 计数器：累加各分支for (Map<Object, List<Sample>> splits : curSplits.values()) {double perSplitCount = 0;for (List<Sample> list : splits.values())perSplitCount += list.size(); // 累计当前分支样本数double perSplitValue = 0.0; // 计数器：当前分支for (List<Sample> list : splits.values()) {double p = list.size() / perSplitCount;perSplitValue -= p * (Math.log(p) / Math.log(2));}curValue += (perSplitCount / allCount) * perSplitValue;}// 选取最小为最优if (minValue > curValue) {minIndex = attrIndex;minValue = curValue;minSplits = curSplits;}}return new Object[] { minIndex, minValue, minSplits };}/*** 将决策树输出到标准输出*/static void outputDecisionTree(Object obj, int level, Object from) {for (int i = 0; i < level; i++)System.out.print("|-----");if (from != null)System.out.printf("(%s):", from);if (obj instanceof Tree) {Tree tree = (Tree) obj;String attrName = tree.getAttribute();System.out.printf("[%s = ?]\n", attrName);for (Object attrValue : tree.getAttributeValues()) {Object child = tree.getChild(attrValue);outputDecisionTree(child, level + 1, attrName + " = "+ attrValue);}} else {System.out.printf("[CATEGORY = %s]\n", obj);}}/*** 样本，包含多个属性和一个指明样本所属分类的分类值*/static class Sample {private Map<String, Object> attributes = new HashMap<String, Object>();private Object category;public Object getAttribute(String name) {return attributes.get(name);}public void setAttribute(String name, Object value) {attributes.put(name, value);}public Object getCategory() {return category;}public void setCategory(Object category) {this.category = category;}public String toString() {return attributes.toString();}}/*** 决策树（非叶结点），决策树中的每个非叶结点都引导了一棵决策树* 每个非叶结点包含一个分支属性和多个分支，分支属性的每个值对应一个分支，该分支引导了一棵子决策树*/static class Tree {private String attribute;private Map<Object, Object> children = new HashMap<Object, Object>();public Tree(String attribute) {this.attribute = attribute;}public String getAttribute() {return attribute;}public Object getChild(Object attrValue) {return children.get(attrValue);}public void setChild(Object attrValue, Object child) {children.put(attrValue, child);}public Set<Object> getAttributeValues() {return children.keySet();}}
}

执行结果如下：

[AGE = ?]
|-----(AGE = >40  ):[CREDIT_RATING = ?]
|-----|-----(CREDIT_RATING = Excellent):[CATEGORY = 0]
|-----|-----(CREDIT_RATING = Fair     ):[CATEGORY = 1]
|-----(AGE = <30  ):[STUDENT = ?]
|-----|-----(STUDENT = Yes):[CATEGORY = 1]
|-----|-----(STUDENT = No ):[CATEGORY = 0]
|-----(AGE = 30-40):[CATEGORY = 1]

机器学习知识点(七)决策树学习算法Java实现相关推荐

视觉机器学习20讲-MATLAB源码示例（4）-决策树学习算法
视觉机器学习20讲-MATLAB源码示例(4)-决策树学习算法 1. 决策树学习算法 2. Matlab仿真 3. 仿真结果 4. 小结 1. 决策树学习算法统计学,数据挖掘和机器学习中的决策树训练 ...
java实现一个感知机_感知机学习算法Java实现
感知机学习算法Java实现. Perceptron类用于实现感知机, 其中的perceptronOriginal()方法用于实现感知机学习算法的原始形式: perceptronAnother()方法用 ...
机器学习笔记16——决策树剪枝算法原理及python实现案例
决策树剪枝算法引言 1.算法目的 2.算法基本思路: 3.决策树损失函数 4.剪枝类型: 4.1 预剪枝 4.2 后剪枝 4.3 两种剪枝策略对比引言 \quad \quad在决策树.ID3.C4 ...
【机器学习原理】决策树分类算法
上一篇:朴素贝叶斯分类算法文章目录一.决策树分类 1. if - else 2. 如何种植一棵有灵魂的"树" 3. 决策条件的选择艺术节点纯度的度量规则纯度的度量方法 4. ...
吴恩达2022机器学习——第二部分高级学习算法第二周笔记
目录 1.Tensorflow实现 2.模型训练细节 3.Sigmoid激活函数的替代方案 4.如何选择激活函数 5.为什么模型需要激活函数 6.多分类问题 7.softmax回归算法 8.神经网络的 ...
机器学习中的流形学习算法 Manifold Learning
1. 流形学习概述流形学习manifold learning,于2000年在Science杂志上首次提出,是一大类基于流形的框架,是机器学习.模式识别中的一种方法,在维数约简(降维)方面具有广泛的应 ...
斯坦福CS229机器学习笔记-Lecture5 - 生成学习算法+高斯判别GDA+朴素贝叶斯+拉普拉斯平滑
作者:teeyohuang 邮箱:teeyohuang@163.com 本文系原创,供交流学习使用,转载请注明出处,谢谢声明:此系列博文根据斯坦福CS229课程,吴恩达主讲所写,为本人自学笔记,写 ...
决策树学习算法 - ID3算法
ID3算法概要: 熵值公式: 举例: 熵值公式用于判断一个样本集是否均匀分布,值的区间在[0,1]内,他有以下特点 1.分布越均匀,值越大 2.如果成单一分布,那么值为0 图例: 故可以推出通式(其中 ...
(转载)机器学习知识点(十四)EM算法原理
1.引言以前我们讨论的概率模型都是只含观测变量(observable variable), 即这些变量都是可以观测出来的,那么给定数据,可以直接使用极大似然估计的方法或者贝叶斯估计的方法:但是当模型 ...

机器学习知识点(七)决策树学习算法Java实现

机器学习知识点(七)决策树学习算法Java实现相关推荐

最新文章

热门文章