Logistic 梯度法进行分类

梯度上升法，求最大似然函数，见http://sbp810050504.blog.51cto.com/2799422/1608064/

梯度下降，最小二乘法结果一样，h=W'X,W=W+alfa*X'*(y-h)或者W=W+alfa*(y-h)*Xi 随机梯度法

# coding=utf-8 #


import numpy as npdef loadDataSet():dataMat = []; labelMat = []fr = open(r'C:\Users\li\Downloads\machinelearninginaction\Ch05\testSet.txt')for line in fr.readlines():lineArr = line.strip().split()dataMat.append([1.0, float(lineArr[0]), float(lineArr[1])]) #list:[[x0,x1,x2],...] listlabelMat.append(int(lineArr[2]))return dataMat,labelMatdef sigmoid(inX):return 1.0/(1+np.exp(-inX)) #overflow encountered in expdef gradAscent(dataMatIn, classLabels):dataMatrix = np.mat(dataMatIn)             #convert to NumPy matrix mXnlabelMat = np.mat(classLabels).transpose() #convert to NumPy matrix mX1m,n = np.shape(dataMatrix)alpha = 0.001maxCycles = 500weights = np.ones((n,1))                #w matrix nX1for k in range(maxCycles):              #heavy on matrix operationsh = sigmoid(dataMatrix*weights)     #matrix mult h mX1error = (labelMat - h)              #vector subtraction (y-h) mX1weights = weights + alpha * dataMatrix.transpose()* error #matrix mult w=w+alfa*X'*(y-h)return weightsdef plotBestFit(weights):import matplotlib.pyplot as pltdataMat,labelMat=loadDataSet()dataArr = np.array(dataMat)n = np.shape(dataArr)[0] xcord1 = []; ycord1 = []xcord2 = []; ycord2 = []for i in range(n):if int(labelMat[i])== 1:xcord1.append(dataArr[i,1]); ycord1.append(dataArr[i,2])else:xcord2.append(dataArr[i,1]); ycord2.append(dataArr[i,2])fig = plt.figure()ax = fig.add_subplot(111)ax.scatter(xcord1, ycord1, s=30, c='red', marker='s')ax.scatter(xcord2, ycord2, s=30, c='green')x = np.arange(-3.0, 4.0, 1)y = (-weights[0]-weights[1]*x)/weights[2] # super line:w0*1+w1*x1+w2*x2=0ax.plot(x, y.T)                      #x array 7 [],y array 1X7[[]]:?y.T array 7X1plt.xlabel('X1'); plt.ylabel('X2');plt.show()def stocGradAscent0(dataMatrix, classLabels):m,n = np.shape(dataMatrix)alpha = 0.01weights = np.ones(n)   #initialize to all onesfor i in range(m):h = sigmoid(sum(dataMatrix[i]*weights))error = classLabels[i] - hweights = weights + alpha * error * np.array(dataMatrix[i]) #calculate only for new data,not for all datareturn weightsdef stocGradAscent1(dataMatrix, classLabels, numIter=150):m,n = np.shape(dataMatrix)weights = np.ones(n)   #initialize to all onesfor j in range(numIter):dataIndex = list(range(m))for i in dataIndex:alpha = 4/(1.0+j+i)+0.01    #apha decreases with iteration, does not go to 0 because of the constantrandIndex = int(np.random.uniform(0,len(dataIndex)))#随机选择样本进行W值计算h = sigmoid(sum(dataMatrix[randIndex]*weights))error = classLabels[randIndex] - hweights = weights + alpha * error * np.array(dataMatrix[randIndex])del(dataIndex[randIndex]) #删除随机选择并完成计算的样本return weightsdef classifyVector(inX, weights):prob = sigmoid(sum(inX*weights))if prob > 0.5: return 1.0else: return 0.0def colicTest():frTrain = open(r'C:\Users\li\Downloads\machinelearninginaction\Ch05\horseColicTraining.txt')frTest = open(r'C:\Users\li\Downloads\machinelearninginaction\Ch05\horseColicTest.txt')trainingSet = []; trainingLabels = []for line in frTrain.readlines():currLine = line.strip().split('\t')lineArr =[]for i in range(21):lineArr.append(float(currLine[i]))trainingSet.append(lineArr)trainingLabels.append(float(currLine[21]))trainWeights = stocGradAscent1(np.array(trainingSet), trainingLabels, 1000)errorCount = 0; numTestVec = 0.0for line in frTest.readlines():numTestVec += 1.0currLine = line.strip().split('\t')lineArr =[]for i in range(21):lineArr.append(float(currLine[i]))if int(classifyVector(np.array(lineArr), trainWeights))!= int(currLine[21]):errorCount += 1errorRate = (float(errorCount)/numTestVec)print ("the error rate of this test is: %f" % errorRate)return errorRatedef multiTest():numTests = 10; errorSum=0.0for k in range(numTests):errorSum += colicTest()print ("after %d iterations the average error rate is: %f" % (numTests, errorSum/float(numTests)))dataMat,labelMat=loadDataSet()print(dataMat,labelMat)
#weights=gradAscent(dataMat, labelMat)
#weights=stocGradAscent0(dataMat, labelMat)
weights=stocGradAscent1(dataMat, labelMat, 200)
print(weights)
plotBestFit(weights)
multiTest()

Logistic 梯度法进行分类相关推荐

R语言基于glmnet构建Logistic回归模型使用L1正则化并可视化系数及最佳lambda值
R语言基于glmnet构建Logistic回归模型使用L1正则化并可视化系数及最佳lambda值 Glmnet主要用于拟合广义线性模型.筛选可以使loss达到最小的正则化参数lambda.该算法非常快 ...
机器学习实战读书笔记--logistic回归
1. 利用logistic回归进行分类的主要思想是:根据现有数据对分类边界线建立回归公式,以此进行分类. 2.sigmoid函数的分类 Sigmoid函数公式定义 3.梯度上升法基本思想:要找个某个 ...
用python做逻辑回归梯度上升_机器学习实例---4.1、Logistic回归基础篇之梯度上升算法...
一前言本文从Logistic回归的原理开始讲起,补充了书上省略的数学推导.本文可能会略显枯燥,理论居多,Sklearn实战内容会放在下一篇文章.自己慢慢推导完公式,还是蛮开心的一件事. 二 Log ...
Logistic 回归(sigmoid函数，手机的评价,梯度上升，批处理梯度,随机梯度，从疝气病症预测病马的死亡率...
(手机的颜色,大小,用户体验来加权统计总体的值)极大似然估计MLE 1.Logistic回归 Logistic regression (逻辑回归),是一种分类方法,用于二分类问题(即输出只有两种).如 ...
机器学习实战（五）——Logistic 回归
文章目录 Logistic 回归 5.2 基于最优化方法的最佳回归系数确定 5.2.1 梯度上升法 5.3 python实战 5.3.1 查看数据集分布情况 5.3.2 训练 5.3.3 绘制决策边界 ...
机器学习实战之logistic回归分类
利用logistic回归进行分类的主要思想:根据现有数据对分类边界建立回归公式,并以此进行分类. logistic优缺点: 优点:计算代价不高,易于理解和实现. 缺点:容易欠拟合,分类精度可能不高. ...
Machine Learning in Action 读书笔记---第5章 Logistic回归
Machine Learning in Action 读书笔记第5章 Logistic回归文章目录 Machine Learning in Action 读书笔记一.Logistic回归 1.L ...
logistic回归分析优点_机器学习实战项目-Logistic回归
Logistic 回归概述 Logistic 回归虽然名字叫回归,但是它是用来做分类的.其主要思想是: 根据现有数据对分类边界线建立回归公式,以此进行分类. 须知概念 Sigmoid 函数回归概 ...
机器学习（四）——逻辑斯蒂回归（Logistic Regression）
机器学习(四)--逻辑斯蒂回归(Logistic Regression) 一.算法简介 1.1 概念二.Logistic回归理论推导 2.1 Logistic回归 2.1.1 参数向量θ 2.2 梯 ...

Logistic 梯度法进行分类

Logistic 梯度法进行分类相关推荐

最新文章

热门文章