Python机器学习（1）-- 自己设计一个感知机（Perceptron）分类算法

红色石头的个人网站：redstonewill.com

Implementing a perceptron learning algorithm in Python

Define a Class

import numpy as np
class Perceptron(object):"""Perceptron classifier.Parameters------------eta : floatLearning rate (between 0.0 and 1.0)n_iter : intPasses over the training dataset.Attributes-----------w_ : 1d-arrayWeights after fitting.errors_ : listNumber of misclassifications (updates) in each epoch."""def __init__(self, eta=0.01, n_iter=10):self.eta = etaself.n_iter = n_iterdef fit(self, X, y):"""Fit training data.Parameters----------X : {array-like}, shape = [n_samples, n_features]Training vectors, where n_samples is the number of samples andn_features is the number of features.y : array-like, shape = [n_samples]Target values.Returns-------self : object"""self.w_ = np.zeros(1 + X.shape[1])self.errors_ = []for _ in range(self.n_iter):errors = 0for xi, target in zip(X, y):update = self.eta*(target - self.predict(xi))self.w_[1:] += update*xiself.w_[0] += updateerrors += int(update != 0.0)self.errors_.append(errors)return selfdef net_input(self, X):"""Calculate net input"""return np.dot(X, self.w_[1:]) + self.w_[0]def predict(self, X):"""Return class label after unit step"""return np.where(self.net_input(X) >= 0.0, 1, -1)

Training a perceptron model on the Iris dataset

import pandas as pd
df = pd.read_csv('https://archive.ics.uci.edu/ml/machine-learning-databases/iris/iris.data', header=None)

df.tail()

	0	1	2	3	4
145	6.7	3.0	5.2	2.3	Iris-virginica
146	6.3	2.5	5.0	1.9	Iris-virginica
147	6.5	3.0	5.2	2.0	Iris-virginica
148	6.2	3.4	5.4	2.3	Iris-virginica
149	5.9	3.0	5.1	1.8	Iris-virginica

We extract the first 100 class labels that correspond to 50 Iris-Setosa and 50 Iris-Versicolor flowers.

%matplotlib inline
import matplotlib.pyplot as plt
import numpy as npy = df.iloc[0:100, 4].values
y = np.where(y == 'Iris-setosa', -1, 1)
X = df.iloc[0:100, [0,2]].values
plt.scatter(X[:50, 0], X[:50, 1], color='red', marker='o', label='setosa')
plt.scatter(X[50:100, 0], X[50:100, 1], color='blue', marker='x', label='versicolor')
plt.xlabel('petal length')
plt.ylabel('sepal length')
plt.legend(loc='upper left')
# plt.show()

To train our perceptron algorithm, plot the misclassification error

ppn = Perceptron(eta=0.1, n_iter=10)
ppn.fit(X,y)
plt.plot(range(1,len(ppn.errors_) + 1), ppn.errors_, marker='o')
plt.xlabel('Epochs')
plt.ylabel('Number of misclassifications')
# plt.show()

Visualize the decision boundaries for 2D datasets

from matplotlib.colors import ListedColormapdef plot_decision_regions(X, y, classifier, resolution=0.02):# setup marker generator and color mapmarkers = ('s', 'x', 'o', '^', 'v')colors = ('red', 'blue', 'lightgreen', 'gray', 'cyan')cmap = ListedColormap(colors[:len(np.unique(y))])# plot the decision surfacex1_min, x1_max = X[:, 0].min() - 1, X[:, 0].max() + 1x2_min, x2_max = X[:, 1].min() - 1, X[:, 1].max() + 1xx1, xx2 = np.meshgrid(np.arange(x1_min, x1_max, resolution), np.arange(x2_min, x2_max, resolution))Z = classifier.predict(np.array([xx1.ravel(), xx2.ravel()]).T)Z = Z.reshape(xx1.shape)plt.contourf(xx1, xx2, Z, alpha=0.4, cmap=cmap)plt.xlim(xx1.min(), xx1.max())plt.ylim(xx2.min(), xx2.max())# plot class samplefor idx, cl in enumerate(np.unique(y)):plt.scatter(x=X[y == cl,0], y=X[y == cl, 1], alpha=0.8, c=cmap(idx), marker=markers[idx], label=cl)

plot_decision_regions(X, y, classifier=ppn)
plt.xlabel('sepal lenght [cm]')
plt.ylabel('petal length [cm]')
plt.legend(loc='upper left')
# plt.show()

Adaptive linear neurons and the convergence of learning

Implementing an Adaptive Linear Neuron in Python

class AdalineGD(object):"""ADAptive LInear NEuron classifier.Parameters-------------eta : floatLearning rate (between 0.0 and 1.0)n_iter : intPasses over the training dataset.Attributes-------------w_ : 1d-arrayWeights after fitting.errors_ : listNumber of misclassifications in every epoch."""def __init__(self, eta=0.01, n_iter=50):self.eta = etaself.n_iter = n_iterdef fit(self, X, y):""" Fit training data.Parameters------------X : {array-like5}, shape = [n_samples, n_features]Training vectors,where n_samples is the number of samples andn_features is the number of features.y　: array-like, shape = [n_samples]Target values.Returns------------self : object"""self.w_ = np.zeros(1 + X.shape[1])self.cost_ = []for i in range(self.n_iter):output = self.net_input(X)errors = (y - output)self.w_[1:] += self.eta * X.T.dot(errors)self.w_[0] += self.eta * errors.sum()cost = (errors**2).sum() / 2.0self.cost_.append(cost)return selfdef net_input(self, X):"""Calculate net input"""return np.dot(X, self.w_[1:]) + self.w_[0]def activation(self, X):"""Compute linear activation"""return self.net_input(X)def predict(self, X):"""Return class label after unit step"""return np.where(self.activation(X) >= 0.0, 1, -1)

fig, ax = plt.subplots(nrows=1, ncols=2, figsize=(8,4))
ada1 = AdalineGD(n_iter=10, eta=0.01).fit(X,y)
ax[0].plot(range(1, len(ada1.cost_) + 1), np.log10(ada1.cost_), marker='o')
ax[0].set_xlabel('Epochs')
ax[0].set_ylabel('log(Sum-squared-error)')
ax[0].set_title('Adaline - Learning rate 0.01')
ada2 = AdalineGD(n_iter=10, eta=0.0001).fit(X,y)
ax[1].plot(range(1, len(ada2.cost_) + 1), ada2.cost_, marker='o')
ax[1].set_xlabel('Epochs')
ax[1].set_ylabel('Sum-squared-error')
ax[1].set_title('Adaline - Learning rate 0.0001')
# plt.show()

standardization

X_std = np.copy(X)
X_std[:,0] = (X_std[:,0] - X_std[:,0].mean()) / X_std[:,0].std()
X_std[:,1] = (X_std[:,1] - X_std[:,1].mean()) / X_std[:,1].std()

ada = AdalineGD(n_iter=15, eta=0.01)
ada.fit(X_std, y)
plot_decision_regions(X_std, y, classifier=ada)
plt.title('Adaline - Gradient Descent')
plt.xlabel('sepal length [standardized]')
plt.ylabel('petal length [standardized]')
plt.legend(loc='upper left')
# plt.show()

plt.plot(range(1, len(ada.cost_) + 1), ada.cost_, marker='o')
plt.xlabel('Epochs')
plt.ylabel('Sum-squared-error')
# plt.show()

Large scale machine learning and stochastic gradient descent

from numpy.random import seedclass AdalineSGD(object):"""ADAptive LInear NEuron classifier.Parameters------------eta : floatLearning rate (between 0.0 and 1.0)n_iter : intPasses over the training dataset.Attributes------------w_ : 1d-arrayWeights after fitting.cost_ : listNumber of misclassifications in every epoch.shuffle : bool (default: True)Shuffles training data every epochif True to prevent cycles.random_state : int (default: None)Set random state for shufflingand initializing the weights."""def __init__(self, eta=0.01, n_iter=10, shuffle=True, random_state=None):self.eta = etaself.n_iter = n_iterself.w_initialized = Falseself.shuffle = shuffleif random_state:seed(random_state)def fit(self, X, y):"""Fit training data.Parameters------------X : {array-like}, shape = [n_samples, n_features]Training vector, where n_samplesis the number of samples andn_features is the number of features.y: arrary-like, shape = [n_samples]Target values.Returns------------self : object"""self._initialize_weights(X.shape[1])self.cost_ = []for i in range(self.n_iter):if self.shuffle:X, y = self._shuffle(X, y)cost = []for xi, target in zip(X, y):cost.append(self._update_weights(xi, target))avg_cost = sum(cost)/len(y)self.cost_.append(avg_cost)return selfdef partial_fit(self, X, y):"""Fit training data without reinitializing the weights"""if not self.w_initialized:self._initialize_weights(X.shape[1])if y.ravel().shape[0] > 1:for xi, target in zip(X, y):self._update_weights(xi, target)else:self._update_weights(X, y)return selfdef _shuffle(self, X, y):"""Shuffle training data"""r = np.random.permutation(len(y))return X[r], y[r]def _initialize_weights(self, m):"""Initialize weighs to zeros"""self.w_ = np.zeros(1+m)self.w_initialized = Truedef _update_weights(self, xi, target):"""Apply Adaline learning rule to update the weights"""output = self.net_input(xi)error = (target - output)self.w_[1:] += self.eta*xi.dot(error)self.w_[0] += self.eta*errorcost = 0.5 * error**2return costdef net_input(self, X):"""Calculate net input"""return np.dot(X, self.w_[1:]) + self.w_[0]def activation(self, X):"""Compute linear activation"""return self.net_input(X)def predict(self, X):"""Return class label after unit step"""return np.where(self.activation(X) >= 0.0, 1, -1)

ada = AdalineSGD(n_iter=15, eta=0.01, random_state=1)
ada.fit(X_std, y)
plot_decision_regions(X_std, y, classifier=ada)
plt.title('Adaline - Stochastic Gradient Descent')
plt.xlabel('sepal length [standardized]')
plt.ylabel('petal length [standardized]')
plt.legend(loc='upper left')
plt.show()
plt.plot(range(1, len(ada.cost_) + 1), ada.cost_, marker='o')
plt.xlabel('Epochs')
plt.ylabel('Average Cost')
plt.show()

Python机器学习（1）-- 自己设计一个感知机（Perceptron）分类算法相关推荐

设计一个时间片轮转法调度的算法
设计一个时间片轮转法调度的算法首先根据进程的到达时间先后进行排序,然后每个进程运行一次,循环往复. #include<stdio.h> struct pcb{char name[10]; ...
(关于单链表的真题)已知一个带有表头结点的单链表...请设计一个尽可能高效的算法,查找链表中倒数第k个位置的结点。
真题描述已知一个带有表头结点的单链表,结点结构为 data next 假设该链表只给出了头指针head.在不改变链表的前提下,请设计一个尽可能高效的算法,查找链表中倒数第k个位置上的结点. 若查找成 ...
已知一个带有表头结点的单链表,结点结构为 data link 假设该链表只给出了头指针list。在不改变链表的前提下,请设计一个尽可能高效的算法,查找链表中倒数第k个位置
分析: 这是一个单链表算法题,题中说要在不改变链表本身的前提下,设计一个尽可能高效的算法,说明时间复杂度.空间复杂度都要尽可能地高效,常数也要尽可能小. 思路: 设置两个指针p和q,指p针在指针q后k ...
已知一个带有表头的单链表，结点结构为data-link，假设该链表只给出了头指针list。在不改变链表的前提下，请设计一个尽可能高效的算法，查找链表中倒数第k个位置上的结点（k为正整数）。
今天和大家分享一道2009年代码为408的一道真题: 已知一个带有表头的单链表,结点结构为data-link,假设该链表只给出了头指针list.在不改变链表的前提下,请设计一个尽可能高效的算法,查找链 ...
吴裕雄 python 机器学习——人工神经网络与原始感知机模型
import numpy as npfrom matplotlib import pyplot as plt from mpl_toolkits.mplot3d import Axes3D from ...
Python面向对象基础练习——设计一个名为 MyRectangle 的矩形类来表示矩形
# 设计一个名为 MyRectangle 的矩形类来表示矩形.这个类包含 # (1) 左上角顶点的坐标:x,y # (2) 宽度和高度:width.height # (3) 构造方法:传入 x,y,w ...
python测试题：请设计一个图书类Book，包括书号(num)，书名(name)，出版日期(Date) 从键盘输入图书册数n，接着输入n个图书信息，按书名排序输出所有图书信息
题目: 请设计一个图书类Book,包括书号(num),书名(name),出版日期(Date) 从键盘输入图书册数n,接着输入n个图书信息,按书名排序输出所有图书信息思路: 首先肯定是先创建一个boo ...
Python基础学习笔记——设计一个名为MyRectangle的矩形类来表示矩形
学习目标: '设计一个名为MyRectangle的矩形类来表示矩形,这个类包含: (1)左上角坐标:x,y (2)宽度和高度 (3)构造方法:传入x,y,width,height.如果(x,y)不传默 ...
机器学习系列（一）感知器分类算法
分类算法有两种类型:感知器和适应性线性神经元神经元的数学表示 w=[w1w2...wm],x=[x1x2...xm]w=\begin{bmatrix} w_1 \\ w_2 \\ ... \\ w_ ...
系统学习机器学习之总结（一）--常见分类算法优缺点
主要是参考网上各种资源,做了整理.其实,这里更多的是从基础版本对比,真正使用的时候,看数据,看改进后的算法. 1. 五大流派 ①符号主义:使用符号.规则和逻辑来表征知识和进行逻辑推理,最喜欢的算法是: ...

Python机器学习（1）-- 自己设计一个感知机（Perceptron）分类算法

Implementing a perceptron learning algorithm in Python

Define a Class

Training a perceptron model on the Iris dataset

We extract the first 100 class labels that correspond to 50 Iris-Setosa and 50 Iris-Versicolor flowers.

To train our perceptron algorithm, plot the misclassification error

Visualize the decision boundaries for 2D datasets

Adaptive linear neurons and the convergence of learning

Implementing an Adaptive Linear Neuron in Python

standardization

Large scale machine learning and stochastic gradient descent

Python机器学习（1）-- 自己设计一个感知机（Perceptron）分类算法相关推荐

最新文章

热门文章