我一直都对数据分析这一行业很感兴趣，因此特地选修了很多数据科学、统计分析方面的专业课程，来不断提高自己的数据分析能力和数据的敏感度，希望用紧密的逻辑思维来帮助我更好的理解世界，理解各大行业。

本次项目是国科大“数据科学导论”这门课的课后作业，认真完成后，特地发到知乎上，与知友门一起见证成长。

一、项目介绍

运用KNN、线性回归、多元回归方法对Iris Data Set数据进行预测，通过sepal length、sepal width、petal length、petal width这四个特征量来预测花的种类("0": Iris Setosa，"1": Iris Versicolour，"2": Iris Virginica)

KNN怎么做回归？

要预测的点的值通过求与它距离最近的K个点的值的平均值得到，这里的“距离最近”可以是欧氏距离，也可以是其他距离，具体的效果依数据而定，思路一样。

数据集：Iris Data Set(鸢尾属植物数据集)，它首次出现在著名的英国统计学家和生物学家

数据集的特征：

该数据集测量了所有150个样本的4个特征，分别是：sepal length(花萼长度)

sepal width(花萼宽度)

petal length(花瓣长度)

petal width(花瓣宽度)

以上四个特征的单位都是厘米(cm)

大体步骤：

1. Load in the iris dataset which is split into a training and testing dataset

2. Do some basic exploratory analysis of the dataset and go through a scatterplot

3. Write out the algorithm for kNN WITHOUT using the sklearn package

4. Use the sklearn package to implement kNN and compare to the one we did by hand

5. Extend the sklearn package to linear and polynomial regression

二、项目步骤

1.导入相应的包和Iris Data Set数据集

import sys

import numpy as np

import pandas as pd

pd.set_option('display.max_rows', 999)

pd.set_option('display.width', 500)

pd.set_option('display.notebook_repr_html', True)

import matplotlib

import matplotlib.pyplot as plt

import seaborn as sns

from sklearn.neighbors import KNeighborsClassifier

from sklearn.model_selection import train_test_split

from sklearn import metrics, datasets

from collections import Counter

import statsmodels.api as sm

from statsmodels.api import OLS

from sklearn.preprocessing import PolynomialFeatures

from sklearn.linear_model import LinearRegression

%matplotlib inline

在数据集的“target”列中，令“0”代表Setosa, “1“代表 Versicolour, ”2“代表 Virginica.

# 加载Iris Data Set数据集(包含在sklearn包里，原始数据集是类似字典结构的对象)

iris_bunch = datasets.load_iris()

# np.c_是按行连接两个矩阵，就是把两矩阵左右相加，要求行数相等

# "target"是我们的"Y"变量，"data"是我们的"X"变量.

iris = pd.DataFrame(data= np.c_[iris_bunch['data'], iris_bunch ['target']],

columns= iris_bunch['feature_names'] + ['target'])

iris.head()

iris.describe()

我们也可以看看通过"target"进行分组后，各个变量的平均值

iris.groupby('target').mean()

2.标准化数据并绘制散点图

In general, it is good practice to normalize data before proceeding. As such, we can create the following functions:

# 标准化Iris数据

def normalize(x):

num = x - np.min(x)

denom = np.max(x) - np.min(x)

return (num / denom)

iris.iloc[:, 0:4] = normalize(iris.iloc[:, 0:4])

iris.head()

#标准化之后的数据特征

iris.describe()

用pairplot()绘制散点图矩阵，由下图可以看出

features_cols = iris.columns[:4]

sns.pairplot(data=iris,hue='target',vars=features_cols)

3. 将数据集分成训练集和测试集

我们想通过数据集中的4个特征(sepal length、sepal width、petal length、petal width)来预测花的类型(Setosa, Versicolour, Virginica)，无论采用什么模型，都要把数据集先分为训练集和测试集，训练集用于用来拟合模型，通过设置分类器的参数，训练分类模型。通过训练集得出最优模型后，使用测试集进行模型预测，即可以把测试集当做从来不存在的数据集，当已经确定模型参数后，使用测试集进行模型性能评价。

(1)编写函数将数据集分为70%的训练集和30%的测试集

def split_data(data,train_size):

# Determine the number of observations we have in our entire data set:

length = len(data)

# Create a list of integer indices ranging over our number of observations:

indices = list(range(length))

# Use numpy's random.shuffle() function to randomly shuffle over our index:

np.random.shuffle(indices)

# Create a list for the first 70% of the shuffled indices and set to training:

train_indices = indices[0:int(length * train_size)]

# Create a list for the remaining 30% of the shuffled indices and set to testing:

test_indices = indices[int(length * train_size):]

# Use the list of training indices to find the corresponding data entries:

train = data.iloc[train_indices]

# Use the list of testing indices to find the corresponding data entries:

# YOUR CODE GOES HERE

test = data.iloc[test_indices]

# Return two dataframes, one with the testing data and one with the training data:

return train, test

We will now run the function and see if it returns actually what we want:

train_size=0.7

iris_train,iris_test = split_data(iris,train_size)

# Return the dimensions of our training dataframe after using the split_data function:

# YOUR CODE GOES HERE

iris_train.shape

(2)用sklearn包中的 train_test_split()函数将数据分为训练集和测试集

train, test = train_test_split(iris, test_size=0.3)

print(train.shape,test.shape)

4. 手动实现KNN算法

def knn_algorithm(train, test, k):

# 建立一个空列表，用来装预测值

predictions = []

# 用来预测的列

predictor_cols = [col for col in train.columns if col != 'target']

# 分别分离训练集和测试集的自变量和预测变量

train_x = train[predictor_cols]

train_y = train['target']

test_x = test[predictor_cols]

test_y = test['target']

for index, row in test_x.iterrows():

# 对于每个测试点，存储所有训练点和测试点之间的欧式距离的平方

vec_distances = pd.DataFrame((train_x.values - row.values)**2, index=train.index, columns = train_x.columns)

# 对每个测试点，按行进行求以获得所有训练点与该点之间的欧式距离的平方之和

distances = vec_distances.sum(axis = 1)

# 对distances排序(升序)并取前k个点

nearest_k = distances.sort_values().iloc[:k]

# 根据nearest k，选取相应的test_y的均值

k_mean = train_y[nearest_k.index].mean()

# 将k_mean的值添加到predictions的列表中

predictions.append(k_mean)

# 构建一个数据框

predict = test.copy()

predict['target'] = pd.Series(predictions, index=test.index)

return predict

令k=5来运行上述的KNN算法

k = 5

predicted_knn = knn_algorithm(iris_train, iris_test, k)

predicted_knn

预测出来后，我们还要评估我们预测的准确性(k=5)，下面编写评估函数：

def evaluate(predicted, true):

# 计算误差的平方

squared_error = (predicted['target'] - true['target'])**2

# 计算回归决定系数(拟合优度)

error_var = squared_error.sum()

sample_var = ((true['target'] - true['target'].mean())**2).sum()

r = (1 - (error_var / sample_var))

return r

evaluate(predicted_knn, iris_test)

我们看到，当k=5时，手动实现的KNN算法的 回归决定系数为R^2 = 0.9745，能较准确地进行预测

(2)用sklearn来实现KNN算法

# 用sklearn包的 train_test_split()功能将数据分成训练集和测试集

train, test = train_test_split(iris, test_size=.3)

x_train, x_test = train[features_cols], test[features_cols]

y_train, y_test = train['target'], test['target']

# 设置KNN参数:

k = 10

# 首先，构建分类对象

neighbors = KNeighborsClassifier(n_neighbors=k)

# 然后，用x_train作为训练集，用y_train作为测试集，拟合模型

neighbors.fit(x_train, y_train)

# 利用拟合好的模型进行预测

prediction_knn = neighbors.predict(x_test)

# 评估模型

r = neighbors.score(x_test, y_test)

r

5.用线性回归进行预测

(1)进行模型拟合

import statsmodels.api as sm

# 用最小二乘法进行模型拟合:

model = sm.OLS(y_train.values, x_train)

regr = model.fit()

# 返回模型的参数

regr.params

(2)计算利用拟合模型预测的均方误差

np.mean((regr.predict(x_test)-y_test)**2)

(3)描述拟合模型

regr.summary()

有结果可知：R^2=0.973，说明拟合效果较好

predicted_knn = np.round(regr.predict(x_test))

pd.DataFrame(metrics.confusion_matrix(expected_knn, predicted_knn))

由上面结果可知，有一个被错误预测(1被预测为了2)

6. 用多元回归方法进行预测

(1)构建PolynomialFeatures对象并赋予维数

# Create the PolynomialFeatures object and specify the number of degrees:

degree = 2

poly = PolynomialFeatures(degree)

x_train_poly = poly.fit_transform(x_train)

x_test_poly = poly.fit_transform(x_test)

pd.DataFrame(x_train_poly).shape

(2)拟合模型

# Create a linear regression object

lg = LinearRegression()

# Fit our training data with polynomial features

lg.fit(x_train_poly, y_train)

# Obtain coefficients

lg.coef_

(3)对花的种类(”target“进行预测)

# Predict from our fitted polynomial regression model based on our test set.

poly.predicted = lg.predict(x_test_poly)

poly.predicted

(4)计算预测结果的均方误差

# Computer the Mean Squared Error for Polynomial Regression:

np.mean((poly.predicted-y_test)**2)

(5)评估模型

predicted_knn = np.round(poly.predicted)

pd.DataFrame(metrics.confusion_matrix(expected_knn, predicted_knn))

分析：和线性回归结果相差不多，有一个被错误预测(1被错误预测成了2)