简介

Andrew Ng 深度学习课程的第一周第三次作业是实现一个浅层神经网络，课程方给的框架很有意思，但该作业的输出是类别，我想实践一下该网络能否改造用来解决回归问题，具体而言是拟合一个函数z = x²+y² ,尝试之后发现效果不是很稳定，容易收敛到局部极小值，但拟合效果大体上还能接受，简要分享，后续准备改用随机梯度下降方法来跳出局部极小值。

神经网络结构

因为是二元函数，所以输入层维度固定为2，输出层维度为1，输出层未采用激活函数，隐藏层只用了一层，设置了20个神经元，激活函数为tanh。误差函数使用均方误差函数，学习率设置为0.2。

拟合效果

绿色点是原始曲面，红色点是拟合曲面。

这样看可能看不出拟合的效果，放一张单独只有拟合曲面的图。

大体上还是拟合出来了，放上误差曲线。每1000次迭代取一次误差，不是很光滑，但能说明问题。

代码

下面是基本代码，框架用的是作业中提供的框架，针对连续数值的输出，我对网络结构和前向传播、后向传播做了适当修改。

# Package imports
import numpy as np
import matplotlib.pyplot as plt
import random
from matplotlib import cm
import mpl_toolkits.mplot3d
np.random.seed(2)def layer_sizes(X, Y):"""Arguments:X -- input dataset of shape (input size, number of examples)Y -- labels of shape (output size, number of examples)Returns:n_x -- the size of the input layern_h -- the size of the hidden layern_y -- the size of the output layer"""### START CODE HERE ### (≈ 3 lines of code)n_x = X.shape[0] # size of input layern_y = Y.shape[0] # size of output layer### END CODE HERE ###return (n_x,  n_y)def initialize_parameters(n_x, n_h, n_y):"""Argument:n_x -- size of the input layern_h -- size of the hidden layern_y -- size of the output layerReturns:params -- python dictionary containing your parameters:W1 -- weight matrix of shape (n_h, n_x)b1 -- bias vector of shape (n_h, 1)W2 -- weight matrix of shape (n_y, n_h)b2 -- bias vector of shape (n_y, 1)"""np.random.seed(20) ### START CODE HERE ### (≈ 4 lines of code)W1 = np.random.randn(n_h, n_x)* 0.01b1 = np.zeros((n_h, 1))W2 = np.random.randn(n_y, n_h)* 0.01b2 = np.zeros((n_y, 1))### END CODE HERE ###parameters = {"W1": W1,"b1": b1,"W2": W2,"b2": b2}return parametersdef forward_propagation(X, parameters):"""Argument:X -- input data of size (n_x, m)parameters -- python dictionary containing your parameters (output of initialization function)Returns:A2 -- The sigmoid output of the second activationcache -- a dictionary containing "Z1", "A1", "Z2" and "A2""""# Retrieve each parameter from the dictionary "parameters"### START CODE HERE ### (≈ 4 lines of code)W1 = parameters['W1']b1 = parameters['b1']W2 = parameters['W2']b2 = parameters['b2']### END CODE HERE #### Implement Forward Propagation to calculate A2 (probabilities)### START CODE HERE ### (≈ 4 lines of code)Z1 = np.dot(W1, X) + b1A1 = np.tanh(Z1)Z2 = np.dot(W2, A1) + b2A2 = Z2### END CODE HERE ###cache = {"Z1": Z1,"A1": A1,"Z2": Z2,"A2": A2}return A2, cachedef compute_cost(A2, Y, parameters):"""Computes the cross-entropy cost given in equation (13)Arguments:A2 -- The sigmoid output of the second activation, of shape (1, number of examples)Y -- "true" labels vector of shape (1, number of examples)parameters -- python dictionary containing your parameters W1, b1, W2 and b2Returns:cost -- cross-entropy cost given equation (13)"""m = Y.shape[1] # number of examplecost = np.sum(np.square(A2-Y))/mcost = np.squeeze(cost)     # makes sure cost is the dimension we expect. # E.g., turns [[17]] into 17 return costdef backward_propagation(parameters, cache, X, Y):"""Implement the backward propagation using the instructions above.Arguments:parameters -- python dictionary containing our parameters cache -- a dictionary containing "Z1", "A1", "Z2" and "A2".X -- input data of shape (2, number of examples)Y -- "true" labels vector of shape (1, number of examples)Returns:grads -- python dictionary containing your gradients with respect to different parameters"""m = X.shape[1]# First, retrieve W1 and W2 from the dictionary "parameters".### START CODE HERE ### (≈ 2 lines of code)W1 = parameters['W1']W2 = parameters['W2']### END CODE HERE #### Retrieve also A1 and A2 from dictionary "cache".### START CODE HERE ### (≈ 2 lines of code)A1 = cache['A1']A2 = cache['A2']### END CODE HERE #### Backward propagation: calculate dW1, db1, dW2, db2. ### START CODE HERE ### (≈ 6 lines of code, corresponding to 6 equations on slide above)dZ2 =( A2 - Y)dW2 = 1/m * np.dot(dZ2, A1.T)    db2 = 1/m * np.sum(dZ2, axis=1, keepdims=True)dZ1 = np.dot(W2.T, dZ2) * (1 - np.power(A1, 2))dW1 = 1/m * np.dot(dZ1, X.T)db1 = 1/m * np.sum(dZ1, axis=1, keepdims=True)### END CODE HERE ###grads = {"dW1": dW1,"db1": db1,"dW2": dW2,"db2": db2}return gradsdef update_parameters(parameters, grads, learning_rate = 0.2):"""Updates parameters using the gradient descent update rule given aboveArguments:parameters -- python dictionary containing your parameters grads -- python dictionary containing your gradients Returns:parameters -- python dictionary containing your updated parameters """# Retrieve each parameter from the dictionary "parameters"### START CODE HERE ### (≈ 4 lines of code)W1 = parameters['W1']b1 = parameters['b1']W2 = parameters['W2']b2 = parameters['b2']### END CODE HERE #### Retrieve each gradient from the dictionary "grads"### START CODE HERE ### (≈ 4 lines of code)dW1 = grads["dW1"]db1 = grads["db1"]dW2 = grads["dW2"]db2 = grads["db2"]## END CODE HERE #### Update rule for each parameter### START CODE HERE ### (≈ 4 lines of code)W1 -= learning_rate * dW1b1 -= learning_rate * db1W2 -= learning_rate * dW2b2 -= learning_rate * db2### END CODE HERE ###parameters = {"W1": W1,"b1": b1,"W2": W2,"b2": b2}return parametersdef nn_model(X, Y, n_h, num_iterations = 10000, print_cost=False):"""Arguments:X -- dataset of shape (2, number of examples)Y -- labels of shape (1, number of examples)n_h -- size of the hidden layernum_iterations -- Number of iterations in gradient descent loopprint_cost -- if True, print the cost every 1000 iterationsReturns:parameters -- parameters learnt by the model. They can then be used to predict."""np.random.seed(3)n_x = layer_sizes(X, Y)[0]n_y = layer_sizes(X, Y)[1]# Initialize parameters, then retrieve W1, b1, W2, b2. Inputs: "n_x, n_h, n_y". Outputs = "W1, b1, W2, b2, parameters".### START CODE HERE ### (≈ 5 lines of code)n_x, n_y = layer_sizes(X, Y)parameters = initialize_parameters(n_x, n_h, n_y)W1 = parameters['W1']b1 = parameters['b1']W2 = parameters['W2']b2 = parameters['b2']### END CODE HERE #### Loop (gradient descent)for i in range(0, num_iterations):### START CODE HERE ### (≈ 4 lines of code)# Forward propagation. Inputs: "X, parameters". Outputs: "A2, cache".A2, cache = forward_propagation(X, parameters)# Cost function. Inputs: "A2, Y, parameters". Outputs: "cost".cost = compute_cost(A2, Y, parameters)# Backpropagation. Inputs: "parameters, cache, X, Y". Outputs: "grads".grads = backward_propagation(parameters, cache, X, Y)# Gradient descent parameter update. Inputs: "parameters, grads". Outputs: "parameters".parameters = update_parameters(parameters, grads)### END CODE HERE #### Print the cost every 1000 iterationsif print_cost and i % 1000 == 0:print ("Cost after iteration %i: %f" %(i, cost))return parametersdef predict(parameters, X):"""Using the learned parameters, predicts a class for each example in XArguments:parameters -- python dictionary containing your parameters X -- input data of size (n_x, m)Returnspredictions -- vector of predictions of our model (red: 0 / blue: 1)"""### START CODE HERE ### (≈ 2 lines of code)A2, cache = forward_propagation(X, parameters)predictions = A2### END CODE HERE ###return predictions#训练集和测试集生成
if __name__ == '__main__':train_data = np.zeros((10000,3))for i in range(10000):train_data[i][0] = random.uniform(-5, 5)train_data[i][1] = random.uniform(-5, 5)train_data[i][2] = train_data[i][0]**2 + train_data[i][1]**2X = train_data[:,0:2].Ty = train_data[:,2].reshape(10000,1).Tparameters = nn_model(X, y, n_h = 20, num_iterations = 20000, print_cost=True)test_data = np.zeros((2000,4))for i in range(2000):test_data[i][0] = random.uniform(-5, 5)test_data[i][1] = random.uniform(-5, 5)test_data[i][2] = test_data[i][0]**2 + test_data[i][1]**2x = test_data[:,0:2].Tpredictions = predict(parameters, x)for i in range(2000):test_data[i][3] = predictions[0][i]ax = plt.subplot(111, projection='3d')  # 创建一个三维的绘图工程#将数据点分成三部分画，在颜色上有区分度ax.scatter(test_data[:,0], test_data[:,1], test_data[:,2], c='g')ax.scatter(test_data[:,0], test_data[:,1], test_data[:,3], c='r')plt.legend()plt.show()

神经网络拟合二元函数曲面实践相关推荐

python 二元函数绘制_Python绘制二元函数曲面
微实践:绘制二元函数曲面我们将演示如何借助于ufunc的广播运算计算下述二元函数的在一个xy平面上的值并将其绘制成3D曲面.其中,x和y的取值范围均为[-2,+2]. 为了达到目的,我们需要一个二维 ...
python二元函数图像在线绘制_Python绘制二元函数曲面
微实践:绘制二元函数曲面我们将演示如何借助于ufunc的广播运算计算下述二元函数的在一个xy平面上的值并将其绘制成3D曲面.其中,x和y的取值范围均为[-2,+2]. 为了达到目的,我们需要一个二维 ...
利用keras搭建神经网络拟合非线性函数
神经网络有着一个非常奇妙的结构,它的数学原理虽然相对简单,但是能做的事情却不少,数学家已经证明,具有2层(输入层除外)和非线性激活函数的神经网络,只要在这些层中有足够多的神经元,就可以近似任何函数(严 ...
如何用matlab拟合二元函数,怎么拟合二元函数？用什么软件比较容易实现？
满意答案 jht20035279 2013.08.15 采纳率:48% 等级:12 已帮助:11337人告诉你吧不用什么matlab,用mathematica就非常方便给你举个例子吧把程 ...
tensorflow神经网络拟合非线性函数与操作指南
本实验通过建立一个含有两个隐含层的BP神经网络,拟合具有二次函数非线性关系的方程,并通过可视化展现学习到的拟合曲线,同时随机给定输入值,输出预测值,最后给出一些关键的提示. 源代码如下: # -*- ...
matlab线性拟合二元函数,求助：怎么用已知数据进行matlab二元二次函数拟合
本帖最后由 l68780230 于 2013-11-26 15:50 编辑 stats01 发表于 2013-11-26 11:24 对于x只有两个点的数据,太少了.若给出全部数据,可以试试. A=[ ...
matlab拟合二元函数图,求助大神。。matlab拟合二元函数，求解系数。函数形式已知...
该楼层疑似违规已被系统折叠隐藏此楼查看此楼 y x2 x1 449083.6 12.96479 9503.75 442743.7 12.12091 9529.59 446092.7 12.29292 ...
使用python绘制3D二元函数曲面
目标函数,其中x,y取值范围均在[-2,2] 代码如下: import numpy as np from matplotlib import pyplot as plt from matplotlib ...
神经网络拟合曲面（tensorflow）
神经网络拟合曲面(tensorflow) 神经网络有拟合高维度向量空间的曲面的强大能力,在本文中利用深度学习对不可知二元函数进行拟合,可以自拟函数关系式,改一下y_data输入就行了,话不多说直接上代 ...

神经网络拟合二元函数曲面实践

简介

神经网络结构

拟合效果

代码

神经网络拟合二元函数曲面实践相关推荐

最新文章

热门文章