详细请参照theano的官方文档，http://deeplearning.net/software/theano/library/tensor/nnet/nnet.html#tensor.nnet.categorical_crossentropy

优化函数：

keras文档： 

下面是一个自定义的优化器的一个demo：

model = Sequential()
model.add(Dense(64, init='uniform', input_dim=10))
model.add(Activation('tanh'))
model.add(Activation('softmax'))  sgd = SGD(lr=0.1, decay=1e-6, momentum=0.9, nesterov=True)
model.compile(loss='mean_squared_error', optimizer=sgd) 

下面是默认的SGD优化器的使用和参数介绍：

model.compile(loss='mean_squared_error', optimizer='sgd')

keras.optimizers.SGD(lr=0.01, momentum=0.0, decay=0.0, nesterov=False)
Stochastic gradient descent, with support for momentum, learning rate decay, and Nesterov momentum.
<span style="color:#ff0000;">随机梯度下降算法， 看到这个名字是不是很熟了</span>
Argumentslr: float >= 0. Learning rate.
momentum: float >= 0. Parameter updates momentum.（<span class="s1" style="font-family: Arial, Helvetica, sans-serif;">参数更新的动量， 我也不知道啥意思</span><span style="font-family: Arial, Helvetica, sans-serif;">）</span>
decay: float >= 0. Learning rate decay over each update.（学习率衰减量，具体是每次都衰减）
nesterov: boolean. Whether to apply Nesterov momentum.（是否启用nesterov动量）

上面好多名词都不明白什么意思，应该把实现的源码贴出来大家看一下

class SGD(Optimizer):'''Stochastic gradient descent, with support for momentum,learning rate decay, and Nesterov momentum.# Argumentslr: float >= 0. Learning rate.momentum: float >= 0. Parameter updates momentum.decay: float >= 0. Learning rate decay over each update.nesterov: boolean. Whether to apply Nesterov momentum.'''def __init__(self, lr=0.01, momentum=0., decay=0., nesterov=False,*args, **kwargs):super(SGD, self).__init__(**kwargs)self.__dict__.update(locals())self.iterations = K.variable(0.)self.lr = K.variable(lr)self.momentum = K.variable(momentum)self.decay = K.variable(decay)def get_updates(self, params, constraints, loss):grads = self.get_gradients(loss, params)lr = self.lr * (1. / (1. + self.decay * self.iterations))self.updates = [(self.iterations, self.iterations + 1.)]# momentumself.weights = [K.variable(np.zeros(K.get_value(p).shape)) for p in params]for p, g, m in zip(params, grads, self.weights):v = self.momentum * m - lr * g  # velocityself.updates.append((m, v))if self.nesterov:new_p = p + self.momentum * v - lr * gelse:new_p = p + v# apply constraintsif p in constraints:c = constraints[p]new_p = c(new_p)self.updates.append((p, new_p))return self.updatesdef get_config(self):return {"name": self.__class__.__name__,"lr": float(K.get_value(self.lr)),"momentum": float(K.get_value(self.momentum)),"decay": float(K.get_value(self.decay)),"nesterov": self.nesterov}

源码就不解释了，接着说优化函数，上面以最常用的随机 梯度下降法（sgd）说了怎么用，下面再说说其他的。

这篇文章写的已经很好了：  http://blog.csdn.net/luo123n/article/details/48239963

时间紧迫，以后在学习