tf.keras.activations.softmax 激活函数示例

文章目录

softmax 将值的向量转换为概率分布
创建数组
计算公式
计算过程
tf.keras.activations.softmax实现
numpy实现

softmax 将值的向量转换为概率分布

import tensorflow as tf
import numpy as np

创建数组

a1 = np.arange(0,6).reshape(2,3).astype(float)
print(a1)

[[0. 1. 2.][3. 4. 5.]]

计算公式

softmax=eai−max∑eai−maxsoftmax = \frac{e^{a_{i}-max}}{\sum e^{a_{i}-max}}softmax=∑eai−maxeai−max

aia_{i}ai : 数组中的元素

e ：自然数

max : 数组中的最大数

计算过程

[[e0−5e0−5+e1−5+e2−5,e1−5e0−5+e1−5+e2−5,e2−5e0−5+e1−5+e2−5][e3−5e3−5+e4−5+e5−5,e4−5e3−5+e4−5+e5−5,e5−5e3−5+e4−5+e5−5]]=[[0.09003057,0.24472847,0.66524096][0.09003057,0.24472847,0.66524096]][[\frac{e^{0-5}}{e^{0-5}+e^{1-5}+e^{2-5}} ,\frac{e^{1-5}}{e^{0-5}+e^{1-5}+e^{2-5}},\frac{e^{2-5}}{e^{0-5}+e^{1-5}+e^{2-5}}][\frac{e^{3-5}}{e^{3-5}+e^{4-5}+e^{5-5}} ,\frac{e^{4-5}}{e^{3-5}+e^{4-5}+e^{5-5}},\frac{e^{5-5}}{e^{3-5}+e^{4-5}+e^{5-5}}]]=[[0.09003057 ,0.24472847 ,0.66524096] [0.09003057 ,0.24472847 ,0.66524096]][[e0−5+e1−5+e2−5e0−5,e0−5+e1−5+e2−5e1−5,e0−5+e1−5+e2−5e2−5][e3−5+e4−5+e5−5e3−5,e3−5+e4−5+e5−5e4−5,e3−5+e4−5+e5−5e5−5]]=[[0.09003057,0.24472847,0.66524096][0.09003057,0.24472847,0.66524096]]

tf.keras.activations.softmax实现

t2 = tf.convert_to_tensor(a1)
print(t2)
t2_softmax = tf.keras.activations.softmax(t2)
print(t2_softmax)

tf.Tensor(
[[0. 1. 2.][3. 4. 5.]], shape=(2, 3), dtype=float64)
tf.Tensor(
[[0.09003057 0.24472847 0.66524096][0.09003057 0.24472847 0.66524096]], shape=(2, 3), dtype=float64)

print(tf.reduce_sum(t2_softmax[0, :]))

tf.Tensor(0.9999999999999999, shape=(), dtype=float64)

numpy实现

a1_max = np.max(a1)
print(a1_max)

5.0

a1_reduce_max = a1-a1_max
print(a1_reduce_max)

[[-5. -4. -3.][-2. -1.  0.]]

a1_exp = np.exp(a1_reduce_max)
print(a1_exp)

[[0.00673795 0.01831564 0.04978707][0.13533528 0.36787944 1.        ]]

a1_exp_sum = np.sum(a1_exp,axis = 1)
print(a1_exp_sum)

[0.07484065 1.50321472]

a1_exp_sum = a1_exp_sum.reshape(-1, 1)
a1_exp_sum_pad = np.pad(a1_exp_sum,  # 数据((0, 0),(1, 1),),  # 最后一维，左边、右边'reflect'  # 填充边缘对称值
)
print(a1_exp_sum_pad)

[[0.07484065 0.07484065 0.07484065][1.50321472 1.50321472 1.50321472]]

# a1_softmax = a1_exp/a1_exp_sum
a1_softmax = np.divide(a1_exp,a1_exp_sum)
print(a1_softmax)

[[0.09003057 0.24472847 0.66524096][0.09003057 0.24472847 0.66524096]]

print(np.sum(a1_softmax,axis=-1))

[1. 1.]