当前时间是2021/6/16，matchzoo作为非常强大的文本匹配库，当前未更新到TF2.4以上版本，无法使用本机3090加速，为此我将源码反向推导实现，使用TF2.4形式实现这些模型

"""
作者英俊
QQ 2227495940
所有权：西安建筑科技大学草堂校区 信控楼704实验室
"""
"暂定只能扣13个模型出来"

'暂定只能扣13个模型出来'

## Denseline```python
input1 = Input(name='text_left', shape=(max_len,))
input2 = Input(name='text_right', shape=(max_len,))
# embedding = Embedding(vocab_size, embedding_size)
# sent1_embed=embedding(input1)
# sent2_embed=embedding(input2)
concatenate_1=concatenate([input1,input2],axis=-1)
dense1=Dense(300)(concatenate_1)
dense2=Dense(300)(dense1)
dense3=Dense(128)(dense2)
outputs=Dense(1,activation='linear',name="output")(dense3)
model=Model(inputs=[input1,input2],outputs=outputs,name="Denseline")
model.summary()

Model: "Denseline"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to
==================================================================================================
text_left (InputLayer)          [(None, 15)]         0
__________________________________________________________________________________________________
text_right (InputLayer)         [(None, 15)]         0
__________________________________________________________________________________________________
concatenate (Concatenate)       (None, 30)           0           text_left[0][0]                  text_right[0][0]
__________________________________________________________________________________________________
dense (Dense)                   (None, 300)          9300        concatenate[0][0]
__________________________________________________________________________________________________
dense_1 (Dense)                 (None, 300)          90300       dense[0][0]
__________________________________________________________________________________________________
dense_2 (Dense)                 (None, 128)          38528       dense_1[0][0]
__________________________________________________________________________________________________
output (Dense)                  (None, 1)            129         dense_2[0][0]
==================================================================================================
Total params: 138,257
Trainable params: 138,257
Non-trainable params: 0
__________________________________________________________________________________________________

model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])# 各种callback函数
tb_callback = tf.keras.callbacks.TensorBoard(logdir, histogram_freq=1)
#如果accuracy在25个epoch后没有提升，学习率减半。
lr_callback = tf.keras.callbacks.ReduceLROnPlateau(monitor="accuracy",factor = 0.5, patience = 25)
#当accuracy在30个epoch后没有提升，则提前终止训练。
stop_callback = tf.keras.callbacks.EarlyStopping(monitor = "accuracy", patience= 30)#获取当前的地址
path=os.path.join(os.getcwd(),'outputs\model',model.name)import os
dirs = pathif not os.path.exists(dirs):os.makedirs(dirs)# 保留最好的模型
mc_callback=tf.keras.callbacks.ModelCheckpoint(filepath=os.path.join(path,'weights.hdf5'),monitor='val_accuracy',verbose=1, save_best_only=True)# 把训练轮结果数据流到 csv 文件的回调函数。
# url=os.path.join(os.getcwd(),'outputs')csv_callback=tf.keras.callbacks.CSVLogger(os.path.join(path,'training.log'))callbacks_list = [tb_callback,lr_callback,stop_callback,mc_callback,csv_callback]t = time()
history=model.fit(x=[sent1_train,sent2_train],y=train_labels,validation_data=([sent1_val,sent2_val],val_labels),epochs=50,\callbacks = callbacks_list)
print(model.name+'运行时间: {} mins'.format(round((time() - t) / 60, 2)))model.save(os.path.join(path,'save',model.name))
print('保存成功','保存地址在',os.path.join(path,'save'))

Epoch 1/50
63/63 [==============================] - 2s 15ms/step - loss: 3.2004 - accuracy: 0.7902 - val_loss: 2.8325 - val_accuracy: 0.8164Epoch 00001: val_accuracy improved from -inf to 0.81637, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\Denseline\weights.hdf5
Epoch 2/50
63/63 [==============================] - 0s 4ms/step - loss: 3.2150 - accuracy: 0.7916 - val_loss: 2.8325 - val_accuracy: 0.8164Epoch 00002: val_accuracy did not improve from 0.81637
Epoch 3/50
63/63 [==============================] - 0s 4ms/step - loss: 2.9334 - accuracy: 0.8098 - val_loss: 2.8325 - val_accuracy: 0.8164Epoch 00003: val_accuracy did not improve from 0.81637
Epoch 4/50
63/63 [==============================] - 0s 4ms/step - loss: 3.1602 - accuracy: 0.7951 - val_loss: 2.8325 - val_accuracy: 0.8164Epoch 00004: val_accuracy did not improve from 0.81637
Epoch 5/50
63/63 [==============================] - 0s 4ms/step - loss: 3.1563 - accuracy: 0.7954 - val_loss: 2.8325 - val_accuracy: 0.8164Epoch 00005: val_accuracy did not improve from 0.81637
Epoch 6/50
63/63 [==============================] - 0s 4ms/step - loss: 2.7483 - accuracy: 0.8218 - val_loss: 2.8325 - val_accuracy: 0.8164Epoch 00006: val_accuracy did not improve from 0.81637
Epoch 7/50
63/63 [==============================] - 0s 4ms/step - loss: 2.9518 - accuracy: 0.8086 - val_loss: 2.8325 - val_accuracy: 0.8164Epoch 00007: val_accuracy did not improve from 0.81637
Epoch 8/50
63/63 [==============================] - 0s 4ms/step - loss: 3.0769 - accuracy: 0.8005 - val_loss: 2.8325 - val_accuracy: 0.8164Epoch 00008: val_accuracy did not improve from 0.81637
Epoch 9/50
63/63 [==============================] - ETA: 0s - loss: 2.9823 - accuracy: 0.80 - 0s 4ms/step - loss: 2.9761 - accuracy: 0.8071 - val_loss: 2.8325 - val_accuracy: 0.8164Epoch 00009: val_accuracy did not improve from 0.81637
Epoch 10/50
63/63 [==============================] - 0s 4ms/step - loss: 2.6330 - accuracy: 0.8293 - val_loss: 2.8325 - val_accuracy: 0.8164Epoch 00010: val_accuracy did not improve from 0.81637
Epoch 11/50
63/63 [==============================] - 0s 4ms/step - loss: 3.2002 - accuracy: 0.7925 - val_loss: 2.8325 - val_accuracy: 0.8164Epoch 00011: val_accuracy did not improve from 0.81637
Epoch 12/50
63/63 [==============================] - 0s 6ms/step - loss: 2.7750 - accuracy: 0.8201 - val_loss: 2.8325 - val_accuracy: 0.8164Epoch 00012: val_accuracy did not improve from 0.81637
Epoch 13/50
63/63 [==============================] - 0s 4ms/step - loss: 3.1518 - accuracy: 0.7957 - val_loss: 2.8325 - val_accuracy: 0.8164Epoch 00013: val_accuracy did not improve from 0.81637
Epoch 14/50
63/63 [==============================] - 0s 4ms/step - loss: 3.0377 - accuracy: 0.8031 - val_loss: 2.8325 - val_accuracy: 0.8164Epoch 00014: val_accuracy did not improve from 0.81637
Epoch 15/50
63/63 [==============================] - 0s 4ms/step - loss: 2.9777 - accuracy: 0.8070 - val_loss: 2.8325 - val_accuracy: 0.8164Epoch 00015: val_accuracy did not improve from 0.81637
Epoch 16/50
63/63 [==============================] - 0s 4ms/step - loss: 2.9072 - accuracy: 0.8115 - val_loss: 2.8325 - val_accuracy: 0.8164Epoch 00016: val_accuracy did not improve from 0.81637
Epoch 17/50
63/63 [==============================] - 0s 4ms/step - loss: 2.6993 - accuracy: 0.8250 - val_loss: 2.8325 - val_accuracy: 0.8164Epoch 00017: val_accuracy did not improve from 0.81637
Epoch 18/50
63/63 [==============================] - 0s 4ms/step - loss: 3.1956 - accuracy: 0.7928 - val_loss: 2.8325 - val_accuracy: 0.8164Epoch 00018: val_accuracy did not improve from 0.81637
Epoch 19/50
63/63 [==============================] - 0s 4ms/step - loss: 3.1464 - accuracy: 0.7960 - val_loss: 2.8325 - val_accuracy: 0.8164Epoch 00019: val_accuracy did not improve from 0.81637
Epoch 20/50
63/63 [==============================] - 0s 4ms/step - loss: 3.0472 - accuracy: 0.8024 - val_loss: 2.8325 - val_accuracy: 0.8164Epoch 00020: val_accuracy did not improve from 0.81637
Epoch 21/50
63/63 [==============================] - 0s 4ms/step - loss: 2.9581 - accuracy: 0.8082 - val_loss: 2.8325 - val_accuracy: 0.8164Epoch 00021: val_accuracy did not improve from 0.81637
Epoch 22/50
63/63 [==============================] - 0s 4ms/step - loss: 3.1285 - accuracy: 0.7972 - val_loss: 2.8325 - val_accuracy: 0.8164Epoch 00022: val_accuracy did not improve from 0.81637
Epoch 23/50
63/63 [==============================] - 0s 4ms/step - loss: 3.0444 - accuracy: 0.8026 - val_loss: 2.8325 - val_accuracy: 0.8164Epoch 00023: val_accuracy did not improve from 0.81637
Epoch 24/50
63/63 [==============================] - 0s 4ms/step - loss: 3.0231 - accuracy: 0.8040 - val_loss: 2.8325 - val_accuracy: 0.8164Epoch 00024: val_accuracy did not improve from 0.81637
Epoch 25/50
63/63 [==============================] - 0s 4ms/step - loss: 3.0962 - accuracy: 0.7993 - val_loss: 2.8325 - val_accuracy: 0.8164Epoch 00025: val_accuracy did not improve from 0.81637
Epoch 26/50
63/63 [==============================] - 0s 4ms/step - loss: 2.9691 - accuracy: 0.8075 - val_loss: 2.8325 - val_accuracy: 0.8164Epoch 00026: val_accuracy did not improve from 0.81637
Epoch 27/50
63/63 [==============================] - 0s 4ms/step - loss: 2.7456 - accuracy: 0.8220 - val_loss: 2.8325 - val_accuracy: 0.8164Epoch 00027: val_accuracy did not improve from 0.81637
Epoch 28/50
63/63 [==============================] - 0s 4ms/step - loss: 3.0548 - accuracy: 0.8020 - val_loss: 2.8325 - val_accuracy: 0.8164Epoch 00028: val_accuracy did not improve from 0.81637
Epoch 29/50
63/63 [==============================] - 0s 4ms/step - loss: 2.8902 - accuracy: 0.8126 - val_loss: 2.8325 - val_accuracy: 0.8164Epoch 00029: val_accuracy did not improve from 0.81637
Epoch 30/50
63/63 [==============================] - 0s 4ms/step - loss: 2.8348 - accuracy: 0.8162 - val_loss: 2.8325 - val_accuracy: 0.8164Epoch 00030: val_accuracy did not improve from 0.81637
Epoch 31/50
63/63 [==============================] - 0s 4ms/step - loss: 2.7539 - accuracy: 0.8215 - val_loss: 2.8325 - val_accuracy: 0.8164Epoch 00031: val_accuracy did not improve from 0.81637
Epoch 32/50
63/63 [==============================] - 0s 4ms/step - loss: 2.9578 - accuracy: 0.8082 - val_loss: 2.8325 - val_accuracy: 0.8164Epoch 00032: val_accuracy did not improve from 0.81637
Denseline运行时间: 0.21 mins
INFO:tensorflow:Assets written to: H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\Denseline\save\Denseline\assets
保存成功 保存地址在 H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\Denseline\save

plot_metric(history,"loss")
plot_metric(history,"accuracy")

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-QeHYE8H6-1623898674919)(output_41_0.svg)]

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-2WMSiOqS-1623898674920)(output_41_1.svg)]

# 以表格形式展示
dfhistory = pd.DataFrame(history.history)
dfhistory.index = range(1,len(dfhistory) + 1)
dfhistory.index.name = 'epoch'
dfhistory.to_csv(os.path.join(path,'save',model.name,'.csv'))dfhistory

	loss	accuracy	val_loss	val_accuracy	lr
epoch
1	3.007824	0.8045	2.832526	0.816367	0.0010
2	2.984728	0.8065	2.832526	0.816367	0.0010
3	2.984727	0.8065	2.832526	0.816367	0.0010
4	2.984727	0.8065	2.832526	0.816367	0.0010
5	2.984728	0.8065	2.832526	0.816367	0.0010
6	2.984727	0.8065	2.832526	0.816367	0.0010
7	2.984727	0.8065	2.832526	0.816367	0.0010
8	2.984727	0.8065	2.832526	0.816367	0.0010
9	2.984727	0.8065	2.832526	0.816367	0.0010
10	2.984728	0.8065	2.832526	0.816367	0.0010
11	2.984727	0.8065	2.832526	0.816367	0.0010
12	2.984728	0.8065	2.832526	0.816367	0.0010
13	2.984727	0.8065	2.832526	0.816367	0.0010
14	2.984727	0.8065	2.832526	0.816367	0.0010
15	2.984727	0.8065	2.832526	0.816367	0.0010
16	2.984728	0.8065	2.832526	0.816367	0.0010
17	2.984727	0.8065	2.832526	0.816367	0.0010
18	2.984727	0.8065	2.832526	0.816367	0.0010
19	2.984728	0.8065	2.832526	0.816367	0.0010
20	2.984727	0.8065	2.832526	0.816367	0.0010
21	2.984728	0.8065	2.832526	0.816367	0.0010
22	2.984728	0.8065	2.832526	0.816367	0.0010
23	2.984727	0.8065	2.832526	0.816367	0.0010
24	2.984728	0.8065	2.832526	0.816367	0.0010
25	2.984728	0.8065	2.832526	0.816367	0.0010
26	2.984727	0.8065	2.832526	0.816367	0.0010
27	2.984728	0.8065	2.832526	0.816367	0.0010
28	2.984728	0.8065	2.832526	0.816367	0.0005
29	2.984727	0.8065	2.832526	0.816367	0.0005
30	2.984728	0.8065	2.832526	0.816367	0.0005
31	2.984727	0.8065	2.832526	0.816367	0.0005
32	2.984727	0.8065	2.832526	0.816367	0.0005

Navie

query= Input(name='text_left', shape=(max_len,))
doc  = Input(name='text_right', shape=(max_len,))
x=concatenate([query,doc])
x_out = Dense(1, activation='linear')(x)
# self._backend = keras.Model(inputs=[query, doc], outputs=x_out)
model=Model(inputs=[query, doc],outputs=x_out,name="Navie")
model.summary()

Model: "Navie"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to
==================================================================================================
text_left (InputLayer)          [(None, 15)]         0
__________________________________________________________________________________________________
text_right (InputLayer)         [(None, 15)]         0
__________________________________________________________________________________________________
concatenate_1 (Concatenate)     (None, 30)           0           text_left[0][0]                  text_right[0][0]
__________________________________________________________________________________________________
dense_3 (Dense)                 (None, 1)            31          concatenate_1[0][0]
==================================================================================================
Total params: 31
Trainable params: 31
Non-trainable params: 0
__________________________________________________________________________________________________

model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])# 各种callback函数
tb_callback = tf.keras.callbacks.TensorBoard(logdir, histogram_freq=1)
#如果accuracy在25个epoch后没有提升，学习率减半。
lr_callback = tf.keras.callbacks.ReduceLROnPlateau(monitor="accuracy",factor = 0.5, patience = 25)
#当accuracy在30个epoch后没有提升，则提前终止训练。
stop_callback = tf.keras.callbacks.EarlyStopping(monitor = "accuracy", patience= 30)#获取当前的地址
path=os.path.join(os.getcwd(),'outputs\model',model.name)import os
dirs = pathif not os.path.exists(dirs):os.makedirs(dirs)# 保留最好的模型
mc_callback=tf.keras.callbacks.ModelCheckpoint(filepath=os.path.join(path,'weights.hdf5'),monitor='val_accuracy',verbose=1, save_best_only=True)# 把训练轮结果数据流到 csv 文件的回调函数。
# url=os.path.join(os.getcwd(),'outputs')csv_callback=tf.keras.callbacks.CSVLogger(os.path.join(path,'training.log'))callbacks_list = [tb_callback,lr_callback,stop_callback,mc_callback,csv_callback]t = time()
history=model.fit(x=[sent1_train,sent2_train],y=train_labels,validation_data=([sent1_val,sent2_val],val_labels),epochs=50,\callbacks = callbacks_list)
print(model.name+'运行时间: {} mins'.format(round((time() - t) / 60, 2)))model.save(os.path.join(path,'save',model.name))
print('保存成功','保存地址在',os.path.join(path,'save'))

Epoch 1/50
63/63 [==============================] - 1s 9ms/step - loss: 8.9142 - accuracy: 0.4155 - val_loss: 8.2654 - val_accuracy: 0.4571Epoch 00001: val_accuracy improved from -inf to 0.45709, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\Navie\weights.hdf5
Epoch 2/50
63/63 [==============================] - 0s 5ms/step - loss: 8.0908 - accuracy: 0.4679 - val_loss: 7.8712 - val_accuracy: 0.4830Epoch 00002: val_accuracy improved from 0.45709 to 0.48303, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\Navie\weights.hdf5
Epoch 3/50
63/63 [==============================] - 0s 4ms/step - loss: 7.5510 - accuracy: 0.5045 - val_loss: 7.6945 - val_accuracy: 0.4950Epoch 00003: val_accuracy improved from 0.48303 to 0.49501, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\Navie\weights.hdf5
Epoch 4/50
63/63 [==============================] - 0s 4ms/step - loss: 7.8960 - accuracy: 0.4823 - val_loss: 7.4436 - val_accuracy: 0.5130Epoch 00004: val_accuracy improved from 0.49501 to 0.51297, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\Navie\weights.hdf5
Epoch 5/50
63/63 [==============================] - 0s 4ms/step - loss: 7.1944 - accuracy: 0.5268 - val_loss: 7.1445 - val_accuracy: 0.5309Epoch 00005: val_accuracy improved from 0.51297 to 0.53094, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\Navie\weights.hdf5
Epoch 6/50
63/63 [==============================] - 0s 4ms/step - loss: 7.2356 - accuracy: 0.5242 - val_loss: 6.7534 - val_accuracy: 0.5549Epoch 00006: val_accuracy improved from 0.53094 to 0.55489, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\Navie\weights.hdf5
Epoch 7/50
63/63 [==============================] - 0s 4ms/step - loss: 6.7651 - accuracy: 0.5518 - val_loss: 6.5213 - val_accuracy: 0.5669Epoch 00007: val_accuracy improved from 0.55489 to 0.56687, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\Navie\weights.hdf5
Epoch 8/50
63/63 [==============================] - 0s 4ms/step - loss: 6.2155 - accuracy: 0.5932 - val_loss: 6.8991 - val_accuracy: 0.5489Epoch 00008: val_accuracy did not improve from 0.56687
Epoch 9/50
63/63 [==============================] - 0s 3ms/step - loss: 6.4371 - accuracy: 0.5759 - val_loss: 6.8685 - val_accuracy: 0.5509Epoch 00009: val_accuracy did not improve from 0.56687
Epoch 10/50
63/63 [==============================] - 0s 3ms/step - loss: 6.5154 - accuracy: 0.5739 - val_loss: 6.4209 - val_accuracy: 0.5788Epoch 00010: val_accuracy improved from 0.56687 to 0.57884, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\Navie\weights.hdf5
Epoch 11/50
63/63 [==============================] - 0s 4ms/step - loss: 5.8972 - accuracy: 0.6133 - val_loss: 5.5397 - val_accuracy: 0.6327Epoch 00011: val_accuracy improved from 0.57884 to 0.63273, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\Navie\weights.hdf5
Epoch 12/50
63/63 [==============================] - 0s 4ms/step - loss: 5.7758 - accuracy: 0.6185 - val_loss: 5.2374 - val_accuracy: 0.6567Epoch 00012: val_accuracy improved from 0.63273 to 0.65669, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\Navie\weights.hdf5
Epoch 13/50
63/63 [==============================] - 0s 4ms/step - loss: 5.1468 - accuracy: 0.6625 - val_loss: 5.3260 - val_accuracy: 0.6507Epoch 00013: val_accuracy did not improve from 0.65669
Epoch 14/50
63/63 [==============================] - 0s 4ms/step - loss: 5.1256 - accuracy: 0.6654 - val_loss: 5.3839 - val_accuracy: 0.6467Epoch 00014: val_accuracy did not improve from 0.65669
Epoch 15/50
63/63 [==============================] - 0s 3ms/step - loss: 5.1535 - accuracy: 0.6591 - val_loss: 5.0819 - val_accuracy: 0.6667Epoch 00015: val_accuracy improved from 0.65669 to 0.66667, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\Navie\weights.hdf5
Epoch 16/50
63/63 [==============================] - 0s 4ms/step - loss: 4.8616 - accuracy: 0.6807 - val_loss: 5.0814 - val_accuracy: 0.6647Epoch 00016: val_accuracy did not improve from 0.66667
Epoch 17/50
63/63 [==============================] - 0s 3ms/step - loss: 5.1588 - accuracy: 0.6602 - val_loss: 4.8977 - val_accuracy: 0.6786Epoch 00017: val_accuracy improved from 0.66667 to 0.67864, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\Navie\weights.hdf5
Epoch 18/50
63/63 [==============================] - 0s 4ms/step - loss: 4.8387 - accuracy: 0.6825 - val_loss: 4.8953 - val_accuracy: 0.6806Epoch 00018: val_accuracy improved from 0.67864 to 0.68064, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\Navie\weights.hdf5
Epoch 19/50
63/63 [==============================] - 0s 3ms/step - loss: 4.8239 - accuracy: 0.6855 - val_loss: 4.8953 - val_accuracy: 0.6806Epoch 00019: val_accuracy did not improve from 0.68064
Epoch 20/50
63/63 [==============================] - 0s 3ms/step - loss: 4.9532 - accuracy: 0.6770 - val_loss: 4.8953 - val_accuracy: 0.6806Epoch 00020: val_accuracy did not improve from 0.68064
Epoch 21/50
63/63 [==============================] - 0s 3ms/step - loss: 5.1150 - accuracy: 0.6663 - val_loss: 4.8967 - val_accuracy: 0.6806Epoch 00021: val_accuracy did not improve from 0.68064
Epoch 22/50
63/63 [==============================] - 0s 3ms/step - loss: 4.9409 - accuracy: 0.6763 - val_loss: 4.8350 - val_accuracy: 0.6846Epoch 00022: val_accuracy improved from 0.68064 to 0.68463, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\Navie\weights.hdf5
Epoch 23/50
63/63 [==============================] - 0s 4ms/step - loss: 4.4491 - accuracy: 0.7095 - val_loss: 4.5635 - val_accuracy: 0.7006Epoch 00023: val_accuracy improved from 0.68463 to 0.70060, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\Navie\weights.hdf5
Epoch 24/50
63/63 [==============================] - 0s 4ms/step - loss: 4.9084 - accuracy: 0.6793 - val_loss: 4.5003 - val_accuracy: 0.7066Epoch 00024: val_accuracy improved from 0.70060 to 0.70659, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\Navie\weights.hdf5
Epoch 25/50
63/63 [==============================] - 0s 4ms/step - loss: 4.5295 - accuracy: 0.7006 - val_loss: 4.5016 - val_accuracy: 0.7066Epoch 00025: val_accuracy did not improve from 0.70659
Epoch 26/50
63/63 [==============================] - 0s 4ms/step - loss: 4.4010 - accuracy: 0.7126 - val_loss: 4.5308 - val_accuracy: 0.7046Epoch 00026: val_accuracy did not improve from 0.70659
Epoch 27/50
63/63 [==============================] - 0s 4ms/step - loss: 4.5775 - accuracy: 0.7008 - val_loss: 4.4738 - val_accuracy: 0.7066Epoch 00027: val_accuracy did not improve from 0.70659
Epoch 28/50
63/63 [==============================] - 0s 3ms/step - loss: 4.4680 - accuracy: 0.7084 - val_loss: 4.4743 - val_accuracy: 0.7046Epoch 00028: val_accuracy did not improve from 0.70659
Epoch 29/50
63/63 [==============================] - 0s 4ms/step - loss: 4.5811 - accuracy: 0.7003 - val_loss: 4.5315 - val_accuracy: 0.7046Epoch 00029: val_accuracy did not improve from 0.70659
Epoch 30/50
63/63 [==============================] - 0s 4ms/step - loss: 4.7530 - accuracy: 0.6898 - val_loss: 4.5332 - val_accuracy: 0.7026Epoch 00030: val_accuracy did not improve from 0.70659
Epoch 31/50
63/63 [==============================] - 0s 3ms/step - loss: 4.5055 - accuracy: 0.7062 - val_loss: 4.5337 - val_accuracy: 0.7026Epoch 00031: val_accuracy did not improve from 0.70659
Epoch 32/50
63/63 [==============================] - 0s 4ms/step - loss: 4.4446 - accuracy: 0.7104 - val_loss: 4.5347 - val_accuracy: 0.7026Epoch 00032: val_accuracy did not improve from 0.70659
Epoch 33/50
63/63 [==============================] - 0s 3ms/step - loss: 4.5923 - accuracy: 0.7006 - val_loss: 5.2948 - val_accuracy: 0.6527Epoch 00033: val_accuracy did not improve from 0.70659
Epoch 34/50
63/63 [==============================] - 0s 3ms/step - loss: 5.6213 - accuracy: 0.6318 - val_loss: 5.7545 - val_accuracy: 0.6188Epoch 00034: val_accuracy did not improve from 0.70659
Epoch 35/50
63/63 [==============================] - 0s 3ms/step - loss: 5.7049 - accuracy: 0.6244 - val_loss: 5.4230 - val_accuracy: 0.6427Epoch 00035: val_accuracy did not improve from 0.70659
Epoch 36/50
63/63 [==============================] - 0s 5ms/step - loss: 5.1246 - accuracy: 0.6643 - val_loss: 5.4163 - val_accuracy: 0.6427Epoch 00036: val_accuracy did not improve from 0.70659
Epoch 37/50
63/63 [==============================] - 0s 3ms/step - loss: 5.1822 - accuracy: 0.6575 - val_loss: 4.9631 - val_accuracy: 0.6727Epoch 00037: val_accuracy did not improve from 0.70659
Epoch 38/50
63/63 [==============================] - 0s 4ms/step - loss: 4.5541 - accuracy: 0.7030 - val_loss: 4.9609 - val_accuracy: 0.6727Epoch 00038: val_accuracy did not improve from 0.70659
Epoch 39/50
63/63 [==============================] - 0s 3ms/step - loss: 4.5174 - accuracy: 0.7054 - val_loss: 4.9610 - val_accuracy: 0.6727Epoch 00039: val_accuracy did not improve from 0.70659
Epoch 40/50
63/63 [==============================] - 0s 3ms/step - loss: 4.7210 - accuracy: 0.6921 - val_loss: 4.9607 - val_accuracy: 0.6747Epoch 00040: val_accuracy did not improve from 0.70659
Epoch 41/50
63/63 [==============================] - 0s 4ms/step - loss: 4.7356 - accuracy: 0.6909 - val_loss: 4.5617 - val_accuracy: 0.7026Epoch 00041: val_accuracy did not improve from 0.70659
Epoch 42/50
63/63 [==============================] - 0s 3ms/step - loss: 4.2498 - accuracy: 0.7209 - val_loss: 4.4727 - val_accuracy: 0.7066Epoch 00042: val_accuracy did not improve from 0.70659
Epoch 43/50
63/63 [==============================] - 0s 4ms/step - loss: 4.4859 - accuracy: 0.7052 - val_loss: 4.0455 - val_accuracy: 0.7365Epoch 00043: val_accuracy improved from 0.70659 to 0.73653, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\Navie\weights.hdf5
Epoch 44/50
63/63 [==============================] - 0s 3ms/step - loss: 4.0808 - accuracy: 0.7324 - val_loss: 3.9542 - val_accuracy: 0.7425Epoch 00044: val_accuracy improved from 0.73653 to 0.74251, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\Navie\weights.hdf5
Epoch 45/50
63/63 [==============================] - 0s 4ms/step - loss: 4.0209 - accuracy: 0.7360 - val_loss: 3.9542 - val_accuracy: 0.7425Epoch 00045: val_accuracy did not improve from 0.74251
Epoch 46/50
63/63 [==============================] - 0s 4ms/step - loss: 4.1993 - accuracy: 0.7262 - val_loss: 3.9272 - val_accuracy: 0.7425Epoch 00046: val_accuracy did not improve from 0.74251
Epoch 47/50
63/63 [==============================] - 0s 4ms/step - loss: 4.2362 - accuracy: 0.7233 - val_loss: 3.8685 - val_accuracy: 0.7465Epoch 00047: val_accuracy improved from 0.74251 to 0.74651, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\Navie\weights.hdf5
Epoch 48/50
63/63 [==============================] - 0s 3ms/step - loss: 3.9913 - accuracy: 0.7392 - val_loss: 3.8669 - val_accuracy: 0.7465Epoch 00048: val_accuracy did not improve from 0.74651
Epoch 49/50
63/63 [==============================] - 0s 3ms/step - loss: 4.1925 - accuracy: 0.7268 - val_loss: 3.8673 - val_accuracy: 0.7465Epoch 00049: val_accuracy did not improve from 0.74651
Epoch 50/50
63/63 [==============================] - 0s 3ms/step - loss: 4.3917 - accuracy: 0.7131 - val_loss: 3.8678 - val_accuracy: 0.7465Epoch 00050: val_accuracy did not improve from 0.74651
Navie运行时间: 0.27 mins
INFO:tensorflow:Assets written to: H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\Navie\save\Navie\assets
保存成功 保存地址在 H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\Navie\save

plot_metric(history,"loss")
plot_metric(history,"accuracy")

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-rz5OIGiQ-1623898674921)(output_46_0.svg)]

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-24kczFoC-1623898674923)(output_46_1.svg)]

# 以表格形式展示
dfhistory = pd.DataFrame(history.history)
dfhistory.index = range(1,len(dfhistory) + 1)
dfhistory.index.name = 'epoch'
dfhistory.to_csv(os.path.join(path,'save',model.name,'.csv'))dfhistory

	loss	accuracy	val_loss	val_accuracy	lr
epoch
1	8.703827	0.4280	8.265403	0.457086	0.001
2	8.127553	0.4645	7.871212	0.483034	0.001
3	7.788067	0.4895	7.694490	0.495010	0.001
4	7.614698	0.5000	7.443610	0.512974	0.001
5	7.249546	0.5235	7.144458	0.530938	0.001
6	7.061227	0.5370	6.753396	0.554890	0.001
7	6.457839	0.5735	6.521289	0.566866	0.001
8	6.580623	0.5690	6.899066	0.548902	0.001
9	6.656266	0.5630	6.868452	0.550898	0.001
10	6.582748	0.5685	6.420915	0.578842	0.001
11	5.829680	0.6170	5.539738	0.632735	0.001
12	5.300040	0.6505	5.237395	0.656687	0.001
13	5.115226	0.6650	5.325986	0.650699	0.001
14	5.164371	0.6630	5.383919	0.646707	0.001
15	5.033154	0.6685	5.081899	0.666667	0.001
16	4.999567	0.6720	5.081350	0.664671	0.001
17	4.864917	0.6805	4.897707	0.678643	0.001
18	4.762197	0.6890	4.895268	0.680639	0.001
19	4.761465	0.6895	4.895268	0.680639	0.001
20	4.761386	0.6895	4.895268	0.680639	0.001
21	4.763627	0.6885	4.896749	0.680639	0.001
22	4.762402	0.6890	4.834998	0.684631	0.001
23	4.664893	0.6945	4.563492	0.700599	0.001
24	4.574459	0.7005	4.500281	0.706587	0.001
25	4.543370	0.7025	4.501622	0.706587	0.001
26	4.542712	0.7030	4.530774	0.704591	0.001
27	4.542055	0.7035	4.473816	0.706587	0.001
28	4.542201	0.7030	4.474332	0.704591	0.001
29	4.541991	0.7035	4.531502	0.704591	0.001
30	4.541663	0.7035	4.533184	0.702595	0.001
31	4.541536	0.7035	4.533740	0.702595	0.001
32	4.541585	0.7040	4.534698	0.702595	0.001
33	4.577245	0.7005	5.294760	0.652695	0.001
34	5.759947	0.6225	5.754533	0.618762	0.001
35	5.581280	0.6330	5.423037	0.642715	0.001
36	5.440868	0.6430	5.416341	0.642715	0.001
37	4.959578	0.6740	4.963100	0.672655	0.001
38	4.746535	0.6905	4.960928	0.672655	0.001
39	4.746130	0.6905	4.960978	0.672655	0.001
40	4.746026	0.6905	4.960693	0.674651	0.001
41	4.634393	0.6970	4.561750	0.702595	0.001
42	4.531757	0.7015	4.472662	0.706587	0.001
43	4.434167	0.7095	4.045545	0.736527	0.001
44	4.260184	0.7205	3.954157	0.742515	0.001
45	4.174057	0.7260	3.954157	0.742515	0.001
46	4.110280	0.7315	3.927210	0.742515	0.001
47	4.095524	0.7325	3.868510	0.746507	0.001
48	4.086236	0.7335	3.866899	0.746507	0.001
49	4.086092	0.7335	3.867256	0.746507	0.001
50	4.086073	0.7335	3.867835	0.746507	0.001

cdssm

def textCNN(x):x=Conv1D(filters=32,kernel_size=3,strides=1,padding='same',activation='relu',kernel_initializer='glorot_normal',bias_initializer='zeros')(x)# Apply max pooling by take max at each dimension across# all word_trigram features.x =Dropout(0.3)(x)x = GlobalMaxPool1D()(x)for _ in range(3):x = Dense(128,activation='relu')(x)return x

filters = 200
kernel_sizes = [3,4,5]
output_dim = 100input1 = keras.layers.Input(name='sent1', shape=(max_len,))
input2 = keras.layers.Input(name='sent2', shape=(max_len,))embedding = keras.layers.Embedding(vocab_size, embedding_size)
sent1_embed = embedding(input1)
sent2_embed = embedding(input2)# [output_sent1, output_sent2]=[textCNN(sent1_embed),textCNN(sent2_embed)]xa = [textCNN(sent1_embed),textCNN(sent2_embed)]# xa = Dot(axes=[1, 1], normalize=True)(xa)cosine_output = keras.layers.Dot(axes=[1, 1], normalize=True)(xa)
outputs = keras.layers.Dense(1, activation='linear', name="output")(cosine_output)model = keras.models.Model(inputs=[input1, input2], outputs=outputs,name="CDSSM")
model.summary()

Model: "CDSSM"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to
==================================================================================================
sent1 (InputLayer)              [(None, 15)]         0
__________________________________________________________________________________________________
sent2 (InputLayer)              [(None, 15)]         0
__________________________________________________________________________________________________
embedding (Embedding)           (None, 15, 128)      278400      sent1[0][0]                      sent2[0][0]
__________________________________________________________________________________________________
conv1d (Conv1D)                 (None, 15, 32)       12320       embedding[0][0]
__________________________________________________________________________________________________
conv1d_1 (Conv1D)               (None, 15, 32)       12320       embedding[1][0]
__________________________________________________________________________________________________
dropout (Dropout)               (None, 15, 32)       0           conv1d[0][0]
__________________________________________________________________________________________________
dropout_1 (Dropout)             (None, 15, 32)       0           conv1d_1[0][0]
__________________________________________________________________________________________________
global_max_pooling1d (GlobalMax (None, 32)           0           dropout[0][0]
__________________________________________________________________________________________________
global_max_pooling1d_1 (GlobalM (None, 32)           0           dropout_1[0][0]
__________________________________________________________________________________________________
dense_4 (Dense)                 (None, 128)          4224        global_max_pooling1d[0][0]
__________________________________________________________________________________________________
dense_7 (Dense)                 (None, 128)          4224        global_max_pooling1d_1[0][0]
__________________________________________________________________________________________________
dense_5 (Dense)                 (None, 128)          16512       dense_4[0][0]
__________________________________________________________________________________________________
dense_8 (Dense)                 (None, 128)          16512       dense_7[0][0]
__________________________________________________________________________________________________
dense_6 (Dense)                 (None, 128)          16512       dense_5[0][0]
__________________________________________________________________________________________________
dense_9 (Dense)                 (None, 128)          16512       dense_8[0][0]
__________________________________________________________________________________________________
dot (Dot)                       (None, 1)            0           dense_6[0][0]                    dense_9[0][0]
__________________________________________________________________________________________________
output (Dense)                  (None, 1)            2           dot[0][0]
==================================================================================================
Total params: 377,538
Trainable params: 377,538
Non-trainable params: 0
__________________________________________________________________________________________________

model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])# 各种callback函数
tb_callback = tf.keras.callbacks.TensorBoard(logdir, histogram_freq=1)
#如果accuracy在25个epoch后没有提升，学习率减半。
lr_callback = tf.keras.callbacks.ReduceLROnPlateau(monitor="accuracy",factor = 0.5, patience = 25)
#当accuracy在30个epoch后没有提升，则提前终止训练。
stop_callback = tf.keras.callbacks.EarlyStopping(monitor = "accuracy", patience= 30)#获取当前的地址
path=os.path.join(os.getcwd(),'outputs\model',model.name)import os
dirs = pathif not os.path.exists(dirs):os.makedirs(dirs)# 保留最好的模型
mc_callback=tf.keras.callbacks.ModelCheckpoint(filepath=os.path.join(path,'weights.hdf5'),monitor='val_accuracy',verbose=1, save_best_only=True)# 把训练轮结果数据流到 csv 文件的回调函数。
# url=os.path.join(os.getcwd(),'outputs')csv_callback=tf.keras.callbacks.CSVLogger(os.path.join(path,'training.log'))callbacks_list = [tb_callback,lr_callback,stop_callback,mc_callback,csv_callback]t = time()
history=model.fit(x=[sent1_train,sent2_train],y=train_labels,validation_data=([sent1_val,sent2_val],val_labels),epochs=50,\callbacks = callbacks_list)
print(model.name+'运行时间: {} mins'.format(round((time() - t) / 60, 2)))model.save(os.path.join(path,'save',model.name))
print('保存成功','保存地址在',os.path.join(path,'save'))

Epoch 1/50
63/63 [==============================] - 5s 20ms/step - loss: 0.5117 - accuracy: 0.8011 - val_loss: 0.4879 - val_accuracy: 0.8144Epoch 00001: val_accuracy improved from -inf to 0.81437, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\CDSSM\weights.hdf5
Epoch 2/50
63/63 [==============================] - 1s 11ms/step - loss: 0.5048 - accuracy: 0.7961 - val_loss: 0.4895 - val_accuracy: 0.8144Epoch 00002: val_accuracy did not improve from 0.81437
Epoch 3/50
63/63 [==============================] - 1s 12ms/step - loss: 0.4382 - accuracy: 0.8074 - val_loss: 0.4815 - val_accuracy: 0.7984Epoch 00003: val_accuracy did not improve from 0.81437
Epoch 4/50
63/63 [==============================] - 1s 11ms/step - loss: 0.3467 - accuracy: 0.8669 - val_loss: 0.5471 - val_accuracy: 0.7465Epoch 00004: val_accuracy did not improve from 0.81437
Epoch 5/50
63/63 [==============================] - 1s 11ms/step - loss: 0.3072 - accuracy: 0.8672 - val_loss: 0.5465 - val_accuracy: 0.7725Epoch 00005: val_accuracy did not improve from 0.81437
Epoch 6/50
63/63 [==============================] - 1s 11ms/step - loss: 0.2380 - accuracy: 0.9023 - val_loss: 0.6638 - val_accuracy: 0.6766Epoch 00006: val_accuracy did not improve from 0.81437
Epoch 7/50
63/63 [==============================] - 1s 11ms/step - loss: 0.2054 - accuracy: 0.9284 - val_loss: 0.6456 - val_accuracy: 0.7565Epoch 00007: val_accuracy did not improve from 0.81437
Epoch 8/50
63/63 [==============================] - 1s 11ms/step - loss: 0.1676 - accuracy: 0.9392 - val_loss: 0.7225 - val_accuracy: 0.7246Epoch 00008: val_accuracy did not improve from 0.81437
Epoch 9/50
63/63 [==============================] - 1s 11ms/step - loss: 0.1127 - accuracy: 0.9616 - val_loss: 1.2580 - val_accuracy: 0.7545Epoch 00009: val_accuracy did not improve from 0.81437
Epoch 10/50
63/63 [==============================] - 1s 11ms/step - loss: 0.1236 - accuracy: 0.9582 - val_loss: 1.9337 - val_accuracy: 0.7924Epoch 00010: val_accuracy did not improve from 0.81437
Epoch 11/50
63/63 [==============================] - 1s 11ms/step - loss: 0.1590 - accuracy: 0.9407 - val_loss: 1.5420 - val_accuracy: 0.7505Epoch 00011: val_accuracy did not improve from 0.81437
Epoch 12/50
63/63 [==============================] - 1s 11ms/step - loss: 0.0837 - accuracy: 0.9729 - val_loss: 2.0833 - val_accuracy: 0.7665Epoch 00012: val_accuracy did not improve from 0.81437
Epoch 13/50
63/63 [==============================] - 1s 11ms/step - loss: 0.1183 - accuracy: 0.9739 - val_loss: 2.1034 - val_accuracy: 0.7246Epoch 00013: val_accuracy did not improve from 0.81437
Epoch 14/50
63/63 [==============================] - 1s 11ms/step - loss: 0.1002 - accuracy: 0.9581 - val_loss: 2.0305 - val_accuracy: 0.7086Epoch 00014: val_accuracy did not improve from 0.81437
Epoch 15/50
63/63 [==============================] - 1s 11ms/step - loss: 0.0706 - accuracy: 0.9759 - val_loss: 1.8748 - val_accuracy: 0.7485Epoch 00015: val_accuracy did not improve from 0.81437
Epoch 16/50
63/63 [==============================] - 1s 12ms/step - loss: 0.0770 - accuracy: 0.9747 - val_loss: 1.6595 - val_accuracy: 0.7924Epoch 00016: val_accuracy did not improve from 0.81437
Epoch 17/50
63/63 [==============================] - 1s 11ms/step - loss: 0.1662 - accuracy: 0.9520 - val_loss: 1.6988 - val_accuracy: 0.7605Epoch 00017: val_accuracy did not improve from 0.81437
Epoch 18/50
63/63 [==============================] - 1s 11ms/step - loss: 0.1409 - accuracy: 0.9498 - val_loss: 0.9682 - val_accuracy: 0.7645Epoch 00018: val_accuracy did not improve from 0.81437
Epoch 19/50
63/63 [==============================] - 1s 11ms/step - loss: 0.0659 - accuracy: 0.9723 - val_loss: 1.7093 - val_accuracy: 0.7285Epoch 00019: val_accuracy did not improve from 0.81437
Epoch 20/50
63/63 [==============================] - 1s 11ms/step - loss: 0.1253 - accuracy: 0.9629 - val_loss: 0.4928 - val_accuracy: 0.8144Epoch 00020: val_accuracy did not improve from 0.81437
Epoch 21/50
63/63 [==============================] - 1s 12ms/step - loss: 0.3694 - accuracy: 0.8108 - val_loss: 1.0434 - val_accuracy: 0.7745Epoch 00021: val_accuracy did not improve from 0.81437
Epoch 22/50
63/63 [==============================] - 1s 11ms/step - loss: 0.2163 - accuracy: 0.9157 - val_loss: 1.3512 - val_accuracy: 0.7605Epoch 00022: val_accuracy did not improve from 0.81437
Epoch 23/50
63/63 [==============================] - 1s 11ms/step - loss: 0.1188 - accuracy: 0.9498 - val_loss: 1.5283 - val_accuracy: 0.7764Epoch 00023: val_accuracy did not improve from 0.81437
Epoch 24/50
63/63 [==============================] - 1s 11ms/step - loss: 0.1007 - accuracy: 0.9636 - val_loss: 1.9045 - val_accuracy: 0.7545Epoch 00024: val_accuracy did not improve from 0.81437
Epoch 25/50
63/63 [==============================] - 1s 11ms/step - loss: 0.0744 - accuracy: 0.9786 - val_loss: 1.3289 - val_accuracy: 0.7585Epoch 00025: val_accuracy did not improve from 0.81437
Epoch 26/50
63/63 [==============================] - 1s 11ms/step - loss: 0.0813 - accuracy: 0.9758 - val_loss: 1.8299 - val_accuracy: 0.7525Epoch 00026: val_accuracy did not improve from 0.81437
Epoch 27/50
63/63 [==============================] - 1s 12ms/step - loss: 0.0597 - accuracy: 0.9770 - val_loss: 2.0222 - val_accuracy: 0.7405Epoch 00027: val_accuracy did not improve from 0.81437
Epoch 28/50
63/63 [==============================] - 1s 11ms/step - loss: 0.0912 - accuracy: 0.9732 - val_loss: 1.7747 - val_accuracy: 0.7605Epoch 00028: val_accuracy did not improve from 0.81437
Epoch 29/50
63/63 [==============================] - 1s 11ms/step - loss: 0.0732 - accuracy: 0.9780 - val_loss: 2.1430 - val_accuracy: 0.7585Epoch 00029: val_accuracy did not improve from 0.81437
Epoch 30/50
63/63 [==============================] - 1s 11ms/step - loss: 0.0303 - accuracy: 0.9893 - val_loss: 2.0570 - val_accuracy: 0.7365Epoch 00030: val_accuracy did not improve from 0.81437
Epoch 31/50
63/63 [==============================] - 1s 11ms/step - loss: 0.0335 - accuracy: 0.9858 - val_loss: 2.2736 - val_accuracy: 0.7525Epoch 00031: val_accuracy did not improve from 0.81437
Epoch 32/50
63/63 [==============================] - 1s 11ms/step - loss: 0.0299 - accuracy: 0.9884 - val_loss: 2.1349 - val_accuracy: 0.7525Epoch 00032: val_accuracy did not improve from 0.81437
Epoch 33/50
63/63 [==============================] - 1s 11ms/step - loss: 0.0418 - accuracy: 0.9923 - val_loss: 1.9088 - val_accuracy: 0.7585Epoch 00033: val_accuracy did not improve from 0.81437
Epoch 34/50
63/63 [==============================] - 1s 11ms/step - loss: 0.0319 - accuracy: 0.9870 - val_loss: 2.1658 - val_accuracy: 0.7405Epoch 00034: val_accuracy did not improve from 0.81437
Epoch 35/50
63/63 [==============================] - 1s 11ms/step - loss: 0.0511 - accuracy: 0.9873 - val_loss: 2.1202 - val_accuracy: 0.7505Epoch 00035: val_accuracy did not improve from 0.81437
Epoch 36/50
63/63 [==============================] - 1s 11ms/step - loss: 0.0156 - accuracy: 0.9948 - val_loss: 2.2616 - val_accuracy: 0.7605Epoch 00036: val_accuracy did not improve from 0.81437
Epoch 37/50
63/63 [==============================] - 1s 11ms/step - loss: 0.0366 - accuracy: 0.9896 - val_loss: 2.1474 - val_accuracy: 0.7625Epoch 00037: val_accuracy did not improve from 0.81437
Epoch 38/50
63/63 [==============================] - 1s 11ms/step - loss: 0.0294 - accuracy: 0.9939 - val_loss: 2.3427 - val_accuracy: 0.7545Epoch 00038: val_accuracy did not improve from 0.81437
Epoch 39/50
63/63 [==============================] - 1s 11ms/step - loss: 0.0230 - accuracy: 0.9937 - val_loss: 2.4009 - val_accuracy: 0.7585Epoch 00039: val_accuracy did not improve from 0.81437
Epoch 40/50
63/63 [==============================] - 1s 10ms/step - loss: 0.0484 - accuracy: 0.9920 - val_loss: 2.4710 - val_accuracy: 0.7685Epoch 00040: val_accuracy did not improve from 0.81437
Epoch 41/50
63/63 [==============================] - 1s 11ms/step - loss: 0.0363 - accuracy: 0.9868 - val_loss: 2.2250 - val_accuracy: 0.7465Epoch 00041: val_accuracy did not improve from 0.81437
Epoch 42/50
63/63 [==============================] - 1s 11ms/step - loss: 0.0194 - accuracy: 0.9907 - val_loss: 2.4387 - val_accuracy: 0.7625Epoch 00042: val_accuracy did not improve from 0.81437
Epoch 43/50
63/63 [==============================] - 1s 11ms/step - loss: 0.0297 - accuracy: 0.9878 - val_loss: 2.3372 - val_accuracy: 0.7505Epoch 00043: val_accuracy did not improve from 0.81437
Epoch 44/50
63/63 [==============================] - 1s 11ms/step - loss: 0.0251 - accuracy: 0.9923 - val_loss: 2.4880 - val_accuracy: 0.7425Epoch 00044: val_accuracy did not improve from 0.81437
Epoch 45/50
63/63 [==============================] - 1s 11ms/step - loss: 0.0354 - accuracy: 0.9895 - val_loss: 2.1338 - val_accuracy: 0.7645Epoch 00045: val_accuracy did not improve from 0.81437
Epoch 46/50
63/63 [==============================] - 1s 11ms/step - loss: 0.0263 - accuracy: 0.9880 - val_loss: 2.3131 - val_accuracy: 0.7246Epoch 00046: val_accuracy did not improve from 0.81437
Epoch 47/50
63/63 [==============================] - 1s 11ms/step - loss: 0.0194 - accuracy: 0.9941 - val_loss: 2.3929 - val_accuracy: 0.7365Epoch 00047: val_accuracy did not improve from 0.81437
Epoch 48/50
63/63 [==============================] - 1s 11ms/step - loss: 0.0197 - accuracy: 0.9922 - val_loss: 2.3312 - val_accuracy: 0.7305Epoch 00048: val_accuracy did not improve from 0.81437
Epoch 49/50
63/63 [==============================] - 1s 12ms/step - loss: 0.0225 - accuracy: 0.9883 - val_loss: 2.3285 - val_accuracy: 0.7525Epoch 00049: val_accuracy did not improve from 0.81437
Epoch 50/50
63/63 [==============================] - 1s 11ms/step - loss: 0.0246 - accuracy: 0.9921 - val_loss: 2.2293 - val_accuracy: 0.7325Epoch 00050: val_accuracy did not improve from 0.81437
CDSSM运行时间: 0.76 mins
INFO:tensorflow:Assets written to: H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\CDSSM\save\CDSSM\assets
保存成功 保存地址在 H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\CDSSM\save

plot_metric(history,"loss")
plot_metric(history,"accuracy")

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-jAlOz8Da-1623898674923)(output_52_0.svg)]

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-ZXVcTZna-1623898674924)(output_52_1.svg)]

os.path.join(path,'save',model.name,model.name+'.csv')

'H:\\code\\BNLP学习\\NLP项目教程\\匹配\\文本匹配\\花呗问答匹配\\outputs\\model\\CDSSM\\save\\CDSSM\\CDSSM.csv'

# 以表格形式展示
dfhistory = pd.DataFrame(history.history)
dfhistory.index = range(1,len(dfhistory) + 1)
dfhistory.index.name = 'epoch'
dfhistory.to_csv(os.path.join(path,'save',model.name,model.name+'.csv'))dfhistory

	loss	accuracy	val_loss	val_accuracy	lr
epoch
1	0.502438	0.8060	0.487863	0.814371	0.001
2	0.482865	0.8060	0.489541	0.814371	0.001
3	0.443680	0.8125	0.481475	0.798403	0.001
4	0.362532	0.8495	0.547100	0.746507	0.001
5	0.315729	0.8590	0.546462	0.772455	0.001
6	0.249621	0.8995	0.663817	0.676647	0.001
7	0.210387	0.9230	0.645588	0.756487	0.001
8	0.177454	0.9345	0.722511	0.724551	0.001
9	0.140597	0.9510	1.257960	0.754491	0.001
10	0.109115	0.9615	1.933702	0.792415	0.001
11	0.146496	0.9490	1.541986	0.750499	0.001
12	0.104541	0.9680	2.083340	0.766467	0.001
13	0.117658	0.9735	2.103375	0.724551	0.001
14	0.106086	0.9615	2.030532	0.708583	0.001
15	0.072325	0.9735	1.874843	0.748503	0.001
16	0.130131	0.9575	1.659451	0.792415	0.001
17	0.147002	0.9540	1.698843	0.760479	0.001
18	0.140070	0.9525	0.968186	0.764471	0.001
19	0.070261	0.9710	1.709319	0.728543	0.001
20	0.259516	0.9030	0.492770	0.814371	0.001
21	0.316189	0.8490	1.043425	0.774451	0.001
22	0.204753	0.9255	1.351184	0.760479	0.001
23	0.127438	0.9505	1.528280	0.776447	0.001
24	0.088141	0.9675	1.904538	0.754491	0.001
25	0.093163	0.9700	1.328880	0.758483	0.001
26	0.070672	0.9770	1.829938	0.752495	0.001
27	0.058763	0.9765	2.022192	0.740519	0.001
28	0.104254	0.9665	1.774662	0.760479	0.001
29	0.056904	0.9785	2.143026	0.758483	0.001
30	0.040742	0.9860	2.056966	0.736527	0.001
31	0.032480	0.9875	2.273608	0.752495	0.001
32	0.033481	0.9880	2.134907	0.752495	0.001
33	0.042117	0.9890	1.908836	0.758483	0.001
34	0.027738	0.9880	2.165784	0.740519	0.001
35	0.045009	0.9875	2.120193	0.750499	0.001
36	0.020214	0.9935	2.261635	0.760479	0.001
37	0.040293	0.9890	2.147407	0.762475	0.001
38	0.027351	0.9925	2.342712	0.754491	0.001
39	0.027095	0.9910	2.400923	0.758483	0.001
40	0.037127	0.9920	2.471040	0.768463	0.001
41	0.038542	0.9855	2.225006	0.746507	0.001
42	0.022141	0.9900	2.438666	0.762475	0.001
43	0.034390	0.9870	2.337209	0.750499	0.001
44	0.017784	0.9955	2.488035	0.742515	0.001
45	0.032875	0.9920	2.133770	0.764471	0.001
46	0.022209	0.9915	2.313134	0.724551	0.001
47	0.024295	0.9935	2.392854	0.736527	0.001
48	0.016932	0.9940	2.331223	0.730539	0.001
49	0.025668	0.9915	2.328529	0.752495	0.001
50	0.023949	0.9920	2.229317	0.732535	0.001

BasictextCNN

def BaseTextCNN(inputs):conv_layers = []max_poolings = []for kernel_size in kernel_sizes:conv_layers.append(keras.layers.Conv1D(filters=filters, kernel_size=kernel_size, activation='relu', padding="same"))max_poolings.append(keras.layers.GlobalMaxPool1D())concatenate = keras.layers.Concatenate()dense = keras.layers.Dense(output_dim, activation='tanh')convs = []for i in range(len(kernel_sizes)):x = conv_layers[i](inputs)x = max_poolings[i](x)convs.append(x)x = concatenate(convs)output = dense(x)return output

filters = 200
kernel_sizes = [3,4,5]
output_dim = 100input1 = keras.layers.Input(name='sent1', shape=(max_len,))
input2 = keras.layers.Input(name='sent2', shape=(max_len,))embedding = keras.layers.Embedding(vocab_size, embedding_size)
sent1_embed = embedding(input1)
sent2_embed = embedding(input2)output_sent1 = BaseTextCNN(sent1_embed)
output_sent2 = BaseTextCNN(sent2_embed)cosine_output = keras.layers.Dot(axes=[1, 1], normalize=True)([output_sent1, output_sent2])
outputs = keras.layers.Dense(1, activation='linear', name="output")(cosine_output)model = keras.models.Model(inputs=[input1, input2], outputs=outputs,name="BaseTextCNN_match")
model.summary()

Model: "BaseTextCNN_match"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to
==================================================================================================
sent1 (InputLayer)              [(None, 15)]         0
__________________________________________________________________________________________________
sent2 (InputLayer)              [(None, 15)]         0
__________________________________________________________________________________________________
embedding_1 (Embedding)         (None, 15, 128)      278400      sent1[0][0]                      sent2[0][0]
__________________________________________________________________________________________________
conv1d_2 (Conv1D)               (None, 15, 200)      77000       embedding_1[0][0]
__________________________________________________________________________________________________
conv1d_3 (Conv1D)               (None, 15, 200)      102600      embedding_1[0][0]
__________________________________________________________________________________________________
conv1d_4 (Conv1D)               (None, 15, 200)      128200      embedding_1[0][0]
__________________________________________________________________________________________________
conv1d_5 (Conv1D)               (None, 15, 200)      77000       embedding_1[1][0]
__________________________________________________________________________________________________
conv1d_6 (Conv1D)               (None, 15, 200)      102600      embedding_1[1][0]
__________________________________________________________________________________________________
conv1d_7 (Conv1D)               (None, 15, 200)      128200      embedding_1[1][0]
__________________________________________________________________________________________________
global_max_pooling1d_2 (GlobalM (None, 200)          0           conv1d_2[0][0]
__________________________________________________________________________________________________
global_max_pooling1d_3 (GlobalM (None, 200)          0           conv1d_3[0][0]
__________________________________________________________________________________________________
global_max_pooling1d_4 (GlobalM (None, 200)          0           conv1d_4[0][0]
__________________________________________________________________________________________________
global_max_pooling1d_5 (GlobalM (None, 200)          0           conv1d_5[0][0]
__________________________________________________________________________________________________
global_max_pooling1d_6 (GlobalM (None, 200)          0           conv1d_6[0][0]
__________________________________________________________________________________________________
global_max_pooling1d_7 (GlobalM (None, 200)          0           conv1d_7[0][0]
__________________________________________________________________________________________________
concatenate_2 (Concatenate)     (None, 600)          0           global_max_pooling1d_2[0][0]     global_max_pooling1d_3[0][0]     global_max_pooling1d_4[0][0]
__________________________________________________________________________________________________
concatenate_3 (Concatenate)     (None, 600)          0           global_max_pooling1d_5[0][0]     global_max_pooling1d_6[0][0]     global_max_pooling1d_7[0][0]
__________________________________________________________________________________________________
dense_10 (Dense)                (None, 100)          60100       concatenate_2[0][0]
__________________________________________________________________________________________________
dense_11 (Dense)                (None, 100)          60100       concatenate_3[0][0]
__________________________________________________________________________________________________
dot_1 (Dot)                     (None, 1)            0           dense_10[0][0]                   dense_11[0][0]
__________________________________________________________________________________________________
output (Dense)                  (None, 1)            2           dot_1[0][0]
==================================================================================================
Total params: 1,014,202
Trainable params: 1,014,202
Non-trainable params: 0
__________________________________________________________________________________________________

model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])# 各种callback函数
tb_callback = tf.keras.callbacks.TensorBoard(logdir, histogram_freq=1)
#如果accuracy在25个epoch后没有提升，学习率减半。
lr_callback = tf.keras.callbacks.ReduceLROnPlateau(monitor="accuracy",factor = 0.5, patience = 25)
#当accuracy在30个epoch后没有提升，则提前终止训练。
stop_callback = tf.keras.callbacks.EarlyStopping(monitor = "accuracy", patience= 30)#获取当前的地址
path=os.path.join(os.getcwd(),'outputs\model',model.name)import os
dirs = pathif not os.path.exists(dirs):os.makedirs(dirs)# 保留最好的模型
mc_callback=tf.keras.callbacks.ModelCheckpoint(filepath=os.path.join(path,'weights.hdf5'),monitor='val_accuracy',verbose=1, save_best_only=True)# 把训练轮结果数据流到 csv 文件的回调函数。
# url=os.path.join(os.getcwd(),'outputs')csv_callback=tf.keras.callbacks.CSVLogger(os.path.join(path,'training.log'))callbacks_list = [tb_callback,lr_callback,stop_callback,mc_callback,csv_callback]t = time()
history=model.fit(x=[sent1_train,sent2_train],y=train_labels,validation_data=([sent1_val,sent2_val],val_labels),epochs=50,\callbacks = callbacks_list)
print(model.name+'运行时间: {} mins'.format(round((time() - t) / 60, 2)))model.save(os.path.join(path,'save',model.name))
print('保存成功','保存地址在',os.path.join(path,'save'))

Epoch 1/50
63/63 [==============================] - 2s 24ms/step - loss: 0.5421 - accuracy: 0.7956 - val_loss: 0.4770 - val_accuracy: 0.8144Epoch 00001: val_accuracy improved from -inf to 0.81437, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\BaseTextCNN_match\weights.hdf5
Epoch 2/50
63/63 [==============================] - 1s 13ms/step - loss: 0.4627 - accuracy: 0.8030 - val_loss: 0.4782 - val_accuracy: 0.8144Epoch 00002: val_accuracy did not improve from 0.81437
Epoch 3/50
63/63 [==============================] - 1s 12ms/step - loss: 0.4340 - accuracy: 0.7894 - val_loss: 0.8211 - val_accuracy: 0.8144Epoch 00003: val_accuracy did not improve from 0.81437
Epoch 4/50
63/63 [==============================] - 1s 13ms/step - loss: 0.3410 - accuracy: 0.8213 - val_loss: 0.7917 - val_accuracy: 0.7964Epoch 00004: val_accuracy did not improve from 0.81437
Epoch 5/50
63/63 [==============================] - 1s 13ms/step - loss: 0.2444 - accuracy: 0.8669 - val_loss: 0.8264 - val_accuracy: 0.7385Epoch 00005: val_accuracy did not improve from 0.81437
Epoch 6/50
63/63 [==============================] - 1s 13ms/step - loss: 0.1985 - accuracy: 0.9501 - val_loss: 1.7119 - val_accuracy: 0.7944Epoch 00006: val_accuracy did not improve from 0.81437
Epoch 7/50
63/63 [==============================] - 1s 15ms/step - loss: 0.1374 - accuracy: 0.9921 - val_loss: 1.6702 - val_accuracy: 0.7844Epoch 00007: val_accuracy did not improve from 0.81437
Epoch 8/50
63/63 [==============================] - 1s 13ms/step - loss: 0.1125 - accuracy: 0.9960 - val_loss: 1.7951 - val_accuracy: 0.7784Epoch 00008: val_accuracy did not improve from 0.81437
Epoch 9/50
63/63 [==============================] - 1s 13ms/step - loss: 0.2621 - accuracy: 0.8833 - val_loss: 1.2079 - val_accuracy: 0.7864Epoch 00009: val_accuracy did not improve from 0.81437
Epoch 10/50
63/63 [==============================] - 1s 13ms/step - loss: 0.1658 - accuracy: 0.9579 - val_loss: 1.4842 - val_accuracy: 0.7305Epoch 00010: val_accuracy did not improve from 0.81437
Epoch 11/50
63/63 [==============================] - 1s 13ms/step - loss: 0.0570 - accuracy: 0.9964 - val_loss: 1.7038 - val_accuracy: 0.7685Epoch 00011: val_accuracy did not improve from 0.81437
Epoch 12/50
63/63 [==============================] - 1s 14ms/step - loss: 0.0102 - accuracy: 0.9995 - val_loss: 1.8691 - val_accuracy: 0.7525Epoch 00012: val_accuracy did not improve from 0.81437
Epoch 13/50
63/63 [==============================] - 1s 13ms/step - loss: 0.0146 - accuracy: 0.9991 - val_loss: 1.9849 - val_accuracy: 0.7545Epoch 00013: val_accuracy did not improve from 0.81437
Epoch 14/50
63/63 [==============================] - 1s 12ms/step - loss: 0.0209 - accuracy: 0.9987 - val_loss: 1.9464 - val_accuracy: 0.7545Epoch 00014: val_accuracy did not improve from 0.81437
Epoch 15/50
63/63 [==============================] - 1s 13ms/step - loss: 0.0297 - accuracy: 0.9981 - val_loss: 1.9654 - val_accuracy: 0.7565Epoch 00015: val_accuracy did not improve from 0.81437
Epoch 16/50
63/63 [==============================] - 1s 12ms/step - loss: 0.0332 - accuracy: 0.9979 - val_loss: 1.9614 - val_accuracy: 0.7605Epoch 00016: val_accuracy did not improve from 0.81437
Epoch 17/50
63/63 [==============================] - 1s 13ms/step - loss: 0.0386 - accuracy: 0.9975 - val_loss: 1.9540 - val_accuracy: 0.7625Epoch 00017: val_accuracy did not improve from 0.81437
Epoch 18/50
63/63 [==============================] - 1s 13ms/step - loss: 0.0446 - accuracy: 0.9971 - val_loss: 1.9512 - val_accuracy: 0.7625Epoch 00018: val_accuracy did not improve from 0.81437
Epoch 19/50
63/63 [==============================] - ETA: 0s - loss: 0.0101 - accuracy: 0.99 - 1s 13ms/step - loss: 0.0108 - accuracy: 0.9993 - val_loss: 1.9319 - val_accuracy: 0.7605Epoch 00019: val_accuracy did not improve from 0.81437
Epoch 20/50
63/63 [==============================] - 1s 13ms/step - loss: 0.0216 - accuracy: 0.9986 - val_loss: 1.9300 - val_accuracy: 0.7625Epoch 00020: val_accuracy did not improve from 0.81437
Epoch 21/50
63/63 [==============================] - 1s 13ms/step - loss: 0.0397 - accuracy: 0.9974 - val_loss: 1.8891 - val_accuracy: 0.7585Epoch 00021: val_accuracy did not improve from 0.81437
Epoch 22/50
63/63 [==============================] - 1s 13ms/step - loss: 0.0320 - accuracy: 0.9979 - val_loss: 1.9140 - val_accuracy: 0.7645Epoch 00022: val_accuracy did not improve from 0.81437
Epoch 23/50
63/63 [==============================] - 1s 13ms/step - loss: 0.0119 - accuracy: 0.9992 - val_loss: 1.9261 - val_accuracy: 0.7545Epoch 00023: val_accuracy did not improve from 0.81437
Epoch 24/50
63/63 [==============================] - 1s 13ms/step - loss: 0.0292 - accuracy: 0.9981 - val_loss: 1.9557 - val_accuracy: 0.7445Epoch 00024: val_accuracy did not improve from 0.81437
Epoch 25/50
63/63 [==============================] - 1s 13ms/step - loss: 0.0224 - accuracy: 0.9986 - val_loss: 2.0265 - val_accuracy: 0.7465Epoch 00025: val_accuracy did not improve from 0.81437
Epoch 26/50
63/63 [==============================] - 1s 13ms/step - loss: 0.0156 - accuracy: 0.9990 - val_loss: 2.0036 - val_accuracy: 0.7505Epoch 00026: val_accuracy did not improve from 0.81437
Epoch 27/50
63/63 [==============================] - 1s 13ms/step - loss: 0.0550 - accuracy: 0.9964 - val_loss: 1.9990 - val_accuracy: 0.7445Epoch 00027: val_accuracy did not improve from 0.81437
Epoch 28/50
63/63 [==============================] - 1s 14ms/step - loss: 0.0418 - accuracy: 0.9973 - val_loss: 2.1372 - val_accuracy: 0.7585Epoch 00028: val_accuracy did not improve from 0.81437
Epoch 29/50
63/63 [==============================] - 1s 13ms/step - loss: 0.0207 - accuracy: 0.9987 - val_loss: 2.0270 - val_accuracy: 0.7525Epoch 00029: val_accuracy did not improve from 0.81437
Epoch 30/50
63/63 [==============================] - 1s 13ms/step - loss: 0.0339 - accuracy: 0.9978 - val_loss: 2.0258 - val_accuracy: 0.7545Epoch 00030: val_accuracy did not improve from 0.81437
Epoch 31/50
63/63 [==============================] - 1s 13ms/step - loss: 0.0199 - accuracy: 0.9987 - val_loss: 1.9759 - val_accuracy: 0.7485Epoch 00031: val_accuracy did not improve from 0.81437
Epoch 32/50
63/63 [==============================] - 1s 14ms/step - loss: 0.0322 - accuracy: 0.9979 - val_loss: 1.9759 - val_accuracy: 0.7485Epoch 00032: val_accuracy did not improve from 0.81437
Epoch 33/50
63/63 [==============================] - 1s 13ms/step - loss: 0.0416 - accuracy: 0.9973 - val_loss: 1.9759 - val_accuracy: 0.7485Epoch 00033: val_accuracy did not improve from 0.81437
Epoch 34/50
63/63 [==============================] - 1s 14ms/step - loss: 0.0351 - accuracy: 0.9977 - val_loss: 1.9759 - val_accuracy: 0.7485Epoch 00034: val_accuracy did not improve from 0.81437
Epoch 35/50
63/63 [==============================] - 1s 13ms/step - loss: 0.0467 - accuracy: 0.9970 - val_loss: 1.9759 - val_accuracy: 0.7485Epoch 00035: val_accuracy did not improve from 0.81437
Epoch 36/50
63/63 [==============================] - 1s 13ms/step - loss: 0.0197 - accuracy: 0.9987 - val_loss: 1.9759 - val_accuracy: 0.7485Epoch 00036: val_accuracy did not improve from 0.81437
Epoch 37/50
63/63 [==============================] - 1s 12ms/step - loss: 0.0113 - accuracy: 0.9993 - val_loss: 1.9759 - val_accuracy: 0.7485Epoch 00037: val_accuracy did not improve from 0.81437
Epoch 38/50
63/63 [==============================] - 1s 12ms/step - loss: 0.0317 - accuracy: 0.9979 - val_loss: 1.9759 - val_accuracy: 0.7485Epoch 00038: val_accuracy did not improve from 0.81437
Epoch 39/50
63/63 [==============================] - 1s 13ms/step - loss: 0.0348 - accuracy: 0.9977 - val_loss: 1.9759 - val_accuracy: 0.7485Epoch 00039: val_accuracy did not improve from 0.81437
Epoch 40/50
63/63 [==============================] - 1s 12ms/step - loss: 0.0219 - accuracy: 0.9986 - val_loss: 1.9759 - val_accuracy: 0.7485Epoch 00040: val_accuracy did not improve from 0.81437
Epoch 41/50
63/63 [==============================] - 1s 13ms/step - loss: 0.0225 - accuracy: 0.9985 - val_loss: 1.9759 - val_accuracy: 0.7485Epoch 00041: val_accuracy did not improve from 0.81437
Epoch 42/50
63/63 [==============================] - 1s 12ms/step - loss: 0.0189 - accuracy: 0.9988 - val_loss: 1.9759 - val_accuracy: 0.7485Epoch 00042: val_accuracy did not improve from 0.81437
BaseTextCNN_match运行时间: 0.72 mins
INFO:tensorflow:Assets written to: H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\BaseTextCNN_match\save\BaseTextCNN_match\assets
保存成功 保存地址在 H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\BaseTextCNN_match\save

DSSM

base_network = _make_multi_layer_perceptron_layer()input_left=Input(name='text_left', shape=(max_len,))
input_right = Input(name='text_right', shape=(max_len,))
# embedding = Embedding(vocab_size, embedding_size)
# embed_left = embedding(input_left)
# embed_right = embedding(input_right)
x = [base_network(input_left),base_network(input_right)]x = Dot(axes=[1, 1], normalize=True)(x)# Make output layer
x_out =  Dense(1, activation='linear')(x)# self._backend =# out =
# self._backend = keras.Model(inputs=[query, doc], outputs=x_out)
model= keras.Model(inputs=[input_left, input_right], outputs=x_out,name="DSSM")
model.summary()

Model: "DSSM"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to
==================================================================================================
text_left (InputLayer)          [(None, 15)]         0
__________________________________________________________________________________________________
text_right (InputLayer)         [(None, 15)]         0
__________________________________________________________________________________________________
dense_12 (Dense)                (None, 128)          2048        text_left[0][0]
__________________________________________________________________________________________________
dense_16 (Dense)                (None, 128)          2048        text_right[0][0]
__________________________________________________________________________________________________
dense_13 (Dense)                (None, 128)          16512       dense_12[0][0]
__________________________________________________________________________________________________
dense_17 (Dense)                (None, 128)          16512       dense_16[0][0]
__________________________________________________________________________________________________
dense_14 (Dense)                (None, 128)          16512       dense_13[0][0]
__________________________________________________________________________________________________
dense_18 (Dense)                (None, 128)          16512       dense_17[0][0]
__________________________________________________________________________________________________
dense_15 (Dense)                (None, 64)           8256        dense_14[0][0]
__________________________________________________________________________________________________
dense_19 (Dense)                (None, 64)           8256        dense_18[0][0]
__________________________________________________________________________________________________
dot_2 (Dot)                     (None, 1)            0           dense_15[0][0]                   dense_19[0][0]
__________________________________________________________________________________________________
dense_20 (Dense)                (None, 1)            2           dot_2[0][0]
==================================================================================================
Total params: 86,658
Trainable params: 86,658
Non-trainable params: 0
__________________________________________________________________________________________________

model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])# 各种callback函数
tb_callback = tf.keras.callbacks.TensorBoard(logdir, histogram_freq=1)
#如果accuracy在25个epoch后没有提升，学习率减半。
lr_callback = tf.keras.callbacks.ReduceLROnPlateau(monitor="accuracy",factor = 0.5, patience = 25)
#当accuracy在30个epoch后没有提升，则提前终止训练。
stop_callback = tf.keras.callbacks.EarlyStopping(monitor = "accuracy", patience= 30)#获取当前的地址
path=os.path.join(os.getcwd(),'outputs\model',model.name)import os
dirs = pathif not os.path.exists(dirs):os.makedirs(dirs)# 保留最好的模型
mc_callback=tf.keras.callbacks.ModelCheckpoint(filepath=os.path.join(path,'weights.hdf5'),monitor='val_accuracy',verbose=1, save_best_only=True)# 把训练轮结果数据流到 csv 文件的回调函数。
# url=os.path.join(os.getcwd(),'outputs')csv_callback=tf.keras.callbacks.CSVLogger(os.path.join(path,'training.log'))callbacks_list = [tb_callback,lr_callback,stop_callback,mc_callback,csv_callback]t = time()
history=model.fit(x=[sent1_train,sent2_train],y=train_labels,validation_data=([sent1_val,sent2_val],val_labels),epochs=50,\callbacks = callbacks_list)
print(model.name+'运行时间: {} mins'.format(round((time() - t) / 60, 2)))model.save(os.path.join(path,'save',model.name))
print('保存成功','保存地址在',os.path.join(path,'save'))

Epoch 1/50
63/63 [==============================] - 1s 13ms/step - loss: 2.9754 - accuracy: 0.8071 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00001: val_accuracy improved from -inf to 0.81437, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\DSSM\weights.hdf5
Epoch 2/50
63/63 [==============================] - 0s 6ms/step - loss: 2.8511 - accuracy: 0.8152 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00002: val_accuracy did not improve from 0.81437
Epoch 3/50
63/63 [==============================] - 0s 6ms/step - loss: 2.9824 - accuracy: 0.8066 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00003: val_accuracy did not improve from 0.81437
Epoch 4/50
63/63 [==============================] - 0s 6ms/step - loss: 2.9774 - accuracy: 0.8070 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00004: val_accuracy did not improve from 0.81437
Epoch 5/50
63/63 [==============================] - 0s 6ms/step - loss: 3.0168 - accuracy: 0.8044 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00005: val_accuracy did not improve from 0.81437
Epoch 6/50
63/63 [==============================] - 0s 6ms/step - loss: 3.0824 - accuracy: 0.8002 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00006: val_accuracy did not improve from 0.81437
Epoch 7/50
63/63 [==============================] - 0s 6ms/step - loss: 3.0305 - accuracy: 0.8035 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00007: val_accuracy did not improve from 0.81437
Epoch 8/50
63/63 [==============================] - 0s 6ms/step - loss: 3.1376 - accuracy: 0.7966 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00008: val_accuracy did not improve from 0.81437
Epoch 9/50
63/63 [==============================] - 0s 6ms/step - loss: 3.1666 - accuracy: 0.7947 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00009: val_accuracy did not improve from 0.81437
Epoch 10/50
63/63 [==============================] - 0s 6ms/step - loss: 2.7342 - accuracy: 0.8227 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00010: val_accuracy did not improve from 0.81437
Epoch 11/50
63/63 [==============================] - 0s 6ms/step - loss: 2.9489 - accuracy: 0.8088 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00011: val_accuracy did not improve from 0.81437
Epoch 12/50
63/63 [==============================] - 0s 6ms/step - loss: 3.0766 - accuracy: 0.8005 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00012: val_accuracy did not improve from 0.81437
Epoch 13/50
63/63 [==============================] - 0s 7ms/step - loss: 2.9885 - accuracy: 0.8063 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00013: val_accuracy did not improve from 0.81437
Epoch 14/50
63/63 [==============================] - 0s 6ms/step - loss: 2.9523 - accuracy: 0.8086 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00014: val_accuracy did not improve from 0.81437
Epoch 15/50
63/63 [==============================] - 0s 6ms/step - loss: 3.0373 - accuracy: 0.8031 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00015: val_accuracy did not improve from 0.81437
Epoch 16/50
63/63 [==============================] - 0s 6ms/step - loss: 2.9053 - accuracy: 0.8117 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00016: val_accuracy did not improve from 0.81437
Epoch 17/50
63/63 [==============================] - 0s 6ms/step - loss: 3.1339 - accuracy: 0.7968 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00017: val_accuracy did not improve from 0.81437
Epoch 18/50
63/63 [==============================] - 0s 6ms/step - loss: 3.2538 - accuracy: 0.7891 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00018: val_accuracy did not improve from 0.81437
Epoch 19/50
63/63 [==============================] - 0s 6ms/step - loss: 3.0011 - accuracy: 0.8054 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00019: val_accuracy did not improve from 0.81437
Epoch 20/50
63/63 [==============================] - 0s 6ms/step - loss: 2.8825 - accuracy: 0.8131 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00020: val_accuracy did not improve from 0.81437
Epoch 21/50
63/63 [==============================] - 0s 6ms/step - loss: 3.1495 - accuracy: 0.7958 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00021: val_accuracy did not improve from 0.81437
Epoch 22/50
63/63 [==============================] - 0s 6ms/step - loss: 2.9505 - accuracy: 0.8087 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00022: val_accuracy did not improve from 0.81437
Epoch 23/50
63/63 [==============================] - 0s 6ms/step - loss: 3.1017 - accuracy: 0.7989 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00023: val_accuracy did not improve from 0.81437
Epoch 24/50
63/63 [==============================] - 0s 6ms/step - loss: 2.9187 - accuracy: 0.8108 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00024: val_accuracy did not improve from 0.81437
Epoch 25/50
63/63 [==============================] - 0s 6ms/step - loss: 2.9029 - accuracy: 0.8118 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00025: val_accuracy did not improve from 0.81437
Epoch 26/50
63/63 [==============================] - 0s 6ms/step - loss: 2.8358 - accuracy: 0.8162 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00026: val_accuracy did not improve from 0.81437
Epoch 27/50
63/63 [==============================] - 0s 6ms/step - loss: 3.1718 - accuracy: 0.7944 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00027: val_accuracy did not improve from 0.81437
Epoch 28/50
63/63 [==============================] - 0s 6ms/step - loss: 3.0214 - accuracy: 0.8041 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00028: val_accuracy did not improve from 0.81437
Epoch 29/50
63/63 [==============================] - 0s 6ms/step - loss: 3.1342 - accuracy: 0.7968 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00029: val_accuracy did not improve from 0.81437
Epoch 30/50
63/63 [==============================] - 0s 6ms/step - loss: 2.9440 - accuracy: 0.8091 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00030: val_accuracy did not improve from 0.81437
Epoch 31/50
63/63 [==============================] - 0s 6ms/step - loss: 2.9426 - accuracy: 0.8092 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00031: val_accuracy did not improve from 0.81437
DSSM运行时间: 0.26 mins
INFO:tensorflow:Assets written to: H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\DSSM\save\DSSM\assets
保存成功 保存地址在 H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\DSSM\save

plot_metric(history,"loss")
plot_metric(history,"accuracy")

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-8LunGROM-1623898674925)(output_62_0.svg)]

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-gXawIqmy-1623898674925)(output_62_1.svg)]

# 以表格形式展示
dfhistory = pd.DataFrame(history.history)
dfhistory.index = range(1,len(dfhistory) + 1)
dfhistory.index.name = 'epoch'
dfhistory.to_csv(os.path.join(path,'save',model.name,model.name+'.csv'))dfhistory

	loss	accuracy	val_loss	val_accuracy	lr
epoch
1	2.992440	0.806	2.863314	0.814371	0.0010
2	2.992440	0.806	2.863314	0.814371	0.0010
3	2.992440	0.806	2.863314	0.814371	0.0010
4	2.992440	0.806	2.863314	0.814371	0.0010
5	2.992440	0.806	2.863314	0.814371	0.0010
6	2.992440	0.806	2.863314	0.814371	0.0010
7	2.992440	0.806	2.863314	0.814371	0.0010
8	2.992439	0.806	2.863314	0.814371	0.0010
9	2.992440	0.806	2.863314	0.814371	0.0010
10	2.992440	0.806	2.863314	0.814371	0.0010
11	2.992440	0.806	2.863314	0.814371	0.0010
12	2.992440	0.806	2.863314	0.814371	0.0010
13	2.992440	0.806	2.863314	0.814371	0.0010
14	2.992440	0.806	2.863314	0.814371	0.0010
15	2.992440	0.806	2.863314	0.814371	0.0010
16	2.992439	0.806	2.863314	0.814371	0.0010
17	2.992440	0.806	2.863314	0.814371	0.0010
18	2.992439	0.806	2.863314	0.814371	0.0010
19	2.992440	0.806	2.863314	0.814371	0.0010
20	2.992441	0.806	2.863314	0.814371	0.0010
21	2.992440	0.806	2.863314	0.814371	0.0010
22	2.992440	0.806	2.863314	0.814371	0.0010
23	2.992440	0.806	2.863314	0.814371	0.0010
24	2.992441	0.806	2.863314	0.814371	0.0010
25	2.992440	0.806	2.863314	0.814371	0.0010
26	2.992440	0.806	2.863314	0.814371	0.0010
27	2.992440	0.806	2.863314	0.814371	0.0005
28	2.992440	0.806	2.863314	0.814371	0.0005
29	2.992440	0.806	2.863314	0.814371	0.0005
30	2.992440	0.806	2.863314	0.814371	0.0005
31	2.992440	0.806	2.863314	0.814371	0.0005

ArcI

input1 = Input(name='text_left', shape=(max_len,))
input2 = Input(name='text_right', shape=(max_len,))
embedding = Embedding(vocab_size, embedding_size)
sent1_embed=embedding(input1)
sent2_embed=embedding(input2)
cnn1=Conv1D(filters=32,kernel_size=3,strides=1,padding='same',activation='relu',kernel_initializer='glorot_normal',bias_initializer='zeros')(sent1_embed)
cnn2=Conv1D(filters=32,kernel_size=3,strides=1,padding='same',activation='relu',kernel_initializer='glorot_normal',bias_initializer='zeros')(sent2_embed)
cnn1=MaxPooling1D(pool_size=2)(cnn1)
flat1=Flatten()(cnn1)
cnn2=MaxPooling1D(pool_size=2)(cnn2)
flat2=Flatten()(cnn2)
# concatenate_1=concatenate([input1,input2],axis=-1)
concat =Concatenate(axis=1)([flat1, flat2])
dropout = Dropout(rate=0)(concat)
dense1=Dense(64)(dropout)
dense2=Dense(32)(dense1)
# dense3=Dense(128)(dense2)
outputs=Dense(1,activation='sigmoid',name="output")(dense2)
model=Model(inputs=[input1,input2],outputs=outputs,name="ArcI")
model.summary()

Model: "ArcI"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to
==================================================================================================
text_left (InputLayer)          [(None, 15)]         0
__________________________________________________________________________________________________
text_right (InputLayer)         [(None, 15)]         0
__________________________________________________________________________________________________
embedding_2 (Embedding)         (None, 15, 128)      278400      text_left[0][0]                  text_right[0][0]
__________________________________________________________________________________________________
conv1d_8 (Conv1D)               (None, 15, 32)       12320       embedding_2[0][0]
__________________________________________________________________________________________________
conv1d_9 (Conv1D)               (None, 15, 32)       12320       embedding_2[1][0]
__________________________________________________________________________________________________
max_pooling1d (MaxPooling1D)    (None, 7, 32)        0           conv1d_8[0][0]
__________________________________________________________________________________________________
max_pooling1d_1 (MaxPooling1D)  (None, 7, 32)        0           conv1d_9[0][0]
__________________________________________________________________________________________________
flatten (Flatten)               (None, 224)          0           max_pooling1d[0][0]
__________________________________________________________________________________________________
flatten_1 (Flatten)             (None, 224)          0           max_pooling1d_1[0][0]
__________________________________________________________________________________________________
concatenate_4 (Concatenate)     (None, 448)          0           flatten[0][0]                    flatten_1[0][0]
__________________________________________________________________________________________________
dropout_2 (Dropout)             (None, 448)          0           concatenate_4[0][0]
__________________________________________________________________________________________________
dense_21 (Dense)                (None, 64)           28736       dropout_2[0][0]
__________________________________________________________________________________________________
dense_22 (Dense)                (None, 32)           2080        dense_21[0][0]
__________________________________________________________________________________________________
output (Dense)                  (None, 1)            33          dense_22[0][0]
==================================================================================================
Total params: 333,889
Trainable params: 333,889
Non-trainable params: 0
__________________________________________________________________________________________________

model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])# 各种callback函数
tb_callback = tf.keras.callbacks.TensorBoard(logdir, histogram_freq=1)
#如果accuracy在25个epoch后没有提升，学习率减半。
lr_callback = tf.keras.callbacks.ReduceLROnPlateau(monitor="accuracy",factor = 0.5, patience = 25)
#当accuracy在30个epoch后没有提升，则提前终止训练。
stop_callback = tf.keras.callbacks.EarlyStopping(monitor = "accuracy", patience= 30)#获取当前的地址
path=os.path.join(os.getcwd(),'outputs\model',model.name)import os
dirs = pathif not os.path.exists(dirs):os.makedirs(dirs)# 保留最好的模型
mc_callback=tf.keras.callbacks.ModelCheckpoint(filepath=os.path.join(path,'weights.hdf5'),monitor='val_accuracy',verbose=1, save_best_only=True)# 把训练轮结果数据流到 csv 文件的回调函数。
# url=os.path.join(os.getcwd(),'outputs')csv_callback=tf.keras.callbacks.CSVLogger(os.path.join(path,'training.log'))callbacks_list = [tb_callback,lr_callback,stop_callback,mc_callback,csv_callback]t = time()
history=model.fit(x=[sent1_train,sent2_train],y=train_labels,validation_data=([sent1_val,sent2_val],val_labels),epochs=50,\callbacks = callbacks_list)
print(model.name+'运行时间: {} mins'.format(round((time() - t) / 60, 2)))model.save(os.path.join(path,'save',model.name))
print('保存成功','保存地址在',os.path.join(path,'save'))

Epoch 1/50
63/63 [==============================] - 1s 16ms/step - loss: 0.5292 - accuracy: 0.7885 - val_loss: 0.4754 - val_accuracy: 0.8144Epoch 00001: val_accuracy improved from -inf to 0.81437, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\ArcI\weights.hdf5
Epoch 2/50
63/63 [==============================] - 1s 8ms/step - loss: 0.4580 - accuracy: 0.7951 - val_loss: 0.4823 - val_accuracy: 0.8144Epoch 00002: val_accuracy did not improve from 0.81437
Epoch 3/50
63/63 [==============================] - 1s 8ms/step - loss: 0.3164 - accuracy: 0.8324 - val_loss: 0.6373 - val_accuracy: 0.7804Epoch 00003: val_accuracy did not improve from 0.81437
Epoch 4/50
63/63 [==============================] - 1s 8ms/step - loss: 0.1099 - accuracy: 0.9617 - val_loss: 0.9353 - val_accuracy: 0.7745Epoch 00004: val_accuracy did not improve from 0.81437
Epoch 5/50
63/63 [==============================] - 1s 9ms/step - loss: 0.0341 - accuracy: 0.9908 - val_loss: 1.1799 - val_accuracy: 0.7305Epoch 00005: val_accuracy did not improve from 0.81437
Epoch 6/50
63/63 [==============================] - 1s 8ms/step - loss: 0.0083 - accuracy: 0.9990 - val_loss: 1.4070 - val_accuracy: 0.7565Epoch 00006: val_accuracy did not improve from 0.81437
Epoch 7/50
63/63 [==============================] - 1s 9ms/step - loss: 0.0024 - accuracy: 1.0000 - val_loss: 1.5152 - val_accuracy: 0.7764Epoch 00007: val_accuracy did not improve from 0.81437
Epoch 8/50
63/63 [==============================] - 1s 8ms/step - loss: 0.0011 - accuracy: 1.0000 - val_loss: 1.5363 - val_accuracy: 0.7465Epoch 00008: val_accuracy did not improve from 0.81437
Epoch 9/50
63/63 [==============================] - 1s 8ms/step - loss: 5.5433e-04 - accuracy: 1.0000 - val_loss: 1.5963 - val_accuracy: 0.7505Epoch 00009: val_accuracy did not improve from 0.81437
Epoch 10/50
63/63 [==============================] - 1s 8ms/step - loss: 4.5467e-04 - accuracy: 1.0000 - val_loss: 1.6363 - val_accuracy: 0.7525Epoch 00010: val_accuracy did not improve from 0.81437
Epoch 11/50
63/63 [==============================] - 1s 8ms/step - loss: 3.1880e-04 - accuracy: 1.0000 - val_loss: 1.6832 - val_accuracy: 0.7565Epoch 00011: val_accuracy did not improve from 0.81437
Epoch 12/50
63/63 [==============================] - 1s 8ms/step - loss: 2.2599e-04 - accuracy: 1.0000 - val_loss: 1.7060 - val_accuracy: 0.7545Epoch 00012: val_accuracy did not improve from 0.81437
Epoch 13/50
63/63 [==============================] - 1s 9ms/step - loss: 2.0653e-04 - accuracy: 1.0000 - val_loss: 1.7369 - val_accuracy: 0.7545Epoch 00013: val_accuracy did not improve from 0.81437
Epoch 14/50
63/63 [==============================] - 1s 8ms/step - loss: 1.7706e-04 - accuracy: 1.0000 - val_loss: 1.7675 - val_accuracy: 0.7545Epoch 00014: val_accuracy did not improve from 0.81437
Epoch 15/50
63/63 [==============================] - 1s 8ms/step - loss: 1.4694e-04 - accuracy: 1.0000 - val_loss: 1.7911 - val_accuracy: 0.7525Epoch 00015: val_accuracy did not improve from 0.81437
Epoch 16/50
63/63 [==============================] - 1s 8ms/step - loss: 1.3680e-04 - accuracy: 1.0000 - val_loss: 1.8178 - val_accuracy: 0.7525Epoch 00016: val_accuracy did not improve from 0.81437
Epoch 17/50
63/63 [==============================] - 1s 9ms/step - loss: 1.1411e-04 - accuracy: 1.0000 - val_loss: 1.8362 - val_accuracy: 0.7525Epoch 00017: val_accuracy did not improve from 0.81437
Epoch 18/50
63/63 [==============================] - 1s 9ms/step - loss: 1.0103e-04 - accuracy: 1.0000 - val_loss: 1.8574 - val_accuracy: 0.7505Epoch 00018: val_accuracy did not improve from 0.81437
Epoch 19/50
63/63 [==============================] - 1s 10ms/step - loss: 8.8149e-05 - accuracy: 1.0000 - val_loss: 1.8767 - val_accuracy: 0.7505Epoch 00019: val_accuracy did not improve from 0.81437
Epoch 20/50
63/63 [==============================] - 1s 8ms/step - loss: 8.0321e-05 - accuracy: 1.0000 - val_loss: 1.8971 - val_accuracy: 0.7505Epoch 00020: val_accuracy did not improve from 0.81437
Epoch 21/50
63/63 [==============================] - 1s 8ms/step - loss: 6.7310e-05 - accuracy: 1.0000 - val_loss: 1.9162 - val_accuracy: 0.7505Epoch 00021: val_accuracy did not improve from 0.81437
Epoch 22/50
63/63 [==============================] - 1s 8ms/step - loss: 6.2473e-05 - accuracy: 1.0000 - val_loss: 1.9336 - val_accuracy: 0.7505Epoch 00022: val_accuracy did not improve from 0.81437
Epoch 23/50
63/63 [==============================] - 1s 9ms/step - loss: 5.3186e-05 - accuracy: 1.0000 - val_loss: 1.9502 - val_accuracy: 0.7505Epoch 00023: val_accuracy did not improve from 0.81437
Epoch 24/50
63/63 [==============================] - 1s 9ms/step - loss: 4.9954e-05 - accuracy: 1.0000 - val_loss: 1.9679 - val_accuracy: 0.7505Epoch 00024: val_accuracy did not improve from 0.81437
Epoch 25/50
63/63 [==============================] - 1s 8ms/step - loss: 4.3161e-05 - accuracy: 1.0000 - val_loss: 1.9824 - val_accuracy: 0.7505Epoch 00025: val_accuracy did not improve from 0.81437
Epoch 26/50
63/63 [==============================] - 1s 8ms/step - loss: 4.5415e-05 - accuracy: 1.0000 - val_loss: 1.9985 - val_accuracy: 0.7505Epoch 00026: val_accuracy did not improve from 0.81437
Epoch 27/50
63/63 [==============================] - 1s 9ms/step - loss: 3.9223e-05 - accuracy: 1.0000 - val_loss: 2.0130 - val_accuracy: 0.7505Epoch 00027: val_accuracy did not improve from 0.81437
Epoch 28/50
63/63 [==============================] - 1s 8ms/step - loss: 3.7845e-05 - accuracy: 1.0000 - val_loss: 2.0262 - val_accuracy: 0.7505Epoch 00028: val_accuracy did not improve from 0.81437
Epoch 29/50
63/63 [==============================] - 1s 9ms/step - loss: 3.3452e-05 - accuracy: 1.0000 - val_loss: 2.0399 - val_accuracy: 0.7485Epoch 00029: val_accuracy did not improve from 0.81437
Epoch 30/50
63/63 [==============================] - 1s 8ms/step - loss: 3.2305e-05 - accuracy: 1.0000 - val_loss: 2.0535 - val_accuracy: 0.7485Epoch 00030: val_accuracy did not improve from 0.81437
Epoch 31/50
63/63 [==============================] - 1s 8ms/step - loss: 3.0399e-05 - accuracy: 1.0000 - val_loss: 2.0668 - val_accuracy: 0.7505Epoch 00031: val_accuracy did not improve from 0.81437
Epoch 32/50
63/63 [==============================] - 1s 8ms/step - loss: 2.5758e-05 - accuracy: 1.0000 - val_loss: 2.0806 - val_accuracy: 0.7505Epoch 00032: val_accuracy did not improve from 0.81437
Epoch 33/50
63/63 [==============================] - 1s 8ms/step - loss: 2.3437e-05 - accuracy: 1.0000 - val_loss: 2.0864 - val_accuracy: 0.7525Epoch 00033: val_accuracy did not improve from 0.81437
Epoch 34/50
63/63 [==============================] - 1s 9ms/step - loss: 2.3289e-05 - accuracy: 1.0000 - val_loss: 2.0922 - val_accuracy: 0.7505Epoch 00034: val_accuracy did not improve from 0.81437
Epoch 35/50
63/63 [==============================] - 1s 9ms/step - loss: 2.3398e-05 - accuracy: 1.0000 - val_loss: 2.0987 - val_accuracy: 0.7505Epoch 00035: val_accuracy did not improve from 0.81437
Epoch 36/50
63/63 [==============================] - 1s 8ms/step - loss: 2.2757e-05 - accuracy: 1.0000 - val_loss: 2.1050 - val_accuracy: 0.7505Epoch 00036: val_accuracy did not improve from 0.81437
Epoch 37/50
63/63 [==============================] - 1s 9ms/step - loss: 2.0153e-05 - accuracy: 1.0000 - val_loss: 2.1116 - val_accuracy: 0.7525Epoch 00037: val_accuracy did not improve from 0.81437
ArcI运行时间: 0.41 mins
INFO:tensorflow:Assets written to: H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\ArcI\save\ArcI\assets
保存成功 保存地址在 H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\ArcI\save

plot_metric(history,"loss")
plot_metric(history,"accuracy")

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-BcHVgoNZ-1623898674926)(output_67_0.svg)]

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-kg40iuAZ-1623898674926)(output_67_1.svg)]

# 以表格形式展示
dfhistory = pd.DataFrame(history.history)
dfhistory.index = range(1,len(dfhistory) + 1)
dfhistory.index.name = 'epoch'
dfhistory.to_csv(os.path.join(path,'save',model.name,model.name+'.csv'))dfhistory

	loss	accuracy	val_loss	val_accuracy	lr
epoch
1	0.503494	0.8030	0.475359	0.814371	0.0010
2	0.438728	0.8060	0.482319	0.814371	0.0010
3	0.302796	0.8500	0.637273	0.780439	0.0010
4	0.116107	0.9580	0.935277	0.774451	0.0010
5	0.033786	0.9910	1.179889	0.730539	0.0010
6	0.008575	0.9985	1.406960	0.756487	0.0010
7	0.002379	1.0000	1.515246	0.776447	0.0010
8	0.000971	1.0000	1.536348	0.746507	0.0010
9	0.000542	1.0000	1.596271	0.750499	0.0010
10	0.000397	1.0000	1.636328	0.752495	0.0010
11	0.000304	1.0000	1.683157	0.756487	0.0010
12	0.000246	1.0000	1.706022	0.754491	0.0010
13	0.000203	1.0000	1.736940	0.754491	0.0010
14	0.000171	1.0000	1.767536	0.754491	0.0010
15	0.000146	1.0000	1.791100	0.752495	0.0010
16	0.000126	1.0000	1.817774	0.752495	0.0010
17	0.000110	1.0000	1.836237	0.752495	0.0010
18	0.000097	1.0000	1.857435	0.750499	0.0010
19	0.000086	1.0000	1.876659	0.750499	0.0010
20	0.000077	1.0000	1.897138	0.750499	0.0010
21	0.000069	1.0000	1.916157	0.750499	0.0010
22	0.000062	1.0000	1.933603	0.750499	0.0010
23	0.000056	1.0000	1.950206	0.750499	0.0010
24	0.000051	1.0000	1.967885	0.750499	0.0010
25	0.000047	1.0000	1.982365	0.750499	0.0010
26	0.000043	1.0000	1.998452	0.750499	0.0010
27	0.000039	1.0000	2.013018	0.750499	0.0010
28	0.000036	1.0000	2.026198	0.750499	0.0010
29	0.000033	1.0000	2.039930	0.748503	0.0010
30	0.000031	1.0000	2.053540	0.748503	0.0010
31	0.000029	1.0000	2.066777	0.750499	0.0010
32	0.000027	1.0000	2.080627	0.750499	0.0010
33	0.000025	1.0000	2.086367	0.752495	0.0005
34	0.000024	1.0000	2.092232	0.750499	0.0005
35	0.000023	1.0000	2.098683	0.750499	0.0005
36	0.000023	1.0000	2.104959	0.750499	0.0005
37	0.000022	1.0000	2.111558	0.752495	0.0005

KNRM

input1 = Input(name='text_left', shape=(max_len,))
input2 = Input(name='text_right', shape=(max_len,))
embedding = Embedding(vocab_size, embedding_size)
sent1_embed=embedding(input1)
sent2_embed=embedding(input2)
mm =Dot(axes=[2, 2], normalize=True)([sent1_embed, sent2_embed])
KM = []
for i in range(11):mu = 1. / (11 - 1) + (2. * i) / (11 - 1) - 1.0sigma = 0.1if mu > 1.0:sigma = 0.001mu = 1.0mm_exp = _kernel_layer(mu, sigma)(mm)mm_doc_sum = Lambda(lambda x: tf.reduce_sum(x, 2))(mm_exp)mm_log = Activation(tf.math.log1p)(mm_doc_sum)mm_sum = Lambda(lambda x: tf.reduce_sum(x, 1))(mm_log)KM.append(mm_sum)phi = Lambda(lambda x: tf.stack(x, 1))(KM)
outputs=Dense(1,activation='sigmoid',name="output")(phi)
model=Model(inputs=[input1,input2],outputs=outputs,name="KNRM")
model.summary()

Model: "KNRM"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to
==================================================================================================
text_left (InputLayer)          [(None, 15)]         0
__________________________________________________________________________________________________
text_right (InputLayer)         [(None, 15)]         0
__________________________________________________________________________________________________
embedding_3 (Embedding)         (None, 15, 128)      278400      text_left[0][0]                  text_right[0][0]
__________________________________________________________________________________________________
dot_3 (Dot)                     (None, 15, 15)       0           embedding_3[0][0]                embedding_3[1][0]
__________________________________________________________________________________________________
activation (Activation)         (None, 15, 15)       0           dot_3[0][0]
__________________________________________________________________________________________________
activation_2 (Activation)       (None, 15, 15)       0           dot_3[0][0]
__________________________________________________________________________________________________
activation_4 (Activation)       (None, 15, 15)       0           dot_3[0][0]
__________________________________________________________________________________________________
activation_6 (Activation)       (None, 15, 15)       0           dot_3[0][0]
__________________________________________________________________________________________________
activation_8 (Activation)       (None, 15, 15)       0           dot_3[0][0]
__________________________________________________________________________________________________
activation_10 (Activation)      (None, 15, 15)       0           dot_3[0][0]
__________________________________________________________________________________________________
activation_12 (Activation)      (None, 15, 15)       0           dot_3[0][0]
__________________________________________________________________________________________________
activation_14 (Activation)      (None, 15, 15)       0           dot_3[0][0]
__________________________________________________________________________________________________
activation_16 (Activation)      (None, 15, 15)       0           dot_3[0][0]
__________________________________________________________________________________________________
activation_18 (Activation)      (None, 15, 15)       0           dot_3[0][0]
__________________________________________________________________________________________________
activation_20 (Activation)      (None, 15, 15)       0           dot_3[0][0]
__________________________________________________________________________________________________
lambda (Lambda)                 (None, 15)           0           activation[0][0]
__________________________________________________________________________________________________
lambda_2 (Lambda)               (None, 15)           0           activation_2[0][0]
__________________________________________________________________________________________________
lambda_4 (Lambda)               (None, 15)           0           activation_4[0][0]
__________________________________________________________________________________________________
lambda_6 (Lambda)               (None, 15)           0           activation_6[0][0]
__________________________________________________________________________________________________
lambda_8 (Lambda)               (None, 15)           0           activation_8[0][0]
__________________________________________________________________________________________________
lambda_10 (Lambda)              (None, 15)           0           activation_10[0][0]
__________________________________________________________________________________________________
lambda_12 (Lambda)              (None, 15)           0           activation_12[0][0]
__________________________________________________________________________________________________
lambda_14 (Lambda)              (None, 15)           0           activation_14[0][0]
__________________________________________________________________________________________________
lambda_16 (Lambda)              (None, 15)           0           activation_16[0][0]
__________________________________________________________________________________________________
lambda_18 (Lambda)              (None, 15)           0           activation_18[0][0]
__________________________________________________________________________________________________
lambda_20 (Lambda)              (None, 15)           0           activation_20[0][0]
__________________________________________________________________________________________________
activation_1 (Activation)       (None, 15)           0           lambda[0][0]
__________________________________________________________________________________________________
activation_3 (Activation)       (None, 15)           0           lambda_2[0][0]
__________________________________________________________________________________________________
activation_5 (Activation)       (None, 15)           0           lambda_4[0][0]
__________________________________________________________________________________________________
activation_7 (Activation)       (None, 15)           0           lambda_6[0][0]
__________________________________________________________________________________________________
activation_9 (Activation)       (None, 15)           0           lambda_8[0][0]
__________________________________________________________________________________________________
activation_11 (Activation)      (None, 15)           0           lambda_10[0][0]
__________________________________________________________________________________________________
activation_13 (Activation)      (None, 15)           0           lambda_12[0][0]
__________________________________________________________________________________________________
activation_15 (Activation)      (None, 15)           0           lambda_14[0][0]
__________________________________________________________________________________________________
activation_17 (Activation)      (None, 15)           0           lambda_16[0][0]
__________________________________________________________________________________________________
activation_19 (Activation)      (None, 15)           0           lambda_18[0][0]
__________________________________________________________________________________________________
activation_21 (Activation)      (None, 15)           0           lambda_20[0][0]
__________________________________________________________________________________________________
lambda_1 (Lambda)               (None,)              0           activation_1[0][0]
__________________________________________________________________________________________________
lambda_3 (Lambda)               (None,)              0           activation_3[0][0]
__________________________________________________________________________________________________
lambda_5 (Lambda)               (None,)              0           activation_5[0][0]
__________________________________________________________________________________________________
lambda_7 (Lambda)               (None,)              0           activation_7[0][0]
__________________________________________________________________________________________________
lambda_9 (Lambda)               (None,)              0           activation_9[0][0]
__________________________________________________________________________________________________
lambda_11 (Lambda)              (None,)              0           activation_11[0][0]
__________________________________________________________________________________________________
lambda_13 (Lambda)              (None,)              0           activation_13[0][0]
__________________________________________________________________________________________________
lambda_15 (Lambda)              (None,)              0           activation_15[0][0]
__________________________________________________________________________________________________
lambda_17 (Lambda)              (None,)              0           activation_17[0][0]
__________________________________________________________________________________________________
lambda_19 (Lambda)              (None,)              0           activation_19[0][0]
__________________________________________________________________________________________________
lambda_21 (Lambda)              (None,)              0           activation_21[0][0]
__________________________________________________________________________________________________
lambda_22 (Lambda)              (None, 11)           0           lambda_1[0][0]                   lambda_3[0][0]                   lambda_5[0][0]                   lambda_7[0][0]                   lambda_9[0][0]                   lambda_11[0][0]                  lambda_13[0][0]                  lambda_15[0][0]                  lambda_17[0][0]                  lambda_19[0][0]                  lambda_21[0][0]
__________________________________________________________________________________________________
output (Dense)                  (None, 1)            12          lambda_22[0][0]
==================================================================================================
Total params: 278,412
Trainable params: 278,412
Non-trainable params: 0
__________________________________________________________________________________________________

model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])# 各种callback函数
tb_callback = tf.keras.callbacks.TensorBoard(logdir, histogram_freq=1)
#如果accuracy在25个epoch后没有提升，学习率减半。
lr_callback = tf.keras.callbacks.ReduceLROnPlateau(monitor="accuracy",factor = 0.5, patience = 25)
#当accuracy在30个epoch后没有提升，则提前终止训练。
stop_callback = tf.keras.callbacks.EarlyStopping(monitor = "accuracy", patience= 30)#获取当前的地址
path=os.path.join(os.getcwd(),'outputs\model',model.name)import os
dirs = pathif not os.path.exists(dirs):os.makedirs(dirs)# 保留最好的模型
mc_callback=tf.keras.callbacks.ModelCheckpoint(filepath=os.path.join(path,'weights.hdf5'),monitor='val_accuracy',verbose=1, save_best_only=True)# 把训练轮结果数据流到 csv 文件的回调函数。
# url=os.path.join(os.getcwd(),'outputs')csv_callback=tf.keras.callbacks.CSVLogger(os.path.join(path,'training.log'))callbacks_list = [tb_callback,lr_callback,stop_callback,mc_callback,csv_callback]t = time()
history=model.fit(x=[sent1_train,sent2_train],y=train_labels,validation_data=([sent1_val,sent2_val],val_labels),epochs=50,\callbacks = callbacks_list)
print(model.name+'运行时间: {} mins'.format(round((time() - t) / 60, 2)))model.save(os.path.join(path,'save',model.name))
print('保存成功','保存地址在',os.path.join(path,'save'))

Epoch 1/50
63/63 [==============================] - 3s 25ms/step - loss: 0.6209 - accuracy: 0.7793 - val_loss: 0.4904 - val_accuracy: 0.7944Epoch 00001: val_accuracy improved from -inf to 0.79441, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\KNRM\weights.hdf5
Epoch 2/50
63/63 [==============================] - 1s 15ms/step - loss: 0.4332 - accuracy: 0.8002 - val_loss: 0.4921 - val_accuracy: 0.8084Epoch 00002: val_accuracy improved from 0.79441 to 0.80838, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\KNRM\weights.hdf5
Epoch 3/50
63/63 [==============================] - 1s 16ms/step - loss: 0.4037 - accuracy: 0.8160 - val_loss: 0.5026 - val_accuracy: 0.7964Epoch 00003: val_accuracy did not improve from 0.80838
Epoch 4/50
63/63 [==============================] - 1s 15ms/step - loss: 0.3563 - accuracy: 0.8346 - val_loss: 0.5039 - val_accuracy: 0.7844Epoch 00004: val_accuracy did not improve from 0.80838
Epoch 5/50
63/63 [==============================] - 1s 16ms/step - loss: 0.3292 - accuracy: 0.8552 - val_loss: 0.5306 - val_accuracy: 0.8104Epoch 00005: val_accuracy improved from 0.80838 to 0.81038, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\KNRM\weights.hdf5
Epoch 6/50
63/63 [==============================] - 1s 15ms/step - loss: 0.2989 - accuracy: 0.8607 - val_loss: 0.5253 - val_accuracy: 0.7764Epoch 00006: val_accuracy did not improve from 0.81038
Epoch 7/50
63/63 [==============================] - 1s 15ms/step - loss: 0.2804 - accuracy: 0.8828 - val_loss: 0.5333 - val_accuracy: 0.7944Epoch 00007: val_accuracy did not improve from 0.81038
Epoch 8/50
63/63 [==============================] - 1s 15ms/step - loss: 0.2495 - accuracy: 0.8869 - val_loss: 0.5279 - val_accuracy: 0.7864Epoch 00008: val_accuracy did not improve from 0.81038
Epoch 9/50
63/63 [==============================] - 1s 15ms/step - loss: 0.2454 - accuracy: 0.9077 - val_loss: 0.5420 - val_accuracy: 0.7864Epoch 00009: val_accuracy did not improve from 0.81038
Epoch 10/50
63/63 [==============================] - 1s 15ms/step - loss: 0.2218 - accuracy: 0.9159 - val_loss: 0.5586 - val_accuracy: 0.7804Epoch 00010: val_accuracy did not improve from 0.81038
Epoch 11/50
63/63 [==============================] - 1s 15ms/step - loss: 0.2041 - accuracy: 0.9302 - val_loss: 0.5507 - val_accuracy: 0.7764Epoch 00011: val_accuracy did not improve from 0.81038
Epoch 12/50
63/63 [==============================] - 1s 17ms/step - loss: 0.1777 - accuracy: 0.9380 - val_loss: 0.5723 - val_accuracy: 0.7465Epoch 00012: val_accuracy did not improve from 0.81038
Epoch 13/50
63/63 [==============================] - 1s 15ms/step - loss: 0.1812 - accuracy: 0.9461 - val_loss: 0.5799 - val_accuracy: 0.7565Epoch 00013: val_accuracy did not improve from 0.81038
Epoch 14/50
63/63 [==============================] - 1s 15ms/step - loss: 0.1676 - accuracy: 0.9510 - val_loss: 0.5875 - val_accuracy: 0.7465Epoch 00014: val_accuracy did not improve from 0.81038
Epoch 15/50
63/63 [==============================] - 1s 15ms/step - loss: 0.1485 - accuracy: 0.9644 - val_loss: 0.5936 - val_accuracy: 0.7545Epoch 00015: val_accuracy did not improve from 0.81038
Epoch 16/50
63/63 [==============================] - 1s 15ms/step - loss: 0.1424 - accuracy: 0.9641 - val_loss: 0.6198 - val_accuracy: 0.7505Epoch 00016: val_accuracy did not improve from 0.81038
Epoch 17/50
63/63 [==============================] - 1s 16ms/step - loss: 0.1340 - accuracy: 0.9714 - val_loss: 0.6263 - val_accuracy: 0.7565Epoch 00017: val_accuracy did not improve from 0.81038
Epoch 18/50
63/63 [==============================] - 1s 15ms/step - loss: 0.1283 - accuracy: 0.9718 - val_loss: 0.6423 - val_accuracy: 0.7585Epoch 00018: val_accuracy did not improve from 0.81038
Epoch 19/50
63/63 [==============================] - 1s 15ms/step - loss: 0.1257 - accuracy: 0.9697 - val_loss: 0.6530 - val_accuracy: 0.7445Epoch 00019: val_accuracy did not improve from 0.81038
Epoch 20/50
63/63 [==============================] - 1s 15ms/step - loss: 0.1158 - accuracy: 0.9797 - val_loss: 0.6629 - val_accuracy: 0.7505Epoch 00020: val_accuracy did not improve from 0.81038
Epoch 21/50
63/63 [==============================] - 1s 15ms/step - loss: 0.1111 - accuracy: 0.9808 - val_loss: 0.6612 - val_accuracy: 0.7525Epoch 00021: val_accuracy did not improve from 0.81038
Epoch 22/50
63/63 [==============================] - 1s 16ms/step - loss: 0.1020 - accuracy: 0.9780 - val_loss: 0.6748 - val_accuracy: 0.7505Epoch 00022: val_accuracy did not improve from 0.81038
Epoch 23/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0907 - accuracy: 0.9854 - val_loss: 0.6868 - val_accuracy: 0.7525Epoch 00023: val_accuracy did not improve from 0.81038
Epoch 24/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0907 - accuracy: 0.9899 - val_loss: 0.7120 - val_accuracy: 0.7325Epoch 00024: val_accuracy did not improve from 0.81038
Epoch 25/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0874 - accuracy: 0.9837 - val_loss: 0.7250 - val_accuracy: 0.7146Epoch 00025: val_accuracy did not improve from 0.81038
Epoch 26/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0779 - accuracy: 0.9919 - val_loss: 0.7326 - val_accuracy: 0.7465Epoch 00026: val_accuracy did not improve from 0.81038
Epoch 27/50
63/63 [==============================] - 1s 16ms/step - loss: 0.0692 - accuracy: 0.9925 - val_loss: 0.7357 - val_accuracy: 0.7465Epoch 00027: val_accuracy did not improve from 0.81038
Epoch 28/50
63/63 [==============================] - 1s 16ms/step - loss: 0.0730 - accuracy: 0.9918 - val_loss: 0.7822 - val_accuracy: 0.7046Epoch 00028: val_accuracy did not improve from 0.81038
Epoch 29/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0698 - accuracy: 0.9865 - val_loss: 0.7895 - val_accuracy: 0.7305Epoch 00029: val_accuracy did not improve from 0.81038
Epoch 30/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0638 - accuracy: 0.9921 - val_loss: 0.7858 - val_accuracy: 0.7265Epoch 00030: val_accuracy did not improve from 0.81038
Epoch 31/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0613 - accuracy: 0.9931 - val_loss: 0.7930 - val_accuracy: 0.7345Epoch 00031: val_accuracy did not improve from 0.81038
Epoch 32/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0586 - accuracy: 0.9937 - val_loss: 0.8081 - val_accuracy: 0.7485Epoch 00032: val_accuracy did not improve from 0.81038
Epoch 33/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0554 - accuracy: 0.9921 - val_loss: 0.7990 - val_accuracy: 0.7525Epoch 00033: val_accuracy did not improve from 0.81038
Epoch 34/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0484 - accuracy: 0.9965 - val_loss: 0.8135 - val_accuracy: 0.7385Epoch 00034: val_accuracy did not improve from 0.81038
Epoch 35/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0453 - accuracy: 0.9968 - val_loss: 0.8380 - val_accuracy: 0.7405Epoch 00035: val_accuracy did not improve from 0.81038
Epoch 36/50
63/63 [==============================] - 1s 16ms/step - loss: 0.0495 - accuracy: 0.9945 - val_loss: 0.8192 - val_accuracy: 0.7485Epoch 00036: val_accuracy did not improve from 0.81038
Epoch 37/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0437 - accuracy: 0.9942 - val_loss: 0.8558 - val_accuracy: 0.7305Epoch 00037: val_accuracy did not improve from 0.81038
Epoch 38/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0395 - accuracy: 0.9979 - val_loss: 0.8623 - val_accuracy: 0.7186Epoch 00038: val_accuracy did not improve from 0.81038
Epoch 39/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0393 - accuracy: 0.9975 - val_loss: 0.8667 - val_accuracy: 0.7345Epoch 00039: val_accuracy did not improve from 0.81038
Epoch 40/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0354 - accuracy: 0.9990 - val_loss: 0.8890 - val_accuracy: 0.7026Epoch 00040: val_accuracy did not improve from 0.81038
Epoch 41/50
63/63 [==============================] - 1s 16ms/step - loss: 0.0383 - accuracy: 0.9973 - val_loss: 0.8938 - val_accuracy: 0.7246Epoch 00041: val_accuracy did not improve from 0.81038
Epoch 42/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0356 - accuracy: 0.9992 - val_loss: 0.8997 - val_accuracy: 0.7226Epoch 00042: val_accuracy did not improve from 0.81038
Epoch 43/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0333 - accuracy: 0.9984 - val_loss: 0.9334 - val_accuracy: 0.7246Epoch 00043: val_accuracy did not improve from 0.81038
Epoch 44/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0281 - accuracy: 0.9987 - val_loss: 0.9452 - val_accuracy: 0.7046Epoch 00044: val_accuracy did not improve from 0.81038
Epoch 45/50
63/63 [==============================] - 1s 16ms/step - loss: 0.0330 - accuracy: 0.9990 - val_loss: 0.9412 - val_accuracy: 0.7126Epoch 00045: val_accuracy did not improve from 0.81038
Epoch 46/50
63/63 [==============================] - 1s 16ms/step - loss: 0.0311 - accuracy: 0.9978 - val_loss: 0.9477 - val_accuracy: 0.7265Epoch 00046: val_accuracy did not improve from 0.81038
Epoch 47/50
63/63 [==============================] - 1s 16ms/step - loss: 0.0279 - accuracy: 0.9990 - val_loss: 0.9422 - val_accuracy: 0.7246Epoch 00047: val_accuracy did not improve from 0.81038
Epoch 48/50
63/63 [==============================] - 1s 16ms/step - loss: 0.0253 - accuracy: 0.9996 - val_loss: 0.9854 - val_accuracy: 0.7226Epoch 00048: val_accuracy did not improve from 0.81038
Epoch 49/50
63/63 [==============================] - 1s 16ms/step - loss: 0.0247 - accuracy: 0.9996 - val_loss: 1.0056 - val_accuracy: 0.7325Epoch 00049: val_accuracy did not improve from 0.81038
Epoch 50/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0234 - accuracy: 1.0000 - val_loss: 0.9983 - val_accuracy: 0.7126Epoch 00050: val_accuracy did not improve from 0.81038
KNRM运行时间: 0.92 mins
INFO:tensorflow:Assets written to: H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\KNRM\save\KNRM\assets
保存成功 保存地址在 H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\KNRM\save

plot_metric(history,"loss")
plot_metric(history,"accuracy")

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-eALvP4MF-1623898674927)(output_72_0.svg)]

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-n7qnQz2o-1623898674927)(output_72_1.svg)]

# 以表格形式展示
dfhistory = pd.DataFrame(history.history)
dfhistory.index = range(1,len(dfhistory) + 1)
dfhistory.index.name = 'epoch'
dfhistory.to_csv(os.path.join(path,'save',model.name,model.name+'.csv'))dfhistory

	loss	accuracy	val_loss	val_accuracy	lr
epoch
1	0.574884	0.7795	0.490436	0.794411	0.001
2	0.440398	0.7985	0.492095	0.808383	0.001
3	0.396315	0.8200	0.502608	0.796407	0.001
4	0.355102	0.8350	0.503900	0.784431	0.001
5	0.331233	0.8550	0.530610	0.810379	0.001
6	0.308696	0.8630	0.525264	0.776447	0.001
7	0.288832	0.8795	0.533331	0.794411	0.001
8	0.273837	0.8805	0.527850	0.786427	0.001
9	0.246026	0.9020	0.541978	0.786427	0.001
10	0.227715	0.9100	0.558648	0.780439	0.001
11	0.212224	0.9275	0.550749	0.776447	0.001
12	0.196974	0.9305	0.572260	0.746507	0.001
13	0.185532	0.9415	0.579861	0.756487	0.001
14	0.172824	0.9470	0.587466	0.746507	0.001
15	0.161000	0.9560	0.593587	0.754491	0.001
16	0.149717	0.9580	0.619803	0.750499	0.001
17	0.142122	0.9625	0.626306	0.756487	0.001
18	0.134651	0.9645	0.642345	0.758483	0.001
19	0.127208	0.9655	0.653023	0.744511	0.001
20	0.119768	0.9750	0.662865	0.750499	0.001
21	0.113280	0.9770	0.661168	0.752495	0.001
22	0.105297	0.9790	0.674814	0.750499	0.001
23	0.097532	0.9815	0.686752	0.752495	0.001
24	0.093924	0.9875	0.711993	0.732535	0.001
25	0.088030	0.9855	0.725046	0.714571	0.001
26	0.081685	0.9895	0.732556	0.746507	0.001
27	0.077664	0.9895	0.735740	0.746507	0.001
28	0.073690	0.9895	0.782233	0.704591	0.001
29	0.074140	0.9855	0.789516	0.730539	0.001
30	0.068505	0.9890	0.785845	0.726547	0.001
31	0.063642	0.9920	0.792976	0.734531	0.001
32	0.058905	0.9940	0.808054	0.748503	0.001
33	0.057791	0.9930	0.798978	0.752495	0.001
34	0.054806	0.9935	0.813470	0.738523	0.001
35	0.052863	0.9940	0.838032	0.740519	0.001
36	0.052058	0.9940	0.819171	0.748503	0.001
37	0.046994	0.9945	0.855772	0.730539	0.001
38	0.044296	0.9965	0.862320	0.718563	0.001
39	0.042409	0.9960	0.866716	0.734531	0.001
40	0.039556	0.9970	0.889020	0.702595	0.001
41	0.039695	0.9965	0.893792	0.724551	0.001
42	0.038656	0.9980	0.899679	0.722555	0.001
43	0.035125	0.9980	0.933449	0.724551	0.001
44	0.033467	0.9975	0.945224	0.704591	0.001
45	0.034590	0.9980	0.941157	0.712575	0.001
46	0.032396	0.9980	0.947692	0.726547	0.001
47	0.031140	0.9990	0.942183	0.724551	0.001
48	0.027756	0.9995	0.985372	0.722555	0.001
49	0.027366	0.9995	1.005574	0.732535	0.001
50	0.025218	0.9990	0.998327	0.712575	0.001

DUET

input1 = Input(name='text_left', shape=(max_len,))
input2 = Input(name='text_right', shape=(max_len,))
embedding = Embedding(vocab_size, embedding_size)
sent1_embed=embedding(input1)
sent2_embed=embedding(input2)
# 这是啥子
lm_xor = keras.layers.Lambda(_xor_match)([sent1_embed, sent2_embed])
lm_conv = keras.layers.Conv1D(32,30,padding='same',activation='relu')(lm_xor)
lm_conv = keras.layers.Dropout(0.5)(lm_conv)
lm_feat = keras.layers.Reshape((-1,))(lm_conv)
for hidden_size in [32]:lm_feat = keras.layers.Dense(hidden_size,activation='relu')(lm_feat)
lm_drop = keras.layers.Dropout(0.5)(lm_feat)
lm_score = keras.layers.Dense(1)(lm_drop)
dm_q_conv = keras.layers.Conv1D(32,3,padding='same',activation='relu')(sent1_embed)
dm_q_conv = keras.layers.Dropout(0.5)(dm_q_conv)
dm_q_mp = keras.layers.MaxPooling1D(pool_size=3)(dm_q_conv)
dm_q_rep = keras.layers.Reshape((-1,))(dm_q_mp)
dm_q_rep = keras.layers.Dense(32)(dm_q_rep)
dm_q_rep = keras.layers.Lambda(lambda x: tf.expand_dims(x, 1))(dm_q_rep)
dm_d_conv1 = keras.layers.Conv1D(32,3,padding='same',activation='relu')(sent2_embed)
dm_d_conv1 = keras.layers.Dropout(0.5)(dm_d_conv1)
dm_d_mp = keras.layers.MaxPooling1D(pool_size=3)(dm_d_conv1)
dm_d_conv2 = keras.layers.Conv1D(32, 3,padding='same',activation='relu')(dm_d_mp)
dm_d_conv2 = keras.layers.Dropout(0.5)(dm_d_conv2)
h_dot = keras.layers.Lambda(_hadamard_dot)([dm_q_rep, dm_d_conv2])
dm_feat = keras.layers.Reshape((-1,))(h_dot)
for hidden_size in [64, 32]:dm_feat = keras.layers.Dense(hidden_size)(dm_feat)
dm_feat_drop = keras.layers.Dropout(0.5)(dm_feat)
dm_score = keras.layers.Dense(1)(dm_feat_drop)
add = keras.layers.Add()([lm_score, dm_score])
outputs=Dense(1,activation='sigmoid',name="output")(add)
model=Model(inputs=[input1,input2],outputs=outputs,name="DUET")
model.summary()

Model: "DUET"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to
==================================================================================================
text_left (InputLayer)          [(None, 15)]         0
__________________________________________________________________________________________________
embedding_4 (Embedding)         (None, 15, 128)      278400      text_left[0][0]                  text_right[0][0]
__________________________________________________________________________________________________
text_right (InputLayer)         [(None, 15)]         0
__________________________________________________________________________________________________
conv1d_11 (Conv1D)              (None, 15, 32)       12320       embedding_4[0][0]
__________________________________________________________________________________________________
dropout_5 (Dropout)             (None, 15, 32)       0           conv1d_11[0][0]
__________________________________________________________________________________________________
conv1d_12 (Conv1D)              (None, 15, 32)       12320       embedding_4[1][0]
__________________________________________________________________________________________________
max_pooling1d_2 (MaxPooling1D)  (None, 5, 32)        0           dropout_5[0][0]
__________________________________________________________________________________________________
dropout_6 (Dropout)             (None, 15, 32)       0           conv1d_12[0][0]
__________________________________________________________________________________________________
reshape_1 (Reshape)             (None, 160)          0           max_pooling1d_2[0][0]
__________________________________________________________________________________________________
max_pooling1d_3 (MaxPooling1D)  (None, 5, 32)        0           dropout_6[0][0]
__________________________________________________________________________________________________
dense_25 (Dense)                (None, 32)           5152        reshape_1[0][0]
__________________________________________________________________________________________________
conv1d_13 (Conv1D)              (None, 5, 32)        3104        max_pooling1d_3[0][0]
__________________________________________________________________________________________________
lambda_23 (Lambda)              (None, 15, 15, 128)  0           embedding_4[0][0]                embedding_4[1][0]
__________________________________________________________________________________________________
lambda_24 (Lambda)              (None, 1, 32)        0           dense_25[0][0]
__________________________________________________________________________________________________
dropout_7 (Dropout)             (None, 5, 32)        0           conv1d_13[0][0]
__________________________________________________________________________________________________
conv1d_10 (Conv1D)              (None, 15, 15, 32)   122912      lambda_23[0][0]
__________________________________________________________________________________________________
lambda_25 (Lambda)              (None, 5, 32)        0           lambda_24[0][0]                  dropout_7[0][0]
__________________________________________________________________________________________________
dropout_3 (Dropout)             (None, 15, 15, 32)   0           conv1d_10[0][0]
__________________________________________________________________________________________________
reshape_2 (Reshape)             (None, 160)          0           lambda_25[0][0]
__________________________________________________________________________________________________
reshape (Reshape)               (None, 7200)         0           dropout_3[0][0]
__________________________________________________________________________________________________
dense_26 (Dense)                (None, 64)           10304       reshape_2[0][0]
__________________________________________________________________________________________________
dense_23 (Dense)                (None, 32)           230432      reshape[0][0]
__________________________________________________________________________________________________
dense_27 (Dense)                (None, 32)           2080        dense_26[0][0]
__________________________________________________________________________________________________
dropout_4 (Dropout)             (None, 32)           0           dense_23[0][0]
__________________________________________________________________________________________________
dropout_8 (Dropout)             (None, 32)           0           dense_27[0][0]
__________________________________________________________________________________________________
dense_24 (Dense)                (None, 1)            33          dropout_4[0][0]
__________________________________________________________________________________________________
dense_28 (Dense)                (None, 1)            33          dropout_8[0][0]
__________________________________________________________________________________________________
add (Add)                       (None, 1)            0           dense_24[0][0]                   dense_28[0][0]
__________________________________________________________________________________________________
output (Dense)                  (None, 1)            2           add[0][0]
==================================================================================================
Total params: 677,092
Trainable params: 677,092
Non-trainable params: 0
__________________________________________________________________________________________________

model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])# 各种callback函数
tb_callback = tf.keras.callbacks.TensorBoard(logdir, histogram_freq=1)
#如果accuracy在25个epoch后没有提升，学习率减半。
lr_callback = tf.keras.callbacks.ReduceLROnPlateau(monitor="accuracy",factor = 0.5, patience = 25)
#当accuracy在30个epoch后没有提升，则提前终止训练。
stop_callback = tf.keras.callbacks.EarlyStopping(monitor = "accuracy", patience= 30)#获取当前的地址
path=os.path.join(os.getcwd(),'outputs\model',model.name)import os
dirs = pathif not os.path.exists(dirs):os.makedirs(dirs)# 保留最好的模型
mc_callback=tf.keras.callbacks.ModelCheckpoint(filepath=os.path.join(path,'weights.hdf5'),monitor='val_accuracy',verbose=1, save_best_only=True)# 把训练轮结果数据流到 csv 文件的回调函数。
# url=os.path.join(os.getcwd(),'outputs')csv_callback=tf.keras.callbacks.CSVLogger(os.path.join(path,'training.log'))callbacks_list = [tb_callback,lr_callback,stop_callback,mc_callback,csv_callback]t = time()
history=model.fit(x=[sent1_train,sent2_train],y=train_labels,validation_data=([sent1_val,sent2_val],val_labels),epochs=50,\callbacks = callbacks_list)
print(model.name+'运行时间: {} mins'.format(round((time() - t) / 60, 2)))model.save(os.path.join(path,'save',model.name))
print('保存成功','保存地址在',os.path.join(path,'save'))

Epoch 1/50
63/63 [==============================] - 3s 26ms/step - loss: 0.5843 - accuracy: 0.7687 - val_loss: 0.4710 - val_accuracy: 0.8144Epoch 00001: val_accuracy improved from -inf to 0.81437, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\DUET\weights.hdf5
Epoch 2/50
63/63 [==============================] - 1s 14ms/step - loss: 0.4494 - accuracy: 0.8254 - val_loss: 0.4809 - val_accuracy: 0.8144Epoch 00002: val_accuracy did not improve from 0.81437
Epoch 3/50
63/63 [==============================] - 1s 14ms/step - loss: 0.4539 - accuracy: 0.8000 - val_loss: 0.4764 - val_accuracy: 0.8124Epoch 00003: val_accuracy did not improve from 0.81437
Epoch 4/50
63/63 [==============================] - 1s 15ms/step - loss: 0.3712 - accuracy: 0.8293 - val_loss: 0.4861 - val_accuracy: 0.8024Epoch 00004: val_accuracy did not improve from 0.81437
Epoch 5/50
63/63 [==============================] - 1s 15ms/step - loss: 0.3177 - accuracy: 0.8645 - val_loss: 0.5183 - val_accuracy: 0.7884Epoch 00005: val_accuracy did not improve from 0.81437
Epoch 6/50
63/63 [==============================] - 1s 14ms/step - loss: 0.2606 - accuracy: 0.8889 - val_loss: 0.5392 - val_accuracy: 0.7705Epoch 00006: val_accuracy did not improve from 0.81437
Epoch 7/50
63/63 [==============================] - 1s 14ms/step - loss: 0.2108 - accuracy: 0.9187 - val_loss: 0.6511 - val_accuracy: 0.7764Epoch 00007: val_accuracy did not improve from 0.81437
Epoch 8/50
63/63 [==============================] - 1s 15ms/step - loss: 0.1904 - accuracy: 0.9296 - val_loss: 0.6336 - val_accuracy: 0.7804Epoch 00008: val_accuracy did not improve from 0.81437
Epoch 9/50
63/63 [==============================] - 1s 14ms/step - loss: 0.1649 - accuracy: 0.9308 - val_loss: 0.6700 - val_accuracy: 0.7705Epoch 00009: val_accuracy did not improve from 0.81437
Epoch 10/50
63/63 [==============================] - 1s 15ms/step - loss: 0.1114 - accuracy: 0.9563 - val_loss: 0.7332 - val_accuracy: 0.7325Epoch 00010: val_accuracy did not improve from 0.81437
Epoch 11/50
63/63 [==============================] - 1s 14ms/step - loss: 0.1278 - accuracy: 0.9540 - val_loss: 0.8749 - val_accuracy: 0.7325Epoch 00011: val_accuracy did not improve from 0.81437
Epoch 12/50
63/63 [==============================] - 1s 14ms/step - loss: 0.1001 - accuracy: 0.9603 - val_loss: 0.8767 - val_accuracy: 0.7745Epoch 00012: val_accuracy did not improve from 0.81437
Epoch 13/50
63/63 [==============================] - 1s 15ms/step - loss: 0.1035 - accuracy: 0.9632 - val_loss: 0.8983 - val_accuracy: 0.7226Epoch 00013: val_accuracy did not improve from 0.81437
Epoch 14/50
63/63 [==============================] - 1s 14ms/step - loss: 0.1125 - accuracy: 0.9580 - val_loss: 0.9088 - val_accuracy: 0.7265Epoch 00014: val_accuracy did not improve from 0.81437
Epoch 15/50
63/63 [==============================] - 1s 14ms/step - loss: 0.0827 - accuracy: 0.9699 - val_loss: 1.0249 - val_accuracy: 0.7625Epoch 00015: val_accuracy did not improve from 0.81437
Epoch 16/50
63/63 [==============================] - 1s 14ms/step - loss: 0.0908 - accuracy: 0.9618 - val_loss: 0.9792 - val_accuracy: 0.7525Epoch 00016: val_accuracy did not improve from 0.81437
Epoch 17/50
63/63 [==============================] - 1s 14ms/step - loss: 0.0699 - accuracy: 0.9713 - val_loss: 1.0313 - val_accuracy: 0.7325Epoch 00017: val_accuracy did not improve from 0.81437
Epoch 18/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0584 - accuracy: 0.9816 - val_loss: 1.1755 - val_accuracy: 0.7485Epoch 00018: val_accuracy did not improve from 0.81437
Epoch 19/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0634 - accuracy: 0.9765 - val_loss: 1.2195 - val_accuracy: 0.7725Epoch 00019: val_accuracy did not improve from 0.81437
Epoch 20/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0534 - accuracy: 0.9796 - val_loss: 1.2413 - val_accuracy: 0.7465Epoch 00020: val_accuracy did not improve from 0.81437
Epoch 21/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0508 - accuracy: 0.9820 - val_loss: 1.4019 - val_accuracy: 0.7545Epoch 00021: val_accuracy did not improve from 0.81437
Epoch 22/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0599 - accuracy: 0.9810 - val_loss: 1.2181 - val_accuracy: 0.7545Epoch 00022: val_accuracy did not improve from 0.81437
Epoch 23/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0485 - accuracy: 0.9796 - val_loss: 1.3431 - val_accuracy: 0.7824Epoch 00023: val_accuracy did not improve from 0.81437
Epoch 24/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0653 - accuracy: 0.9788 - val_loss: 1.3694 - val_accuracy: 0.7705Epoch 00024: val_accuracy did not improve from 0.81437
Epoch 25/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0529 - accuracy: 0.9836 - val_loss: 1.4829 - val_accuracy: 0.7924Epoch 00025: val_accuracy did not improve from 0.81437
Epoch 26/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0445 - accuracy: 0.9863 - val_loss: 1.4505 - val_accuracy: 0.7545Epoch 00026: val_accuracy did not improve from 0.81437
Epoch 27/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0345 - accuracy: 0.9831 - val_loss: 1.6170 - val_accuracy: 0.7665Epoch 00027: val_accuracy did not improve from 0.81437
Epoch 28/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0374 - accuracy: 0.9862 - val_loss: 1.4732 - val_accuracy: 0.7365Epoch 00028: val_accuracy did not improve from 0.81437
Epoch 29/50
63/63 [==============================] - 1s 16ms/step - loss: 0.0697 - accuracy: 0.9748 - val_loss: 1.3316 - val_accuracy: 0.7705Epoch 00029: val_accuracy did not improve from 0.81437
Epoch 30/50
63/63 [==============================] - 1s 14ms/step - loss: 0.0465 - accuracy: 0.9814 - val_loss: 1.4061 - val_accuracy: 0.7745Epoch 00030: val_accuracy did not improve from 0.81437
Epoch 31/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0464 - accuracy: 0.9851 - val_loss: 1.4792 - val_accuracy: 0.7465Epoch 00031: val_accuracy did not improve from 0.81437
Epoch 32/50
63/63 [==============================] - 1s 14ms/step - loss: 0.0287 - accuracy: 0.9921 - val_loss: 1.7669 - val_accuracy: 0.7784Epoch 00032: val_accuracy did not improve from 0.81437
Epoch 33/50
63/63 [==============================] - 1s 16ms/step - loss: 0.0428 - accuracy: 0.9803 - val_loss: 1.5651 - val_accuracy: 0.7305Epoch 00033: val_accuracy did not improve from 0.81437
Epoch 34/50
63/63 [==============================] - 1s 17ms/step - loss: 0.0310 - accuracy: 0.9892 - val_loss: 1.9694 - val_accuracy: 0.7665Epoch 00034: val_accuracy did not improve from 0.81437
Epoch 35/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0405 - accuracy: 0.9876 - val_loss: 1.5619 - val_accuracy: 0.7685Epoch 00035: val_accuracy did not improve from 0.81437
Epoch 36/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0529 - accuracy: 0.9818 - val_loss: 1.6438 - val_accuracy: 0.7505Epoch 00036: val_accuracy did not improve from 0.81437
Epoch 37/50
63/63 [==============================] - 1s 16ms/step - loss: 0.0437 - accuracy: 0.9833 - val_loss: 1.5540 - val_accuracy: 0.7385Epoch 00037: val_accuracy did not improve from 0.81437
Epoch 38/50
63/63 [==============================] - 1s 17ms/step - loss: 0.0212 - accuracy: 0.9926 - val_loss: 1.7939 - val_accuracy: 0.7545Epoch 00038: val_accuracy did not improve from 0.81437
Epoch 39/50
63/63 [==============================] - 1s 16ms/step - loss: 0.0223 - accuracy: 0.9889 - val_loss: 1.6505 - val_accuracy: 0.7345Epoch 00039: val_accuracy did not improve from 0.81437
Epoch 40/50
63/63 [==============================] - 1s 14ms/step - loss: 0.0251 - accuracy: 0.9928 - val_loss: 1.8213 - val_accuracy: 0.7285Epoch 00040: val_accuracy did not improve from 0.81437
Epoch 41/50
63/63 [==============================] - 1s 14ms/step - loss: 0.0232 - accuracy: 0.9898 - val_loss: 1.9106 - val_accuracy: 0.7665Epoch 00041: val_accuracy did not improve from 0.81437
Epoch 42/50
63/63 [==============================] - 1s 14ms/step - loss: 0.0374 - accuracy: 0.9891 - val_loss: 1.9039 - val_accuracy: 0.7645Epoch 00042: val_accuracy did not improve from 0.81437
Epoch 43/50
63/63 [==============================] - 1s 14ms/step - loss: 0.0453 - accuracy: 0.9881 - val_loss: 1.7437 - val_accuracy: 0.7625Epoch 00043: val_accuracy did not improve from 0.81437
Epoch 44/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0278 - accuracy: 0.9929 - val_loss: 1.5498 - val_accuracy: 0.7585Epoch 00044: val_accuracy did not improve from 0.81437
Epoch 45/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0297 - accuracy: 0.9884 - val_loss: 1.7476 - val_accuracy: 0.7605Epoch 00045: val_accuracy did not improve from 0.81437
Epoch 46/50
63/63 [==============================] - 1s 14ms/step - loss: 0.0243 - accuracy: 0.9943 - val_loss: 1.4905 - val_accuracy: 0.7525Epoch 00046: val_accuracy did not improve from 0.81437
Epoch 47/50
63/63 [==============================] - 1s 14ms/step - loss: 0.0212 - accuracy: 0.9938 - val_loss: 1.5048 - val_accuracy: 0.7585Epoch 00047: val_accuracy did not improve from 0.81437
Epoch 48/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0237 - accuracy: 0.9954 - val_loss: 1.8148 - val_accuracy: 0.7585Epoch 00048: val_accuracy did not improve from 0.81437
Epoch 49/50
63/63 [==============================] - 1s 14ms/step - loss: 0.0101 - accuracy: 0.9975 - val_loss: 1.9479 - val_accuracy: 0.7725Epoch 00049: val_accuracy did not improve from 0.81437
Epoch 50/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0287 - accuracy: 0.9905 - val_loss: 1.7721 - val_accuracy: 0.7784Epoch 00050: val_accuracy did not improve from 0.81437
DUET运行时间: 0.94 mins
INFO:tensorflow:Assets written to: H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\DUET\save\DUET\assets
保存成功 保存地址在 H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\DUET\save

plot_metric(history,"loss")
plot_metric(history,"accuracy")

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-GVTqWmSq-1623898674928)(output_77_0.svg)]

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-k1ggkhZC-1623898674928)(output_77_1.svg)]

# 以表格形式展示
dfhistory = pd.DataFrame(history.history)
dfhistory.index = range(1,len(dfhistory) + 1)
dfhistory.index.name = 'epoch'
dfhistory.to_csv(os.path.join(path,'save',model.name,model.name+'.csv'))dfhistory

	loss	accuracy	val_loss	val_accuracy	lr
epoch
1	0.546751	0.7935	0.471018	0.814371	0.001
2	0.485875	0.8060	0.480949	0.814371	0.001
3	0.441322	0.8140	0.476435	0.812375	0.001
4	0.384277	0.8240	0.486088	0.802395	0.001
5	0.321170	0.8655	0.518297	0.788423	0.001
6	0.273633	0.8810	0.539247	0.770459	0.001
7	0.215189	0.9125	0.651118	0.776447	0.001
8	0.197219	0.9225	0.633586	0.780439	0.001
9	0.185313	0.9240	0.670023	0.770459	0.001
10	0.136315	0.9405	0.733244	0.732535	0.001
11	0.119038	0.9530	0.874861	0.732535	0.001
12	0.115549	0.9535	0.876669	0.774451	0.001
13	0.105906	0.9575	0.898279	0.722555	0.001
14	0.109583	0.9615	0.908792	0.726547	0.001
15	0.087504	0.9700	1.024870	0.762475	0.001
16	0.084737	0.9655	0.979203	0.752495	0.001
17	0.078465	0.9685	1.031322	0.732535	0.001
18	0.063128	0.9795	1.175452	0.748503	0.001
19	0.069982	0.9735	1.219517	0.772455	0.001
20	0.053760	0.9805	1.241341	0.746507	0.001
21	0.049837	0.9835	1.401922	0.754491	0.001
22	0.063830	0.9800	1.218138	0.754491	0.001
23	0.052452	0.9795	1.343117	0.782435	0.001
24	0.054416	0.9790	1.369412	0.770459	0.001
25	0.049495	0.9820	1.482910	0.792415	0.001
26	0.046076	0.9860	1.450504	0.754491	0.001
27	0.043246	0.9820	1.617050	0.766467	0.001
28	0.047107	0.9830	1.473243	0.736527	0.001
29	0.059975	0.9795	1.331571	0.770459	0.001
30	0.042262	0.9845	1.406110	0.774451	0.001
31	0.038564	0.9855	1.479194	0.746507	0.001
32	0.033506	0.9900	1.766863	0.778443	0.001
33	0.041570	0.9815	1.565071	0.730539	0.001
34	0.033610	0.9885	1.969369	0.766467	0.001
35	0.039869	0.9855	1.561921	0.768463	0.001
36	0.039280	0.9865	1.643804	0.750499	0.001
37	0.046237	0.9825	1.553974	0.738523	0.001
38	0.024144	0.9920	1.793919	0.754491	0.001
39	0.032593	0.9860	1.650468	0.734531	0.001
40	0.027968	0.9910	1.821276	0.728543	0.001
41	0.026392	0.9910	1.910593	0.766467	0.001
42	0.027297	0.9920	1.903870	0.764471	0.001
43	0.040179	0.9870	1.743671	0.762475	0.001
44	0.031724	0.9910	1.549826	0.758483	0.001
45	0.031551	0.9915	1.747630	0.760479	0.001
46	0.032493	0.9920	1.490503	0.752495	0.001
47	0.027788	0.9920	1.504812	0.758483	0.001
48	0.024576	0.9940	1.814816	0.758483	0.001
49	0.016065	0.9950	1.947903	0.772455	0.001
50	0.026085	0.9915	1.772141	0.778443	0.001

DRMMTKS

# def _make_multi_layer_perceptron_layer() -> keras.layers.Layer:
#         # TODO: do not create new layers for a second call
#         if not True:
#             raise AttributeError(
#                 'Parameter `with_multi_layer_perception` not set.')#         def _wrapper(x):
#             activation = 'relu'
#             for _ in range(3):
#                 x = keras.layers.Dense(128,
#                                        activation=activation)(x)
#             return keras.layers.Dense(64,
#                                       activation=activation)(x)#         return _wrapper

input1 = Input(name='text_left', shape=(max_len,))
input2 = Input(name='text_right', shape=(max_len,))
embedding = Embedding(vocab_size, embedding_size)
sent1_embed=embedding(input1)
sent2_embed=embedding(input2)
atten_mask = tf.not_equal(input1, -1)
atten_mask = tf.cast(atten_mask,K.floatx())
# shape = [B, L, 1]
atten_mask = tf.expand_dims(atten_mask, axis=2)
# shape = [B, L, 1]
attention_probs =attention_layer(sent1_embed, atten_mask)# Matching histogram of top-k
# shape = [B, L, R]
matching_matrix =Dot(axes=[2, 2], normalize=True)([sent1_embed,sent2_embed])
# shape = [B, L, K]
effective_top_k = min(20,5,300)
matching_topk = Lambda(lambda x: tf.nn.top_k(x, k=effective_top_k, sorted=True)[0]
)(matching_matrix)# Process right input.
# shape = [B, L, 1]
dense_output =_make_multi_layer_perceptron_layer()(matching_topk)# shape = [B, 1, 1]
dot_score = Dot(axes=[1, 1])([attention_probs, dense_output])flatten_score = Flatten()(dot_score)
outputs=Dense(1,activation='sigmoid',name="output")(flatten_score)
model=Model(inputs=[input1,input2],outputs=outputs,name="DRMMTKS")
model.summary()

====================
Model: "DRMMTKS"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to
==================================================================================================
text_left (InputLayer)          [(None, 15)]         0
__________________________________________________________________________________________________
text_right (InputLayer)         [(None, 15)]         0
__________________________________________________________________________________________________
embedding_5 (Embedding)         (None, 15, 128)      278400      text_left[0][0]                  text_right[0][0]
__________________________________________________________________________________________________
dot_4 (Dot)                     (None, 15, 15)       0           embedding_5[0][0]                embedding_5[1][0]
__________________________________________________________________________________________________
lambda_26 (Lambda)              (None, 15, 5)        0           dot_4[0][0]
__________________________________________________________________________________________________
dense_30 (Dense)                (None, 15, 128)      768         lambda_26[0][0]
__________________________________________________________________________________________________
dense_29 (Dense)                (None, 15, 1)        128         embedding_5[0][0]
__________________________________________________________________________________________________
dense_31 (Dense)                (None, 15, 128)      16512       dense_30[0][0]
__________________________________________________________________________________________________
attention_mask (Lambda)         (None, 15, 1)        0           dense_29[0][0]
__________________________________________________________________________________________________
dense_32 (Dense)                (None, 15, 128)      16512       dense_31[0][0]
__________________________________________________________________________________________________
attention_probs (Lambda)        (None, 15, 1)        0           attention_mask[0][0]
__________________________________________________________________________________________________
dense_33 (Dense)                (None, 15, 64)       8256        dense_32[0][0]
__________________________________________________________________________________________________
dot_5 (Dot)                     (None, 1, 64)        0           attention_probs[0][0]            dense_33[0][0]
__________________________________________________________________________________________________
flatten_2 (Flatten)             (None, 64)           0           dot_5[0][0]
__________________________________________________________________________________________________
output (Dense)                  (None, 1)            65          flatten_2[0][0]
==================================================================================================
Total params: 320,641
Trainable params: 320,641
Non-trainable params: 0
__________________________________________________________________________________________________

model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])# 各种callback函数
tb_callback = tf.keras.callbacks.TensorBoard(logdir, histogram_freq=1)
#如果accuracy在25个epoch后没有提升，学习率减半。
lr_callback = tf.keras.callbacks.ReduceLROnPlateau(monitor="accuracy",factor = 0.5, patience = 25)
#当accuracy在30个epoch后没有提升，则提前终止训练。
stop_callback = tf.keras.callbacks.EarlyStopping(monitor = "accuracy", patience= 30)#获取当前的地址
path=os.path.join(os.getcwd(),'outputs\model',model.name)import os
dirs = pathif not os.path.exists(dirs):os.makedirs(dirs)# 这里由于是自己定义模型，所以不能保存格式，一下午全耽误在这里了，妈的
# 保留最好的模型
# mc_callback=tf.keras.callbacks.ModelCheckpoint(filepath=os.path.join(path,'weights.hdf5'),
#                             monitor='val_accuracy',
#                             verbose=1, save_best_only=True)# 把训练轮结果数据流到 csv 文件的回调函数。
# url=os.path.join(os.getcwd(),'outputs')csv_callback=tf.keras.callbacks.CSVLogger(os.path.join(path,'training.log'))callbacks_list = [tb_callback,lr_callback,stop_callback,csv_callback]t = time()
history=model.fit(x=[sent1_train,sent2_train],y=train_labels,validation_data=([sent1_val,sent2_val],val_labels),epochs=50,\callbacks = callbacks_list)
print(model.name+'运行时间: {} mins'.format(round((time() - t) / 60, 2)))# 此模型由于包含自定义的模型，所以暂时不知如何保存
# model.save(os.path.join(path,'save',model.name))
model.save_weights(os.path.join(path,'save',model.name))
print('保存成功','保存地址在',os.path.join(path,'save'))

WARNING:tensorflow:Model failed to serialize as JSON. Ignoring... ('Not JSON Serializable:', <KerasTensor: shape=(None, 15, 1) dtype=float32 (created by layer 'tf.expand_dims')>)
Epoch 1/50
63/63 [==============================] - 2s 21ms/step - loss: 5.8493e-04 - accuracy: 0.9998 - val_loss: 4.4134 - val_accuracy: 0.7425
Epoch 2/50
63/63 [==============================] - 1s 12ms/step - loss: 0.0048 - accuracy: 0.9989 - val_loss: 4.4415 - val_accuracy: 0.7066
Epoch 3/50
63/63 [==============================] - 1s 12ms/step - loss: 6.6143e-04 - accuracy: 1.0000 - val_loss: 4.3381 - val_accuracy: 0.7126
Epoch 4/50
63/63 [==============================] - 1s 12ms/step - loss: 7.1693e-04 - accuracy: 0.9996 - val_loss: 4.7279 - val_accuracy: 0.7246
Epoch 5/50
63/63 [==============================] - 1s 12ms/step - loss: 0.0345 - accuracy: 0.9949 - val_loss: 3.7630 - val_accuracy: 0.6667
Epoch 6/50
63/63 [==============================] - 1s 12ms/step - loss: 0.0168 - accuracy: 0.9934 - val_loss: 3.1826 - val_accuracy: 0.6886
Epoch 7/50
63/63 [==============================] - 1s 12ms/step - loss: 0.0016 - accuracy: 0.9999 - val_loss: 3.0072 - val_accuracy: 0.6926
Epoch 8/50
63/63 [==============================] - 1s 12ms/step - loss: 0.0037 - accuracy: 0.9991 - val_loss: 3.1302 - val_accuracy: 0.7166
Epoch 9/50
63/63 [==============================] - 1s 14ms/step - loss: 1.4906e-04 - accuracy: 1.0000 - val_loss: 3.1637 - val_accuracy: 0.7126
Epoch 10/50
63/63 [==============================] - 1s 12ms/step - loss: 1.7446e-04 - accuracy: 1.0000 - val_loss: 3.1810 - val_accuracy: 0.7166
Epoch 11/50
63/63 [==============================] - 1s 12ms/step - loss: 8.6755e-05 - accuracy: 1.0000 - val_loss: 3.2007 - val_accuracy: 0.7146
Epoch 12/50
63/63 [==============================] - 1s 12ms/step - loss: 9.7124e-05 - accuracy: 1.0000 - val_loss: 3.2211 - val_accuracy: 0.7166
Epoch 13/50
63/63 [==============================] - 1s 12ms/step - loss: 6.5392e-05 - accuracy: 1.0000 - val_loss: 3.2383 - val_accuracy: 0.7166
Epoch 14/50
63/63 [==============================] - 1s 12ms/step - loss: 5.3699e-05 - accuracy: 1.0000 - val_loss: 3.2553 - val_accuracy: 0.7206
Epoch 15/50
63/63 [==============================] - 1s 12ms/step - loss: 5.7876e-05 - accuracy: 1.0000 - val_loss: 3.2714 - val_accuracy: 0.7206
Epoch 16/50
63/63 [==============================] - 1s 12ms/step - loss: 4.5222e-05 - accuracy: 1.0000 - val_loss: 3.2849 - val_accuracy: 0.7206
Epoch 17/50
63/63 [==============================] - 1s 12ms/step - loss: 4.4179e-05 - accuracy: 1.0000 - val_loss: 3.3002 - val_accuracy: 0.7206
Epoch 18/50
63/63 [==============================] - 1s 12ms/step - loss: 4.1445e-05 - accuracy: 1.0000 - val_loss: 3.3130 - val_accuracy: 0.7206
Epoch 19/50
63/63 [==============================] - 1s 12ms/step - loss: 4.2989e-05 - accuracy: 1.0000 - val_loss: 3.3269 - val_accuracy: 0.7226
Epoch 20/50
63/63 [==============================] - 1s 12ms/step - loss: 3.6369e-05 - accuracy: 1.0000 - val_loss: 3.3399 - val_accuracy: 0.7226
Epoch 21/50
63/63 [==============================] - 1s 12ms/step - loss: 3.4277e-05 - accuracy: 1.0000 - val_loss: 3.3522 - val_accuracy: 0.7226
Epoch 22/50
63/63 [==============================] - 1s 12ms/step - loss: 2.9918e-05 - accuracy: 1.0000 - val_loss: 3.3644 - val_accuracy: 0.7226
Epoch 23/50
63/63 [==============================] - 1s 12ms/step - loss: 2.2931e-05 - accuracy: 1.0000 - val_loss: 3.3751 - val_accuracy: 0.7226
Epoch 24/50
63/63 [==============================] - 1s 12ms/step - loss: 2.2884e-05 - accuracy: 1.0000 - val_loss: 3.3872 - val_accuracy: 0.7226
Epoch 25/50
63/63 [==============================] - 1s 12ms/step - loss: 3.1830e-05 - accuracy: 1.0000 - val_loss: 3.3990 - val_accuracy: 0.7226
Epoch 26/50
63/63 [==============================] - 1s 12ms/step - loss: 2.1965e-05 - accuracy: 1.0000 - val_loss: 3.4101 - val_accuracy: 0.7226
Epoch 27/50
63/63 [==============================] - 1s 12ms/step - loss: 2.0477e-05 - accuracy: 1.0000 - val_loss: 3.4210 - val_accuracy: 0.7226
Epoch 28/50
63/63 [==============================] - 1s 12ms/step - loss: 1.4543e-05 - accuracy: 1.0000 - val_loss: 3.4321 - val_accuracy: 0.7226
Epoch 29/50
63/63 [==============================] - 1s 12ms/step - loss: 1.8529e-05 - accuracy: 1.0000 - val_loss: 3.4380 - val_accuracy: 0.7246
Epoch 30/50
63/63 [==============================] - 1s 12ms/step - loss: 1.8538e-05 - accuracy: 1.0000 - val_loss: 3.4432 - val_accuracy: 0.7246
Epoch 31/50
63/63 [==============================] - 1s 12ms/step - loss: 1.9024e-05 - accuracy: 1.0000 - val_loss: 3.4492 - val_accuracy: 0.7246
Epoch 32/50
63/63 [==============================] - 1s 12ms/step - loss: 1.5145e-05 - accuracy: 1.0000 - val_loss: 3.4547 - val_accuracy: 0.7246
Epoch 33/50
63/63 [==============================] - 1s 12ms/step - loss: 1.6403e-05 - accuracy: 1.0000 - val_loss: 3.4603 - val_accuracy: 0.7246
DRMMTKS运行时间: 0.5 mins
保存成功 保存地址在 H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\DRMMTKS\save

plot_metric(history,"loss")
plot_metric(history,"accuracy")

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-el6np97w-1623898674928)(output_83_0.svg)]

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-jtD73anI-1623898674929)(output_83_1.svg)]

# 以表格形式展示
dfhistory = pd.DataFrame(history.history)
dfhistory.index = range(1,len(dfhistory) + 1)
dfhistory.index.name = 'epoch'
dfhistory.to_csv(os.path.join(path,'save',model.name,model.name+'.csv'))dfhistory

	loss	accuracy	val_loss	val_accuracy	lr
epoch
1	0.001804	0.9995	4.413449	0.742515	0.0010
2	0.009143	0.9975	4.441485	0.706587	0.0010
3	0.000664	1.0000	4.338068	0.712575	0.0010
4	0.001877	0.9990	4.727859	0.724551	0.0010
5	0.040458	0.9935	3.763010	0.666667	0.0010
6	0.021687	0.9920	3.182616	0.688623	0.0010
7	0.002447	0.9995	3.007156	0.692615	0.0010
8	0.002465	0.9995	3.130153	0.716567	0.0010
9	0.000166	1.0000	3.163674	0.712575	0.0010
10	0.000125	1.0000	3.180999	0.716567	0.0010
11	0.000101	1.0000	3.200660	0.714571	0.0010
12	0.000086	1.0000	3.221150	0.716567	0.0010
13	0.000075	1.0000	3.238322	0.716567	0.0010
14	0.000067	1.0000	3.255303	0.720559	0.0010
15	0.000060	1.0000	3.271362	0.720559	0.0010
16	0.000053	1.0000	3.284854	0.720559	0.0010
17	0.000048	1.0000	3.300204	0.720559	0.0010
18	0.000043	1.0000	3.313007	0.720559	0.0010
19	0.000039	1.0000	3.326913	0.722555	0.0010
20	0.000036	1.0000	3.339939	0.722555	0.0010
21	0.000033	1.0000	3.352233	0.722555	0.0010
22	0.000031	1.0000	3.364448	0.722555	0.0010
23	0.000028	1.0000	3.375112	0.722555	0.0010
24	0.000026	1.0000	3.387231	0.722555	0.0010
25	0.000024	1.0000	3.399038	0.722555	0.0010
26	0.000023	1.0000	3.410101	0.722555	0.0010
27	0.000021	1.0000	3.420998	0.722555	0.0010
28	0.000020	1.0000	3.432114	0.722555	0.0010
29	0.000019	1.0000	3.438015	0.724551	0.0005
30	0.000018	1.0000	3.443197	0.724551	0.0005
31	0.000017	1.0000	3.449170	0.724551	0.0005
32	0.000017	1.0000	3.454719	0.724551	0.0005
33	0.000016	1.0000	3.460270	0.724551	0.0005

MVLSTM

query= Input(name='text_left', shape=(max_len,))
doc  = Input(name='text_right', shape=(max_len,))
embedding = Embedding(vocab_size, embedding_size,mask_zero=True)
embed_query=embedding(query)
embed_doc=embedding(doc)
# Bi-directional LSTM layer
rep_query = keras.layers.Bidirectional(keras.layers.LSTM(
32,return_sequences=True,dropout=0.2
))(embed_query)
rep_doc = keras.layers.Bidirectional(keras.layers.LSTM(
32,return_sequences=True,dropout=0.2
))(embed_doc)
# Top-k matching layer
matching_matrix = keras.layers.Dot(axes=[2, 2], normalize=False)([rep_query, rep_doc])
matching_signals = keras.layers.Reshape((-1,))(matching_matrix)
matching_topk = keras.layers.Lambda(
lambda x: tf.nn.top_k(x, k=10, sorted=True)[0]
)(matching_signals)
# mlp = _make_multi_layer_perceptron_layer()(matching_topk)
mlp =_make_multi_layer_perceptron_layer()(matching_topk)
mlp = keras.layers.Dropout(rate=0.2)(mlp)x_out = Dense(1, activation='linear')(mlp)
# self._backend = keras.Model(inputs=[query, doc], outputs=x_out)
model=Model(inputs=[query, doc],outputs=x_out,name="MVLSTM")
model.summary()

Model: "MVLSTM"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to
==================================================================================================
text_left (InputLayer)          [(None, 15)]         0
__________________________________________________________________________________________________
text_right (InputLayer)         [(None, 15)]         0
__________________________________________________________________________________________________
embedding_6 (Embedding)         (None, 15, 128)      278400      text_left[0][0]                  text_right[0][0]
__________________________________________________________________________________________________
bidirectional (Bidirectional)   (None, 15, 64)       41216       embedding_6[0][0]
__________________________________________________________________________________________________
bidirectional_1 (Bidirectional) (None, 15, 64)       41216       embedding_6[1][0]
__________________________________________________________________________________________________
dot_6 (Dot)                     (None, 15, 15)       0           bidirectional[0][0]              bidirectional_1[0][0]
__________________________________________________________________________________________________
reshape_3 (Reshape)             (None, 225)          0           dot_6[0][0]
__________________________________________________________________________________________________
lambda_27 (Lambda)              (None, 10)           0           reshape_3[0][0]
__________________________________________________________________________________________________
dense_34 (Dense)                (None, 128)          1408        lambda_27[0][0]
__________________________________________________________________________________________________
dense_35 (Dense)                (None, 128)          16512       dense_34[0][0]
__________________________________________________________________________________________________
dense_36 (Dense)                (None, 128)          16512       dense_35[0][0]
__________________________________________________________________________________________________
dense_37 (Dense)                (None, 64)           8256        dense_36[0][0]
__________________________________________________________________________________________________
dropout_9 (Dropout)             (None, 64)           0           dense_37[0][0]
__________________________________________________________________________________________________
dense_38 (Dense)                (None, 1)            65          dropout_9[0][0]
==================================================================================================
Total params: 403,585
Trainable params: 403,585
Non-trainable params: 0
__________________________________________________________________________________________________

model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])# 各种callback函数
tb_callback = tf.keras.callbacks.TensorBoard(logdir, histogram_freq=1)
#如果accuracy在25个epoch后没有提升，学习率减半。
lr_callback = tf.keras.callbacks.ReduceLROnPlateau(monitor="accuracy",factor = 0.5, patience = 25)
#当accuracy在30个epoch后没有提升，则提前终止训练。
stop_callback = tf.keras.callbacks.EarlyStopping(monitor = "accuracy", patience= 30)#获取当前的地址
path=os.path.join(os.getcwd(),'outputs\model',model.name)import os
dirs = pathif not os.path.exists(dirs):os.makedirs(dirs)# 保留最好的模型
mc_callback=tf.keras.callbacks.ModelCheckpoint(filepath=os.path.join(path,'weights.hdf5'),monitor='val_accuracy',verbose=1, save_best_only=True)# 把训练轮结果数据流到 csv 文件的回调函数。
# url=os.path.join(os.getcwd(),'outputs')csv_callback=tf.keras.callbacks.CSVLogger(os.path.join(path,'training.log'))callbacks_list = [tb_callback,lr_callback,stop_callback,mc_callback,csv_callback]t = time()
history=model.fit(x=[sent1_train,sent2_train],y=train_labels,validation_data=([sent1_val,sent2_val],val_labels),epochs=50,\callbacks = callbacks_list)
print(model.name+'运行时间: {} mins'.format(round((time() - t) / 60, 2)))model.save(os.path.join(path,'save',model.name))
print('保存成功','保存地址在',os.path.join(path,'save'))

Epoch 1/50
63/63 [==============================] - 36s 433ms/step - loss: 3.1179 - accuracy: 0.7979 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00001: val_accuracy improved from -inf to 0.81437, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\MVLSTM\weights.hdf5
Epoch 2/50
63/63 [==============================] - 23s 364ms/step - loss: 2.9303 - accuracy: 0.8100 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00002: val_accuracy did not improve from 0.81437
Epoch 3/50
63/63 [==============================] - 23s 371ms/step - loss: 2.9783 - accuracy: 0.8069 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00003: val_accuracy did not improve from 0.81437
Epoch 4/50
63/63 [==============================] - 23s 371ms/step - loss: 3.0006 - accuracy: 0.8055 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00004: val_accuracy did not improve from 0.81437
Epoch 5/50
63/63 [==============================] - 23s 373ms/step - loss: 2.8179 - accuracy: 0.8173 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00005: val_accuracy did not improve from 0.81437
Epoch 6/50
63/63 [==============================] - 23s 372ms/step - loss: 3.0154 - accuracy: 0.8045 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00006: val_accuracy did not improve from 0.81437
Epoch 7/50
63/63 [==============================] - 23s 372ms/step - loss: 3.1732 - accuracy: 0.7943 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00007: val_accuracy did not improve from 0.81437
Epoch 8/50
63/63 [==============================] - 24s 374ms/step - loss: 3.0198 - accuracy: 0.8042 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00008: val_accuracy did not improve from 0.81437
Epoch 9/50
63/63 [==============================] - 23s 369ms/step - loss: 3.0138 - accuracy: 0.8046 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00009: val_accuracy did not improve from 0.81437
Epoch 10/50
63/63 [==============================] - 24s 376ms/step - loss: 2.9997 - accuracy: 0.8055 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00010: val_accuracy did not improve from 0.81437
Epoch 11/50
63/63 [==============================] - 23s 373ms/step - loss: 2.9870 - accuracy: 0.8064 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00011: val_accuracy did not improve from 0.81437
Epoch 12/50
63/63 [==============================] - 23s 368ms/step - loss: 2.9587 - accuracy: 0.8082 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00012: val_accuracy did not improve from 0.81437
Epoch 13/50
63/63 [==============================] - 23s 366ms/step - loss: 2.9547 - accuracy: 0.8084 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00013: val_accuracy did not improve from 0.81437
Epoch 14/50
63/63 [==============================] - 23s 368ms/step - loss: 2.7369 - accuracy: 0.8226 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00014: val_accuracy did not improve from 0.81437
Epoch 15/50
63/63 [==============================] - 23s 363ms/step - loss: 3.0141 - accuracy: 0.8046 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00015: val_accuracy did not improve from 0.81437
Epoch 16/50
63/63 [==============================] - 23s 362ms/step - loss: 3.0549 - accuracy: 0.8019 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00016: val_accuracy did not improve from 0.81437
Epoch 17/50
63/63 [==============================] - 23s 362ms/step - loss: 2.9716 - accuracy: 0.8073 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00017: val_accuracy did not improve from 0.81437
Epoch 18/50
63/63 [==============================] - 23s 363ms/step - loss: 2.9648 - accuracy: 0.8078 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00018: val_accuracy did not improve from 0.81437
Epoch 19/50
63/63 [==============================] - 23s 362ms/step - loss: 2.9977 - accuracy: 0.8057 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00019: val_accuracy did not improve from 0.81437
Epoch 20/50
63/63 [==============================] - 23s 362ms/step - loss: 2.9069 - accuracy: 0.8115 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00020: val_accuracy did not improve from 0.81437
Epoch 21/50
63/63 [==============================] - 23s 363ms/step - loss: 2.7864 - accuracy: 0.8194 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00021: val_accuracy did not improve from 0.81437
Epoch 22/50
63/63 [==============================] - 23s 363ms/step - loss: 3.0828 - accuracy: 0.8001 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00022: val_accuracy did not improve from 0.81437
Epoch 23/50
63/63 [==============================] - 23s 362ms/step - loss: 2.9424 - accuracy: 0.8092 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00023: val_accuracy did not improve from 0.81437
Epoch 24/50
63/63 [==============================] - 23s 364ms/step - loss: 3.1591 - accuracy: 0.7952 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00024: val_accuracy did not improve from 0.81437
Epoch 25/50
63/63 [==============================] - 23s 364ms/step - loss: 3.0106 - accuracy: 0.8048 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00025: val_accuracy did not improve from 0.81437
Epoch 26/50
63/63 [==============================] - 23s 363ms/step - loss: 2.9116 - accuracy: 0.8112 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00026: val_accuracy did not improve from 0.81437
Epoch 27/50
63/63 [==============================] - 23s 362ms/step - loss: 2.9635 - accuracy: 0.8079 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00027: val_accuracy did not improve from 0.81437
Epoch 28/50
63/63 [==============================] - 23s 362ms/step - loss: 3.1087 - accuracy: 0.7985 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00028: val_accuracy did not improve from 0.81437
Epoch 29/50
63/63 [==============================] - 23s 361ms/step - loss: 3.0865 - accuracy: 0.7999 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00029: val_accuracy did not improve from 0.81437
Epoch 30/50
63/63 [==============================] - 23s 362ms/step - loss: 3.0061 - accuracy: 0.8051 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00030: val_accuracy did not improve from 0.81437
Epoch 31/50
63/63 [==============================] - 23s 362ms/step - loss: 2.8798 - accuracy: 0.8133 - val_loss: 2.8633 - val_accuracy: 0.8144Epoch 00031: val_accuracy did not improve from 0.81437
MVLSTM运行时间: 12.2 minsWARNING:absl:Found untraced functions such as lstm_cell_1_layer_call_fn, lstm_cell_1_layer_call_and_return_conditional_losses, lstm_cell_2_layer_call_fn, lstm_cell_2_layer_call_and_return_conditional_losses, lstm_cell_4_layer_call_fn while saving (showing 5 of 20). These functions will not be directly callable after loading.
WARNING:absl:Found untraced functions such as lstm_cell_1_layer_call_fn, lstm_cell_1_layer_call_and_return_conditional_losses, lstm_cell_2_layer_call_fn, lstm_cell_2_layer_call_and_return_conditional_losses, lstm_cell_4_layer_call_fn while saving (showing 5 of 20). These functions will not be directly callable after loading.INFO:tensorflow:Assets written to: H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\MVLSTM\save\MVLSTM\assetsINFO:tensorflow:Assets written to: H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\MVLSTM\save\MVLSTM\assets保存成功 保存地址在 H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\MVLSTM\save

plot_metric(history,"loss")
plot_metric(history,"accuracy")

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-AX7pRTRt-1623898674929)(output_88_0.svg)]

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-XjHSJbUO-1623898674930)(output_88_1.svg)]

# 以表格形式展示
dfhistory = pd.DataFrame(history.history)
dfhistory.index = range(1,len(dfhistory) + 1)
dfhistory.index.name = 'epoch'
dfhistory.to_csv(os.path.join(path,'save',model.name,model.name+'.csv'))dfhistory

	loss	accuracy	val_loss	val_accuracy	lr
epoch
1	2.992440	0.806	2.863314	0.814371	0.0010
2	2.992440	0.806	2.863314	0.814371	0.0010
3	2.992439	0.806	2.863314	0.814371	0.0010
4	2.992440	0.806	2.863314	0.814371	0.0010
5	2.992440	0.806	2.863314	0.814371	0.0010
6	2.992440	0.806	2.863314	0.814371	0.0010
7	2.992440	0.806

花呗问答匹配（part3 各种模型运行）相关推荐

1. 花呗问答匹配（part1 数据预处理）

   当前时间是2021/6/16,matchzoo作为非常强大的文本匹配库,当前未更新到TF2.4以上版本,无法使用本机3090加速,为此我将源码反向推导实现,使用TF2.4形式实现这些模型 " ...

2. 华为云之ModelArts模型页卡死、模型运行一直失败、云服务器ip访问失败【无法访问此页面】【花了太长时间进行响应】

   记录一些让我很崩溃的问题 一.ModelArts模型页卡死 问题描述: 解决方案: 二.ModelArts模型运行失败 问题描述: 解决方案: 三.云服务器ip访问失败[无法访问此页面][花了太长时间 ...

3. 贝壳房产问答匹配比赛-划水之旅

   一个月前,参加了贝壳找房的房产问答匹配比赛,因为和我的方向比较契合都是属于MRC的.第一次参加比赛,因此记录一下. 为什么要参加比赛. 参加比赛为了让自己有实战经验,简历上可以写上一笔.同时还可以获取 ...

4. CCF2020问答匹配比赛：如何只用“bert”夺冠

   CCF2020问答匹配比赛:如何只用"bert"夺冠 团队介绍:看我看啥 成员1:许明明 中国石油大学 成员2:刘猛 南京大学 非常感谢第一名"看我干啥"分享的 ...

5. 【SWAT水文模型】SWAT水文模型建立及应用第五期：SWAT模型运行设置

   SWAT水文模型建立及应用:SWAT模型运行设置 1 SWAT模型运行设置 1.1 水文响应单元(HRU,Hydrologic Response Unit)划分 1.1.1 Land use/Soil ...

6. 用共振频率去理解神经网络-将乙烯模型运行300次的数据

   在<用神经网络做分子模型:乙烯和乙炔的实验数据>文中将乙烯模型运行了60次,得到C=C的位置的输出是0.5051,本文中将乙烯模型运行了300次,测试这个值是否可以重现. 得到数据 平均值 ...

7. 混合动力汽车SIMULINK整车模型，并联P2构型，基于规则的控制策略，模型运行及仿真无误

   混合动力汽车SIMULINK整车模型,并联P2构型,基于规则的控制策略,模型运行及仿真无误id=660613016217&

8. 单相逆变器S函数仿真，控制算法采用 S函数编写，在模型运行时调用S函数。 逆变器输出电压品质非常高

   单相逆变器S函数仿真,控制算法采用 S函数编写,在模型运行时调用S函数. 逆变器输出电压品质非常高! ID:1765668638115213幸福生活

9. Biome-BGCMuso模型运行

   Biome-BGCMuso模型初步学习 文章目录 Biome-BGCMuso模型初步学习 前言 一.模型下载 二.模型运行类似,此处以Biome-BGCMuso为例,以官网提供的站点数据为基础运行 1 ...

10. 【电气专业知识问答】问：断路器运行中发生异常情形时如何判别和处理？

    [电气专业知识问答] 问:断路器运行中发生异常情形时如何判别和处理? 答:有以下情形之一时,应申请立即停电处理. (1)当发现断路器的套管有严重破损和放电现象.绝缘子或拉杆断裂等时,应立即设法停用该断 ...

最新文章

1. Android之内存管理-内存监测-内存优化
2. MDK linker和debug的设置以及在RAM中调试
3. 【数据结构与算法】拓扑排序问题C语言实现
4. matlab求解常微分方程组/传染病模型并绘制SIR曲线
5. 在Python3中将字符串转换为字节的最佳方法
6. docker : unable to prepare context: context must be a directory
7. java动物类_使用java面向对象创建动物类并输出动物信息
8. python 获取当前目录，上级目录，上上级目录，N级目录
9. 安装 composer 并启动 yii2 项目
10. windows常用端口对应表
11. [图形学]拉普拉斯网格变形（Laplace Deformation）原理及复现
12. 开放大学毕业生自我鉴定计算机,开放大学毕业自我鉴定4篇
13. CI/CD到底是什么？看完就能很快理解
14. x在计算机中是哪个按键,电脑键盘x号怎么打出来
15. XXE漏洞以及XXE漏洞如何修复
16. 计算机图形学迷宫论文,三维迷宫的设计与制造
17. (转)如何成为一名优秀的程序员？
18. 帝骑k触屏模拟器_终骑diend模拟器
19. 火狐浏览器This address is restricted.端口问题
20. 清浅时光，岁月静好——我的2016

热门文章

1. Mac pip安装allennlp
2. 【WPA TSP】基于matlab狼群算法求解旅行商问题【含Matlab源码 211期】
3. [RK3288][Android6.0] 调试笔记 --- RT5640播放时的Codec寄存器列表
4. Tensorflow函数学习笔记2---tf.multipy和tf.matmul
5. 解决Ubuntu 显卡驱动升级导致的 显卡驱动和内核版本不匹配的问题
6. [培训-DSP快速入门-1]：DSP概述（基本框架、CPU, GPU, FPGA比较，常见型号）
7. ISO18000-6C 电子标签数据存储空间及数据加密说明
8. r430服务器如何用u盘做系统,DELL R430服务器U盘安装操作系统指南.docx
9. 相机dc内置滤镜 千奇百怪不离其踪 漫谈数码相机内置滤镜
10. eclipse svn回滚之前版本