Tensorflow模型各部分自定义方式
2024-04-16 04:44:01
文章目录
- 1 自定义损失函数
- 1.1 第一种方法-函数
- 1.2 第二种方法-装饰器
- 1.3 第三种方法-类
- 1.4 总结
- 2 其它自定义函数-正则\激活函数\初始化\Constraint
- 2.1 函数定义-正则\激活函数\初始化\Constraint
- 2.2 类定义方式-正则
- 2.3 类定义方式-激活函数
- 2.4 类定义方式-初始化方式
- 2.5 类定义方式-Constraint
- 3 自定义层
- 4 自定义Metrics
- 5 关于serialize deserialize get register_keras_serializable的使用
- 5.1 serialize
- 5.2 deserialize
- 5.3 get
- 5.4 register_keras_serializable
Tensorflow模型的定义有几种不同的方式，Sequentia序列方式，Functional 函数方式及subclassing子类方式，对于像pytorch和paddle的子类方式,如查要把整个模型完整保存需要做一些自定义部分，本文将把各种记录都做个简单的定义
本文参考了tensorflow 和keras官网以及https://github.com/ageron/handson-ml3 下自定义部分的内容。
配套代码：https://gitee.com/tdddeeel/tf2_custom_ops/tree/main
import numpy as np
import tensorflow as tf
print(tf.__version__)
gpu = tf.config.list_physical_devices('GPU')[-1]
tf.config.experimental.set_memory_growth(gpu, True)
tf.config.set_visible_devices(gpu,'GPU')
2022-07-26 14:09:09.401799: I tensorflow/core/util/util.cc:169] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.2.9.1
1 自定义损失函数
1.1 第一种方法-函数
def huber_fn(y_true,y_pred):error = y_true-y_predis_small_error = tf.abs(error)<1squared_error = 0.5*tf.square(error)linear_error = tf.abs(error)-0.5return tf.where(is_small_error,squared_error,linear_error)
接着预测波斯顿房价
(x_train,y_train),(x_test,y_test) = tf.keras.datasets.boston_housing.load_data()
from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
x_train_scaled=scaler.fit_transform(x_train)
x_test_scaled= scaler.transform(x_test)
tf.random.set_seed(42)
input_shape = x_train.shape[1:]
model = tf.keras.Sequential([tf.keras.layers.Dense(30,activation='relu',kernel_initializer='he_normal',input_shape=input_shape),tf.keras.layers.Dense(1)]
)
model.compile(loss=huber_fn,optimizer='adam',metrics=["mae"])
history=model.fit(x_train_scaled,y_train,epochs=10,validation_data=(x_test_scaled,y_test))
Epoch 1/10
13/13 [==============================] - 0s 14ms/step - loss: 19.8414 - mae: 20.3408 - val_loss: 20.3195 - val_mae: 20.8195
Epoch 2/10
13/13 [==============================] - 0s 6ms/step - loss: 19.4023 - mae: 19.9019 - val_loss: 19.8747 - val_mae: 20.3747
Epoch 3/10
13/13 [==============================] - 0s 5ms/step - loss: 18.9530 - mae: 19.4519 - val_loss: 19.4094 - val_mae: 19.9094
Epoch 4/10
13/13 [==============================] - 0s 5ms/step - loss: 18.4791 - mae: 18.9776 - val_loss: 18.9208 - val_mae: 19.4203
Epoch 5/10
13/13 [==============================] - 0s 5ms/step - loss: 17.9883 - mae: 18.4878 - val_loss: 18.4100 - val_mae: 18.9053
Epoch 6/10
13/13 [==============================] - 0s 6ms/step - loss: 17.4719 - mae: 17.9718 - val_loss: 17.8766 - val_mae: 18.3734
Epoch 7/10
13/13 [==============================] - 0s 5ms/step - loss: 16.9348 - mae: 17.4325 - val_loss: 17.3165 - val_mae: 17.8153
Epoch 8/10
13/13 [==============================] - 0s 5ms/step - loss: 16.3632 - mae: 16.8602 - val_loss: 16.7346 - val_mae: 17.2311
Epoch 9/10
13/13 [==============================] - 0s 5ms/step - loss: 15.7597 - mae: 16.2561 - val_loss: 16.1315 - val_mae: 16.6279
Epoch 10/10
13/13 [==============================] - 0s 5ms/step - loss: 15.1408 - mae: 15.6358 - val_loss: 15.5066 - val_mae: 15.9913
保存模型
model.save('my_model_with_a_custom_loss')
INFO:tensorflow:Assets written to: my_model_with_a_custom_loss/assets
! ls my_model_with_a_custom_loss
assets keras_metadata.pb saved_model.pb variables
可以看到模型正常保存,接着进行加载并继续进行训练
newmodel=tf.keras.models.load_model('my_model_with_a_custom_loss')
---------------------------------------------------------------------------ValueError Traceback (most recent call last)/tmp/ipykernel_4719/1683424579.py in <module>
----> 1 newmodel=tf.keras.models.load_model('my_model_with_a_custom_loss')~/anaconda3/envs/tf/lib/python3.7/site-packages/keras/utils/traceback_utils.py in error_handler(*args, **kwargs)65 except Exception as e: # pylint: disable=broad-except66 filtered_tb = _process_traceback_frames(e.__traceback__)
---> 67 raise e.with_traceback(filtered_tb) from None68 finally:69 del filtered_tb~/anaconda3/envs/tf/lib/python3.7/site-packages/keras/utils/generic_utils.py in deserialize_keras_object(identifier, module_objects, custom_objects, printable_module_name)708 if obj is None:709 raise ValueError(
--> 710 f'Unknown {printable_module_name}: {object_name}. Please ensure '711 'this object is passed to the `custom_objects` argument. See '712 'https://www.tensorflow.org/guide/keras/save_and_serialize'ValueError: Unknown loss function: huber_fn. Please ensure this object is passed to the `custom_objects` argument. See https://www.tensorflow.org/guide/keras/save_and_serialize#registering_the_custom_object for details.
报错信息中可以看到Unknow loss,所以要重新加载模型会报错,按照提示进行操作
newmodel=tf.keras.models.load_model('my_model_with_a_custom_loss',custom_objects={'huber_fn':huber_fn})
history1=newmodel.fit(x_train_scaled,y_train,epochs=5,validation_data=(x_test_scaled,y_test))
Epoch 1/5
13/13 [==============================] - 0s 15ms/step - loss: 14.5342 - mae: 15.0307 - val_loss: 14.9579 - val_mae: 15.4550
Epoch 2/5
13/13 [==============================] - 0s 5ms/step - loss: 13.9666 - mae: 14.4617 - val_loss: 14.4416 - val_mae: 14.9335
Epoch 3/5
13/13 [==============================] - 0s 5ms/step - loss: 13.4062 - mae: 13.9023 - val_loss: 13.9684 - val_mae: 14.4664
Epoch 4/5
13/13 [==============================] - 0s 6ms/step - loss: 12.8451 - mae: 13.3421 - val_loss: 13.5037 - val_mae: 13.9950
Epoch 5/5
13/13 [==============================] - 0s 5ms/step - loss: 12.2937 - mae: 12.7887 - val_loss: 13.0708 - val_mae: 13.5672
以上就完成的自定义损失函数模型的训练、保存和加载。可以看来损失值是接着原先的值变化的。接着往下看：
1.2 第二种方法-装饰器
def create_huber(threshold=1.0):def huber_fn(y_true,y_pred):error = y_true-y_predis_small_error = tf.abs(error)<thresholdsquared_error = 0.5*tf.square(error)linear_error = threshold*tf.abs(error)-0.5*threshold**2return tf.where(is_small_error,squared_error,linear_error)return huber_fn
tf.random.set_seed(42)
input_shape = x_train.shape[1:]
model = tf.keras.Sequential([tf.keras.layers.Dense(30,activation='relu',kernel_initializer='he_normal',input_shape=input_shape),tf.keras.layers.Dense(1)]
)
model.compile(loss=create_huber(2.0),optimizer='adam',metrics=["mae"]) #重新compile后，所有状态都会进行重新更新
history2=model.fit(x_train_scaled,y_train,epochs=10,validation_data=(x_test_scaled,y_test))
Epoch 1/10
13/13 [==============================] - 1s 15ms/step - loss: 38.6856 - mae: 20.3408 - val_loss: 39.6391 - val_mae: 20.8196
Epoch 2/10
13/13 [==============================] - 0s 6ms/step - loss: 37.8080 - mae: 19.9019 - val_loss: 38.7495 - val_mae: 20.3747
Epoch 3/10
13/13 [==============================] - 0s 6ms/step - loss: 36.9100 - mae: 19.4519 - val_loss: 37.8214 - val_mae: 19.9096
Epoch 4/10
13/13 [==============================] - 0s 5ms/step - loss: 35.9649 - mae: 18.9778 - val_loss: 36.8493 - val_mae: 19.4203
Epoch 5/10
13/13 [==============================] - 0s 6ms/step - loss: 34.9810 - mae: 18.4875 - val_loss: 35.8320 - val_mae: 18.9041
Epoch 6/10
13/13 [==============================] - 0s 6ms/step - loss: 33.9451 - mae: 17.9707 - val_loss: 34.7692 - val_mae: 18.3724
Epoch 7/10
13/13 [==============================] - 0s 6ms/step - loss: 32.8763 - mae: 17.4321 - val_loss: 33.6509 - val_mae: 17.8151
Epoch 8/10
13/13 [==============================] - 0s 6ms/step - loss: 31.7416 - mae: 16.8602 - val_loss: 32.4894 - val_mae: 17.2308
Epoch 9/10
13/13 [==============================] - 0s 5ms/step - loss: 30.5389 - mae: 16.2564 - val_loss: 31.2985 - val_mae: 16.6272
Epoch 10/10
13/13 [==============================] - 0s 7ms/step - loss: 29.3051 - mae: 15.6356 - val_loss: 30.0576 - val_mae: 15.9893
model.save('my_model_with_a_custom_loss_threshold_2')
INFO:tensorflow:Assets written to: my_model_with_a_custom_loss_threshold_2/assets
! ls my_model_with_a_custom_loss_threshold_2
assets keras_metadata.pb saved_model.pb variables
可以看到，同样是保存成功的
newmodel = tf.keras.models.load_model('my_model_with_a_custom_loss_threshold_2/')
---------------------------------------------------------------------------ValueError Traceback (most recent call last)/tmp/ipykernel_4719/3766741098.py in <module>
----> 1 newmodel = tf.keras.models.load_model('my_model_with_a_custom_loss_threshold_2/')~/anaconda3/envs/tf/lib/python3.7/site-packages/keras/utils/traceback_utils.py in error_handler(*args, **kwargs)65 except Exception as e: # pylint: disable=broad-except66 filtered_tb = _process_traceback_frames(e.__traceback__)
---> 67 raise e.with_traceback(filtered_tb) from None68 finally:69 del filtered_tb~/anaconda3/envs/tf/lib/python3.7/site-packages/keras/utils/generic_utils.py in deserialize_keras_object(identifier, module_objects, custom_objects, printable_module_name)708 if obj is None:709 raise ValueError(
--> 710 f'Unknown {printable_module_name}: {object_name}. Please ensure '711 'this object is passed to the `custom_objects` argument. See '712 'https://www.tensorflow.org/guide/keras/save_and_serialize'ValueError: Unknown loss function: huber_fn. Please ensure this object is passed to the `custom_objects` argument. See https://www.tensorflow.org/guide/keras/save_and_serialize#registering_the_custom_object for details.
如果不指定custom objects还是会报错
newmodel = tf.keras.models.load_model('my_model_with_a_custom_loss_threshold_2/',custom_objects={'huber_fn':create_huber(2.0)})
history1=newmodel.fit(x_train_scaled,y_train,epochs=5,validation_data=(x_test_scaled,y_test))
Epoch 1/5
13/13 [==============================] - 1s 15ms/step - loss: 28.0896 - mae: 15.0308 - val_loss: 28.9675 - val_mae: 15.4553
Epoch 2/5
13/13 [==============================] - 0s 6ms/step - loss: 26.9547 - mae: 14.4610 - val_loss: 27.9318 - val_mae: 14.9332
Epoch 3/5
13/13 [==============================] - 0s 5ms/step - loss: 25.8384 - mae: 13.9020 - val_loss: 26.9647 - val_mae: 14.4667
Epoch 4/5
13/13 [==============================] - 0s 5ms/step - loss: 24.7175 - mae: 13.3422 - val_loss: 26.0434 - val_mae: 13.9959
Epoch 5/5
13/13 [==============================] - 0s 5ms/step - loss: 23.6254 - mae: 12.7892 - val_loss: 25.1706 - val_mae: 13.5669
1.3 第三种方法-类
class HuberLoss(tf.keras.losses.Loss):def __init__(self,threshold=1.0,**kwargs):self.threshold=thresholdsuper().__init__(**kwargs)def call(self,y_true,y_pred):error = y_true-y_predis_small_error = tf.abs(error)<self.thresholdsquared_error = 0.5*tf.square(error)linear_error = self.threshold*tf.abs(error)-0.5*self.threshold**2return tf.where(is_small_error,squared_error,linear_error)def get_config(self):base_config = super().get_config()all_config = {**base_config,"threshold":self.threshold}return all_config
#再次定义新的模型
tf.random.set_seed(42)
input_shape = x_train.shape[1:]
model = tf.keras.Sequential([tf.keras.layers.Dense(30,activation='relu',kernel_initializer='he_normal',input_shape=input_shape),tf.keras.layers.Dense(1)]
)model.compile(loss=HuberLoss(2.),optimizer='adam',metrics=["mae"])
history=model.fit(x_train_scaled,y_train,epochs=10,validation_data=(x_test_scaled,y_test))
Epoch 1/10
13/13 [==============================] - 1s 16ms/step - loss: 38.6856 - mae: 20.3408 - val_loss: 39.6391 - val_mae: 20.8196
Epoch 2/10
13/13 [==============================] - 0s 7ms/step - loss: 37.8080 - mae: 19.9019 - val_loss: 38.7495 - val_mae: 20.3747
Epoch 3/10
13/13 [==============================] - 0s 7ms/step - loss: 36.9100 - mae: 19.4519 - val_loss: 37.8214 - val_mae: 19.9096
Epoch 4/10
13/13 [==============================] - 0s 6ms/step - loss: 35.9649 - mae: 18.9778 - val_loss: 36.8493 - val_mae: 19.4203
Epoch 5/10
13/13 [==============================] - 0s 7ms/step - loss: 34.9810 - mae: 18.4875 - val_loss: 35.8320 - val_mae: 18.9041
Epoch 6/10
13/13 [==============================] - 0s 6ms/step - loss: 33.9451 - mae: 17.9707 - val_loss: 34.7692 - val_mae: 18.3724
Epoch 7/10
13/13 [==============================] - 0s 6ms/step - loss: 32.8763 - mae: 17.4321 - val_loss: 33.6509 - val_mae: 17.8151
Epoch 8/10
13/13 [==============================] - 0s 6ms/step - loss: 31.7416 - mae: 16.8602 - val_loss: 32.4894 - val_mae: 17.2308
Epoch 9/10
13/13 [==============================] - 0s 6ms/step - loss: 30.5389 - mae: 16.2564 - val_loss: 31.2985 - val_mae: 16.6272
Epoch 10/10
13/13 [==============================] - 0s 6ms/step - loss: 29.3051 - mae: 15.6356 - val_loss: 30.0576 - val_mae: 15.9893
可以看到，和方法二的结果是一样的，tensorflow2.9中加入tf.random.set_seed()等其它优化，使得结果尽可能可以复现
model.save('my_model_with_a_custom_loss_class')
INFO:tensorflow:Assets written to: my_model_with_a_custom_loss_class/assets
newmodel = tf.keras.models.load_model('my_model_with_a_custom_loss_class/')
---------------------------------------------------------------------------ValueError Traceback (most recent call last)/tmp/ipykernel_4719/829131420.py in <module>
----> 1 newmodel = tf.keras.models.load_model('my_model_with_a_custom_loss_class/')~/anaconda3/envs/tf/lib/python3.7/site-packages/keras/utils/traceback_utils.py in error_handler(*args, **kwargs)65 except Exception as e: # pylint: disable=broad-except66 filtered_tb = _process_traceback_frames(e.__traceback__)
---> 67 raise e.with_traceback(filtered_tb) from None68 finally:69 del filtered_tb~/anaconda3/envs/tf/lib/python3.7/site-packages/keras/utils/generic_utils.py in class_and_config_for_serialized_keras_object(config, module_objects, custom_objects, printable_module_name)561 if cls is None:562 raise ValueError(
--> 563 f'Unknown {printable_module_name}: {class_name}. Please ensure this '564 'object is passed to the `custom_objects` argument. See '565 'https://www.tensorflow.org/guide/keras/save_and_serialize'ValueError: Unknown loss function: HuberLoss. Please ensure this object is passed to the `custom_objects` argument. See https://www.tensorflow.org/guide/keras/save_and_serialize#registering_the_custom_object for details.
可以看到，模型还是无法正常加载
newmodel = tf.keras.models.load_model('my_model_with_a_custom_loss_class/',custom_objects={"HuberLoss":HuberLoss(2.)})
history1=newmodel.fit(x_train_scaled,y_train,epochs=5,validation_data=(x_test_scaled,y_test))Epoch 1/5
13/13 [==============================] - 1s 15ms/step - loss: 28.0885 - mae: 15.0302 - val_loss: 28.9629 - val_mae: 15.4528
Epoch 2/5
13/13 [==============================] - 0s 6ms/step - loss: 26.9527 - mae: 14.4601 - val_loss: 27.9266 - val_mae: 14.9309
Epoch 3/5
13/13 [==============================] - 0s 6ms/step - loss: 25.8358 - mae: 13.9006 - val_loss: 26.9588 - val_mae: 14.4639
Epoch 4/5
13/13 [==============================] - 0s 6ms/step - loss: 24.7142 - mae: 13.3406 - val_loss: 26.0373 - val_mae: 13.9925
Epoch 5/5
13/13 [==============================] - 0s 7ms/step - loss: 23.6216 - mae: 12.7874 - val_loss: 25.1644 - val_mae: 13.5636
1.4 总结
以上三种方法，效果是相同的,但都有一个不足,那就是在恢复模型时必须要有自定义部分的原始代码,这给模型部署带来不便,要是想要部署一个模型还要带上这个自定义函数,这是很不合适的后边会写处理方式.
2 其它自定义函数-正则\激活函数\初始化\Constraint
2.1 函数定义-正则\激活函数\初始化\Constraint
#激活函数
def my_softplus(z):return tf.math.log(1.0+tf.exp(z))
#初始化函数
def my_glorot_initializer(shape,dtype=tf.float32):stddev = tf.sqrt(2./(shape[0]+shape[1]))return tf.random.normal(shape,stddev=stddev,dtype=dtype)
#正则化
def my_l1_regularizer(weights):return tf.reduce_sum(tf.abs(0.01*weights))
#constraint
def my_positive_weights(weights):return tf.where(weights<0.,tf.zeros_like(weights),weights)
layer = tf.keras.layers.Dense(1,activation=my_softplus,kernel_initializer=my_glorot_initializer,kernel_regularizer=my_l1_regularizer,kernel_constraint=my_positive_weights)
layer.get_config()
{'name': 'dense_6','trainable': True,'dtype': 'float32','units': 1,'activation': 'my_softplus','use_bias': True,'kernel_initializer': 'my_glorot_initializer','bias_initializer': {'class_name': 'Zeros', 'config': {}},'kernel_regularizer': 'my_l1_regularizer','bias_regularizer': None,'activity_regularizer': None,'kernel_constraint': 'my_positive_weights','bias_constraint': None}
接着定义模型,训练,保存,加载,再训练
tf.random.set_seed(42)
model = tf.keras.Sequential([tf.keras.layers.Dense(30,activation='relu',kernel_initializer='he_normal'),tf.keras.layers.Dense(1,activation=my_softplus,kernel_initializer=my_glorot_initializer,kernel_regularizer=my_l1_regularizer,kernel_constraint=my_positive_weights)
])
model.compile(loss='mse',optimizer='adam',metrics=["mae"])
history=model.fit(x_train_scaled,y_train,epochs=10,validation_data=(x_test_scaled,y_test))
model.save('my_model_with_a_custom_parts')
Epoch 1/10
13/13 [==============================] - 1s 14ms/step - loss: 515.5219 - mae: 20.7559 - val_loss: 531.4690 - val_mae: 21.1663
Epoch 2/10
13/13 [==============================] - 0s 6ms/step - loss: 501.5470 - mae: 20.4198 - val_loss: 519.0331 - val_mae: 20.8716
Epoch 3/10
13/13 [==============================] - 0s 5ms/step - loss: 488.6118 - mae: 20.1095 - val_loss: 504.7845 - val_mae: 20.5305
Epoch 4/10
13/13 [==============================] - 0s 5ms/step - loss: 473.7936 - mae: 19.7504 - val_loss: 488.5923 - val_mae: 20.1391
Epoch 5/10
13/13 [==============================] - 0s 5ms/step - loss: 456.8321 - mae: 19.3409 - val_loss: 470.3868 - val_mae: 19.6946
Epoch 6/10
13/13 [==============================] - 0s 6ms/step - loss: 437.9981 - mae: 18.8721 - val_loss: 450.0984 - val_mae: 19.1896
Epoch 7/10
13/13 [==============================] - 0s 5ms/step - loss: 417.4472 - mae: 18.3609 - val_loss: 428.3565 - val_mae: 18.6335
Epoch 8/10
13/13 [==============================] - 0s 5ms/step - loss: 395.6647 - mae: 17.7891 - val_loss: 404.8337 - val_mae: 18.0099
Epoch 9/10
13/13 [==============================] - 0s 5ms/step - loss: 372.3109 - mae: 17.1659 - val_loss: 380.1979 - val_mae: 17.3433
Epoch 10/10
13/13 [==============================] - 0s 5ms/step - loss: 347.9553 - mae: 16.4938 - val_loss: 354.7127 - val_mae: 16.6338
INFO:tensorflow:Assets written to: my_model_with_a_custom_parts/assets
model.get_config()
{'name': 'sequential_3','layers': [{'class_name': 'InputLayer','config': {'batch_input_shape': (None, 13),'dtype': 'float32','sparse': False,'ragged': False,'name': 'dense_7_input'}},{'class_name': 'Dense','config': {'name': 'dense_7','trainable': True,'dtype': 'float32','units': 30,'activation': 'relu','use_bias': True,'kernel_initializer': {'class_name': 'HeNormal', 'config': {'seed': None}},'bias_initializer': {'class_name': 'Zeros', 'config': {}},'kernel_regularizer': None,'bias_regularizer': None,'activity_regularizer': None,'kernel_constraint': None,'bias_constraint': None}},{'class_name': 'Dense','config': {'name': 'dense_8','trainable': True,'dtype': 'float32','units': 1,'activation': 'my_softplus','use_bias': True,'kernel_initializer': 'my_glorot_initializer','bias_initializer': {'class_name': 'Zeros', 'config': {}},'kernel_regularizer': 'my_l1_regularizer','bias_regularizer': None,'activity_regularizer': None,'kernel_constraint': 'my_positive_weights','bias_constraint': None}}]}
model.layers[1].get_config()
{'name': 'dense_8','trainable': True,'dtype': 'float32','units': 1,'activation': 'my_softplus','use_bias': True,'kernel_initializer': 'my_glorot_initializer','bias_initializer': {'class_name': 'Zeros', 'config': {}},'kernel_regularizer': 'my_l1_regularizer','bias_regularizer': None,'activity_regularizer': None,'kernel_constraint': 'my_positive_weights','bias_constraint': None}
newmodel = tf.keras.models.load_model('my_model_with_a_custom_parts/')
history1=newmodel.fit(x_train_scaled,y_train,epochs=5,validation_data=(x_test_scaled,y_test))
Epoch 1/5
13/13 [==============================] - 1s 15ms/step - loss: 325.0072 - mae: 15.8323 - val_loss: 333.4890 - val_mae: 16.0263
Epoch 2/5
13/13 [==============================] - 0s 6ms/step - loss: 304.5963 - mae: 15.2173 - val_loss: 312.2325 - val_mae: 15.3937
Epoch 3/5
13/13 [==============================] - 0s 6ms/step - loss: 284.0917 - mae: 14.5946 - val_loss: 291.3275 - val_mae: 14.7601
Epoch 4/5
13/13 [==============================] - 0s 5ms/step - loss: 264.2493 - mae: 13.9393 - val_loss: 271.0325 - val_mae: 14.1239
Epoch 5/5
13/13 [==============================] - 0s 6ms/step - loss: 244.9014 - mae: 13.2921 - val_loss: 251.4772 - val_mae: 13.5366
newmodel.layers[1].get_config()
{'name': 'dense_8','trainable': True,'dtype': 'float32','units': 1,'activation': 'my_softplus','use_bias': True,'kernel_initializer': 'my_glorot_initializer','bias_initializer': {'class_name': 'Zeros','config': {},'shared_object_id': 4},'kernel_regularizer': 'my_l1_regularizer','bias_regularizer': None,'activity_regularizer': None,'kernel_constraint': 'my_positive_weights','bias_constraint': None}
所以要判断模型是否一致,我们可以看一下他们的config
可以看到,对于这些自定义的函数无需人为指定也可以加载
newmodel1 = tf.keras.models.load_model('my_model_with_a_custom_parts/',custom_objects={"my_l1_regularizer":my_l1_regularizer,"my_positive_weights":my_positive_weights,"my_glorot_initializer":my_glorot_initializer,"my_softplus":my_softplus})
history2=newmodel1.fit(x_train_scaled,y_train,epochs=5,validation_data=(x_test_scaled,y_test))
Epoch 1/5
13/13 [==============================] - 1s 17ms/step - loss: 325.0072 - mae: 15.8323 - val_loss: 333.4890 - val_mae: 16.0263
Epoch 2/5
13/13 [==============================] - 0s 6ms/step - loss: 304.5963 - mae: 15.2173 - val_loss: 312.2325 - val_mae: 15.3937
Epoch 3/5
13/13 [==============================] - 0s 7ms/step - loss: 284.0917 - mae: 14.5946 - val_loss: 291.3275 - val_mae: 14.7601
Epoch 4/5
13/13 [==============================] - 0s 6ms/step - loss: 264.2493 - mae: 13.9393 - val_loss: 271.0325 - val_mae: 14.1239
Epoch 5/5
13/13 [==============================] - 0s 7ms/step - loss: 244.9014 - mae: 13.2921 - val_loss: 251.4772 - val_mae: 13.5366
这次可以看出,人为指定后,结果是一样的
2.2 类定义方式-正则
正则化也可以用类方式定义
class MyL1Regularizer(tf.keras.regularizers.Regularizer):def __init__(self,factor):self.factor=factordef __call__(self,weights):return tf.reduce_mean(tf.math.abs(self.factor*weights))def get_config(self):base_config = super().get_config()all_config={**base_config,'factor':self.factor}
tf.random.set_seed(42)
model = tf.keras.Sequential([tf.keras.layers.Dense(30,activation='relu',kernel_initializer='he_normal'),tf.keras.layers.Dense(1,activation=my_softplus,kernel_initializer=my_glorot_initializer,kernel_regularizer=MyL1Regularizer(0.01),kernel_constraint=my_positive_weights)
])
model.compile(loss='mse',optimizer='adam',metrics=["mae"])
history=model.fit(x_train_scaled,y_train,epochs=10,validation_data=(x_test_scaled,y_test))
model.save('my_model_with_a_custom_parts1')
Epoch 1/10
13/13 [==============================] - 1s 19ms/step - loss: 515.4951 - mae: 20.7559 - val_loss: 531.4422 - val_mae: 21.1663
Epoch 2/10
13/13 [==============================] - 0s 5ms/step - loss: 501.5185 - mae: 20.4198 - val_loss: 519.0024 - val_mae: 20.8716
Epoch 3/10
13/13 [==============================] - 0s 5ms/step - loss: 488.5792 - mae: 20.1095 - val_loss: 504.7497 - val_mae: 20.5305
Epoch 4/10
13/13 [==============================] - 0s 5ms/step - loss: 473.7570 - mae: 19.7503 - val_loss: 488.5533 - val_mae: 20.1391
Epoch 5/10
13/13 [==============================] - 0s 5ms/step - loss: 456.7914 - mae: 19.3408 - val_loss: 470.3438 - val_mae: 19.6946
Epoch 6/10
13/13 [==============================] - 0s 5ms/step - loss: 437.9532 - mae: 18.8721 - val_loss: 450.0512 - val_mae: 19.1895
Epoch 7/10
13/13 [==============================] - 0s 5ms/step - loss: 417.3983 - mae: 18.3609 - val_loss: 428.3052 - val_mae: 18.6335
Epoch 8/10
13/13 [==============================] - 0s 5ms/step - loss: 395.6117 - mae: 17.7890 - val_loss: 404.7784 - val_mae: 18.0099
Epoch 9/10
13/13 [==============================] - 0s 5ms/step - loss: 372.2539 - mae: 17.1659 - val_loss: 380.1387 - val_mae: 17.3433
Epoch 10/10
13/13 [==============================] - 0s 6ms/step - loss: 347.8943 - mae: 16.4938 - val_loss: 354.6496 - val_mae: 16.6338
INFO:tensorflow:Assets written to: my_model_with_a_custom_parts1/assets
newmodel = tf.keras.models.load_model('my_model_with_a_custom_parts1/')
history1=newmodel.fit(x_train_scaled,y_train,epochs=5,validation_data=(x_test_scaled,y_test))
Epoch 1/5
13/13 [==============================] - 1s 16ms/step - loss: 325.8748 - mae: 15.8555 - val_loss: 333.4524 - val_mae: 16.0323
Epoch 2/5
13/13 [==============================] - 0s 6ms/step - loss: 304.9491 - mae: 15.2394 - val_loss: 312.9211 - val_mae: 15.4228
Epoch 3/5
13/13 [==============================] - 0s 5ms/step - loss: 285.0223 - mae: 14.6250 - val_loss: 292.1203 - val_mae: 14.7955
Epoch 4/5
13/13 [==============================] - 0s 6ms/step - loss: 264.9132 - mae: 13.9857 - val_loss: 271.5786 - val_mae: 14.1489
Epoch 5/5
13/13 [==============================] - 0s 5ms/step - loss: 245.6193 - mae: 13.3311 - val_loss: 251.5693 - val_mae: 13.5434
newmodel1 = tf.keras.models.load_model('my_model_with_a_custom_parts1/',custom_objects={"MyL1Regularizer":MyL1Regularizer,"my_positive_weights":my_positive_weights,"my_glorot_initializer":my_glorot_initializer,"my_softplus":my_softplus})
history2=newmodel1.fit(x_train_scaled,y_train,epochs=5,validation_data=(x_test_scaled,y_test))
2022-07-13 10:10:51.405105: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 AVX512F AVX512_VNNI FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-07-13 10:10:52.073048: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 14637 MB memory: -> device: 3, name: Tesla V100-PCIE-16GB-LS, pci bus id: 0000:dd:00.0, compute capability: 7.0---------------------------------------------------------------------------RuntimeError Traceback (most recent call last)/tmp/ipykernel_28741/247645023.py in <module>2 "my_positive_weights":my_positive_weights,3 "my_glorot_initializer":my_glorot_initializer,
----> 4 "my_softplus":my_softplus5 })6 history2=newmodel1.fit(x_train_scaled,y_train,epochs=5,validation_data=(x_test_scaled,y_test))~/anaconda3/envs/tf/lib/python3.7/site-packages/keras/utils/traceback_utils.py in error_handler(*args, **kwargs)65 except Exception as e: # pylint: disable=broad-except66 filtered_tb = _process_traceback_frames(e.__traceback__)
---> 67 raise e.with_traceback(filtered_tb) from None68 finally:69 del filtered_tb~/anaconda3/envs/tf/lib/python3.7/site-packages/keras/saving/saved_model/load.py in _revive_layer_or_model_from_config(self, metadata, node_id)536 'your class with `@keras.utils.register_keras_serializable` and '537 'include that file in your program, or pass your class in a '
--> 538 '`keras.utils.CustomObjectScope` that wraps this load call.') from e539 else:540 raiseRuntimeError: Unable to restore object of class 'Dense' likely due to name conflict with built-in Keras class '<class 'keras.layers.core.dense.Dense'>'. To override the built-in Keras definition of the object, decorate your class with `@keras.utils.register_keras_serializable` and include that file in your program, or pass your class in a `keras.utils.CustomObjectScope` that wraps this load call.
这种情况,不知道为什么会报错,如果不指定反而不会报错,可能是因为第二个Dense已经是一个自定义层了需要做更多处理.
2.3 类定义方式-激活函数
对于激活函数
可以直接使用tf.keras.layers.Activation或tf.keras.layers.Layer来进行构建,因为有些激活函数带有要学习的参数,所以定义激活函数要像定义层一样,请参看下一节
2.4 类定义方式-初始化方式
接着使用类来定义初始化方法
class MyGlortInitializer(tf.keras.initializers.Initializer):def __call__(self,shape,dtype):self.stddev = tf.sqrt(2./(shape[0]+shape[1]))return tf.random.normal(shape,stddev=self.stddev,dtype=dtype) #正常还要写一个get_config
初始化没必要做到能保存，因为模型训练后或不训练，再次加载时用不到，只有第一次才能用到，所以get_config之类的，在initializer中是没有的
tf.random.set_seed(42)
model = tf.keras.Sequential([tf.keras.layers.Dense(30,activation='relu',kernel_initializer='he_normal'),tf.keras.layers.Dense(1,activation=my_softplus,kernel_initializer=MyGlortInitializer,kernel_regularizer=my_l1_regularizer,kernel_constraint=my_positive_weights)
])
model.compile(loss='mse',optimizer='adam',metrics=["mae"])
history=model.fit(x_train_scaled,y_train,epochs=10,validation_data=(x_test_scaled,y_test))
model.save('my_model_with_a_custom_parts2')
Epoch 1/10
13/13 [==============================] - 1s 15ms/step - loss: 515.5219 - mae: 20.7559 - val_loss: 531.4690 - val_mae: 21.1663
Epoch 2/10
13/13 [==============================] - 0s 6ms/step - loss: 501.5470 - mae: 20.4198 - val_loss: 519.0331 - val_mae: 20.8716
Epoch 3/10
13/13 [==============================] - 0s 5ms/step - loss: 488.6118 - mae: 20.1095 - val_loss: 504.7845 - val_mae: 20.5305
Epoch 4/10
13/13 [==============================] - 0s 5ms/step - loss: 473.7936 - mae: 19.7504 - val_loss: 488.5923 - val_mae: 20.1391
Epoch 5/10
13/13 [==============================] - 0s 5ms/step - loss: 456.8321 - mae: 19.3409 - val_loss: 470.3868 - val_mae: 19.6946
Epoch 6/10
13/13 [==============================] - 0s 5ms/step - loss: 437.9981 - mae: 18.8721 - val_loss: 450.0984 - val_mae: 19.1896
Epoch 7/10
13/13 [==============================] - 0s 6ms/step - loss: 417.4472 - mae: 18.3609 - val_loss: 428.3565 - val_mae: 18.6335
Epoch 8/10
13/13 [==============================] - 0s 5ms/step - loss: 395.6647 - mae: 17.7891 - val_loss: 404.8337 - val_mae: 18.0099
Epoch 9/10
13/13 [==============================] - 0s 5ms/step - loss: 372.3109 - mae: 17.1659 - val_loss: 380.1979 - val_mae: 17.3433
Epoch 10/10
13/13 [==============================] - 0s 5ms/step - loss: 347.9553 - mae: 16.4938 - val_loss: 354.7128 - val_mae: 16.6338
INFO:tensorflow:Assets written to: my_model_with_a_custom_parts2/assets
newmodel = tf.keras.models.load_model('my_model_with_a_custom_parts2')
history1=newmodel.fit(x_train_scaled,y_train,epochs=5,validation_data=(x_test_scaled,y_test))
Epoch 1/5
13/13 [==============================] - 1s 14ms/step - loss: 325.0072 - mae: 15.8323 - val_loss: 333.4890 - val_mae: 16.0263
Epoch 2/5
13/13 [==============================] - 0s 6ms/step - loss: 304.5963 - mae: 15.2173 - val_loss: 312.2325 - val_mae: 15.3937
Epoch 3/5
13/13 [==============================] - 0s 5ms/step - loss: 284.0917 - mae: 14.5946 - val_loss: 291.3275 - val_mae: 14.7601
Epoch 4/5
13/13 [==============================] - 0s 5ms/step - loss: 264.2493 - mae: 13.9393 - val_loss: 271.0325 - val_mae: 14.1239
Epoch 5/5
13/13 [==============================] - 0s 6ms/step - loss: 244.9014 - mae: 13.2921 - val_loss: 251.4772 - val_mae: 13.5366
newmodel1 = tf.keras.models.load_model('my_model_with_a_custom_parts2/',custom_objects={"my_l1_regularizer":my_l1_regularizer,"my_positive_weights":my_positive_weights,"MyGlorotInitizlizer":MyGlortInitializer,"my_softplus":my_softplus})
history2=newmodel1.fit(x_train_scaled,y_train,epochs=5,validation_data=(x_test_scaled,y_test))
Epoch 1/5
13/13 [==============================] - 1s 15ms/step - loss: 325.0072 - mae: 15.8323 - val_loss: 333.4890 - val_mae: 16.0263
Epoch 2/5
13/13 [==============================] - 0s 5ms/step - loss: 304.5963 - mae: 15.2173 - val_loss: 312.2325 - val_mae: 15.3937
Epoch 3/5
13/13 [==============================] - 0s 5ms/step - loss: 284.0917 - mae: 14.5946 - val_loss: 291.3275 - val_mae: 14.7601
Epoch 4/5
13/13 [==============================] - 0s 6ms/step - loss: 264.2493 - mae: 13.9393 - val_loss: 271.0325 - val_mae: 14.1239
Epoch 5/5
13/13 [==============================] - 0s 5ms/step - loss: 244.9014 - mae: 13.2921 - val_loss: 251.4772 - val_mae: 13.5366
2.5 类定义方式-Constraint
接着对于constraint这里就不写了,官方文档中写到,constraint是一种stateless的,对于这种,无__init__,无需get_config,只要重写__call__就完事
class NonNegative(tf.keras.constraints.Constraint):def __call__(self,w):return w*tf.cast(tf.math.greater_equal(w,0),w.dtype)def get_config(self):base_config=super().get_config()return base_config
3 自定义层
exponential_layer=tf.keras.layers.Lambda(lambda x :tf.math.exp(x))
exponential_layer([-1.,0.,1.])
<tf.Tensor: shape=(3,), dtype=float32, numpy=array([0.36787948, 1. , 2.7182817 ], dtype=float32)>
如果要预测的结果是正,且有很大并异的比例[0.001,0.1,10,1000] ,可以在最后一层加上指数函数
tf.random.set_seed(42)
input_shape = x_train.shape[1:]
model = tf.keras.Sequential([tf.keras.layers.Dense(30,activation='relu',kernel_initializer='he_normal',input_shape=input_shape),tf.keras.layers.Dense(1),exponential_layer]
)
model.compile(loss='mse',optimizer='adam',metrics=["mae"])
history2=model.fit(x_train_scaled,y_train,epochs=10,validation_data=(x_test_scaled,y_test))
Epoch 1/10
13/13 [==============================] - 0s 13ms/step - loss: 5567.9629 - mae: 32.7794 - val_loss: 6352.1050 - val_mae: 32.6988
Epoch 2/10
13/13 [==============================] - 0s 6ms/step - loss: 1320.4006 - mae: 23.1720 - val_loss: 1681.2922 - val_mae: 24.9112
Epoch 3/10
13/13 [==============================] - 0s 5ms/step - loss: 755.1310 - mae: 20.5381 - val_loss: 980.6192 - val_mae: 22.5717
Epoch 4/10
13/13 [==============================] - 0s 5ms/step - loss: 538.5795 - mae: 19.2997 - val_loss: 816.2770 - val_mae: 21.7834
Epoch 5/10
13/13 [==============================] - 0s 5ms/step - loss: 490.4356 - mae: 18.8144 - val_loss: 729.4755 - val_mae: 21.2690
Epoch 6/10
13/13 [==============================] - 0s 5ms/step - loss: 463.7896 - mae: 18.5437 - val_loss: 675.5165 - val_mae: 20.8762
Epoch 7/10
13/13 [==============================] - 0s 5ms/step - loss: 448.9651 - mae: 18.3412 - val_loss: 632.6336 - val_mae: 20.5424
Epoch 8/10
13/13 [==============================] - 0s 5ms/step - loss: 431.8742 - mae: 18.1127 - val_loss: 606.0991 - val_mae: 20.2699
Epoch 9/10
13/13 [==============================] - 0s 5ms/step - loss: 419.8213 - mae: 17.9123 - val_loss: 587.3072 - val_mae: 20.0439
Epoch 10/10
13/13 [==============================] - 0s 5ms/step - loss: 410.5321 - mae: 17.7241 - val_loss: 565.5510 - val_mae: 19.7840
model.save('my_model_with_a_custom_layer1')
INFO:tensorflow:Assets written to: my_model_with_a_custom_layer1/assets
newmodel = tf.keras.models.load_model('my_model_with_a_custom_layer1')
history1=newmodel.fit(x_train_scaled,y_train,epochs=5,validation_data=(x_test_scaled,y_test))
Epoch 1/5
13/13 [==============================] - 0s 14ms/step - loss: 385.0820 - mae: 17.2846 - val_loss: 453.2182 - val_mae: 18.4360
Epoch 2/5
13/13 [==============================] - 0s 5ms/step - loss: 359.4727 - mae: 16.8214 - val_loss: 418.5759 - val_mae: 17.7304
Epoch 3/5
13/13 [==============================] - 0s 5ms/step - loss: 344.2751 - mae: 16.4029 - val_loss: 402.5103 - val_mae: 17.4366
Epoch 4/5
13/13 [==============================] - 0s 6ms/step - loss: 328.9195 - mae: 15.9526 - val_loss: 390.0414 - val_mae: 17.2147
Epoch 5/5
13/13 [==============================] - 0s 5ms/step - loss: 312.5679 - mae: 15.4785 - val_loss: 363.0509 - val_mae: 16.6054
对于更复杂的自定义层
class MyDense(tf.keras.layers.Layer):def __init__(self,units,activation=None,**kwargs):super().__init__(**kwargs)self.units= unitsself.activation=tf.keras.activations.get(activation)def build(self,input_shape):self.kernel = self.add_weight(name='kernel',shape=[input_shape[-1],self.units],initializer='he_normal',trainable=True)self.bias = self.add_weight(name='bias',shape=[self.units],initializer='zeros',trainable=True)# super().build(input_shape)def call(self,x):return self.activation(x@self.kernel+self.bias)def get_config(self):base_config=super().get_config()all_config={**base_config,'units':self.units,'activation':tf.keras.activations.serialize(self.activation)}return all_config
看一下这个激活函数的操作
ac = tf.keras.activations.get('relu')
print(tf.keras.activations.serialize(ac))
relu
tf.random.set_seed(42)
input_shape = x_train.shape[1:]
model = tf.keras.Sequential([MyDense(30,activation='relu'),MyDense(1),]
)
model.compile(loss='mse',optimizer='adam',metrics=["mae"])
history2=model.fit(x_train_scaled,y_train,epochs=10,validation_data=(x_test_scaled,y_test))
Epoch 1/10
13/13 [==============================] - 0s 13ms/step - loss: 616.1592 - mae: 23.1205 - val_loss: 639.0024 - val_mae: 23.6284
Epoch 2/10
13/13 [==============================] - 0s 5ms/step - loss: 598.2690 - mae: 22.7416 - val_loss: 620.9468 - val_mae: 23.2488
Epoch 3/10
13/13 [==============================] - 0s 5ms/step - loss: 580.7922 - mae: 22.3643 - val_loss: 603.4016 - val_mae: 22.8720
Epoch 4/10
13/13 [==============================] - 0s 5ms/step - loss: 563.6730 - mae: 21.9861 - val_loss: 586.1691 - val_mae: 22.4956
Epoch 5/10
13/13 [==============================] - 0s 5ms/step - loss: 546.5667 - mae: 21.6060 - val_loss: 569.0162 - val_mae: 22.1156
Epoch 6/10
13/13 [==============================] - 0s 5ms/step - loss: 529.6218 - mae: 21.2165 - val_loss: 551.5815 - val_mae: 21.7227
Epoch 7/10
13/13 [==============================] - 0s 5ms/step - loss: 512.6191 - mae: 20.8243 - val_loss: 534.1431 - val_mae: 21.3218
Epoch 8/10
13/13 [==============================] - 0s 5ms/step - loss: 495.5235 - mae: 20.4124 - val_loss: 516.2220 - val_mae: 20.8990
Epoch 9/10
13/13 [==============================] - 0s 5ms/step - loss: 477.9213 - mae: 19.9843 - val_loss: 497.9936 - val_mae: 20.4632
Epoch 10/10
13/13 [==============================] - 0s 5ms/step - loss: 459.9037 - mae: 19.5355 - val_loss: 479.3126 - val_mae: 20.0070
model.save('my_model_with_a_custom_layer2')
INFO:tensorflow:Assets written to: my_model_with_a_custom_layer2/assets
newmodel = tf.keras.models.load_model('my_model_with_a_custom_layer2')
history1=newmodel.fit(x_train_scaled,y_train,epochs=5,validation_data=(x_test_scaled,y_test))
Epoch 1/5
13/13 [==============================] - 1s 13ms/step - loss: 441.9317 - mae: 19.0821 - val_loss: 461.3201 - val_mae: 19.5506
Epoch 2/5
13/13 [==============================] - 0s 5ms/step - loss: 424.6645 - mae: 18.6328 - val_loss: 443.4474 - val_mae: 19.0898
Epoch 3/5
13/13 [==============================] - 0s 5ms/step - loss: 407.4524 - mae: 18.1880 - val_loss: 425.7647 - val_mae: 18.6200
Epoch 4/5
13/13 [==============================] - 0s 5ms/step - loss: 390.4624 - mae: 17.7256 - val_loss: 408.2603 - val_mae: 18.1513
Epoch 5/5
13/13 [==============================] - 0s 5ms/step - loss: 373.4209 - mae: 17.2619 - val_loss: 390.7575 - val_mae: 17.6752
接着看一个多输入多输出的层
class MyMultiLayer(tf.keras.layers.Layer):def call(self,x):x1,x2=xprint("x1.shape:",x1.shape,"x2.shape:",x2.shape)return x1+x2,x1*x2,x1/x2
inputs1 = tf.keras.layers.Input(shape=[2])
inputs2 = tf.keras.layers.Input(shape=[2])
MyMultiLayer()((inputs1,inputs2))
x1.shape: (None, 2) x2.shape: (None, 2)(<KerasTensor: shape=(None, 2) dtype=float32 (created by layer 'my_multi_layer')>,<KerasTensor: shape=(None, 2) dtype=float32 (created by layer 'my_multi_layer')>,<KerasTensor: shape=(None, 2) dtype=float32 (created by layer 'my_multi_layer')>)
以上只是输放Placeholder的输入,当然也可以输入实际数值
x1,x2=np.array([[3.,6],[2.,7]]),np.array([[6.,12],[4.,3]])
MyMultiLayer()((x1,x2))
x1.shape: (2, 2) x2.shape: (2, 2)(<tf.Tensor: shape=(2, 2), dtype=float32, numpy=array([[ 9., 18.],[ 6., 10.]], dtype=float32)>,<tf.Tensor: shape=(2, 2), dtype=float32, numpy=array([[18., 72.],[ 8., 21.]], dtype=float32)>,<tf.Tensor: shape=(2, 2), dtype=float32, numpy=array([[0.5 , 0.5 ],[0.5 , 2.3333333]], dtype=float32)>)
接着创建一个在训练和推理过程中表现不一样的层
class MyGaussianNoise(tf.keras.layers.Layer):def __init__(self,stddev,**kwargs):super().__init__(**kwargs)self.stddev = stddevdef call(self,x,training=None):if training:noise = tf.random.normal(tf.shape(x),stddev=self.stddev)return x+noiseelse:return xdef get_config(self):base_config = super().get_config()all_config={**base_config,'stddev':self.stddev}return all_config
tf.random.set_seed(42)
input_shape = x_train.shape[1:]
model = tf.keras.Sequential([MyGaussianNoise(1.0,input_shape=input_shape),tf.keras.layers.Dense(30,activation='relu',kernel_initializer='he_normal'),tf.keras.layers.Dense(1),]
)
model.compile(loss='mse',optimizer='adam',metrics=["mae"])
history=model.fit(x_train_scaled,y_train,epochs=10,validation_data=(x_test_scaled,y_test))
Epoch 1/10
13/13 [==============================] - 1s 14ms/step - loss: 468.3629 - mae: 19.6780 - val_loss: 516.2980 - val_mae: 20.8568
Epoch 2/10
13/13 [==============================] - 0s 6ms/step - loss: 440.2289 - mae: 19.0264 - val_loss: 499.2351 - val_mae: 20.4551
Epoch 3/10
13/13 [==============================] - 0s 5ms/step - loss: 427.3663 - mae: 18.7168 - val_loss: 482.1429 - val_mae: 20.0414
Epoch 4/10
13/13 [==============================] - 0s 6ms/step - loss: 410.8777 - mae: 18.2924 - val_loss: 465.1442 - val_mae: 19.6237
Epoch 5/10
13/13 [==============================] - 0s 6ms/step - loss: 388.5227 - mae: 17.6104 - val_loss: 448.0427 - val_mae: 19.1952
Epoch 6/10
13/13 [==============================] - 0s 6ms/step - loss: 372.6901 - mae: 17.1882 - val_loss: 430.6292 - val_mae: 18.7507
Epoch 7/10
13/13 [==============================] - 0s 5ms/step - loss: 357.3874 - mae: 16.8031 - val_loss: 413.2674 - val_mae: 18.2961
Epoch 8/10
13/13 [==============================] - 0s 5ms/step - loss: 338.7885 - mae: 16.2308 - val_loss: 395.8601 - val_mae: 17.8260
Epoch 9/10
13/13 [==============================] - 0s 6ms/step - loss: 319.7017 - mae: 15.6614 - val_loss: 378.3233 - val_mae: 17.3410
Epoch 10/10
13/13 [==============================] - 0s 6ms/step - loss: 300.2906 - mae: 15.0687 - val_loss: 360.9408 - val_mae: 16.8501
model.save('my_model_with_a_custom_layer3/')
INFO:tensorflow:Assets written to: my_model_with_a_custom_layer3/assets
model.layers[0].get_config()
{'name': 'my_gaussian_noise_4','trainable': True,'batch_input_shape': (None, 13),'dtype': 'float32','stddev': 1.0}
newmodel = tf.keras.models.load_model('my_model_with_a_custom_layer3')
history1=newmodel.fit(x_train_scaled,y_train,epochs=5,validation_data=(x_test_scaled,y_test))
Epoch 1/5
13/13 [==============================] - 0s 14ms/step - loss: 278.7925 - mae: 14.3733 - val_loss: 343.0135 - val_mae: 16.3244
Epoch 2/5
13/13 [==============================] - 0s 6ms/step - loss: 254.7551 - mae: 13.5867 - val_loss: 325.6881 - val_mae: 15.7965
Epoch 3/5
13/13 [==============================] - 0s 5ms/step - loss: 243.1024 - mae: 13.2733 - val_loss: 308.8557 - val_mae: 15.2673
Epoch 4/5
13/13 [==============================] - 0s 5ms/step - loss: 231.9742 - mae: 12.8450 - val_loss: 292.6225 - val_mae: 14.7712
Epoch 5/5
13/13 [==============================] - 0s 5ms/step - loss: 215.9254 - mae: 12.1293 - val_loss: 276.9524 - val_mae: 14.3115
newmodel.layers[0].get_config()
{'name': 'my_gaussian_noise_4','trainable': True,'batch_input_shape': (None, 13),'dtype': 'float32','stddev': 1.0}
可以看出,模型实现原模型加载
4 自定义Metrics
tf.random.set_seed(42)
input_shape = x_train.shape[1:]
model = tf.keras.Sequential([tf.keras.layers.Dense(30,activation='relu',kernel_initializer='he_normal',input_shape=input_shape),tf.keras.layers.Dense(1)]
)
model.compile(loss="mse",optimizer='adam',metrics=[create_huber(2.0)])
history2=model.fit(x_train_scaled,y_train,epochs=3)
2022-07-26 10:29:30.486789: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 AVX512F AVX512_VNNI FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-07-26 10:29:31.131913: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 14637 MB memory: -> device: 3, name: Tesla V100-PCIE-16GB-LS, pci bus id: 0000:dd:00.0, compute capability: 7.0Epoch 1/3
13/13 [==============================] - 3s 4ms/step - loss: 491.9991 - huber_fn: 38.6896
Epoch 2/3
13/13 [==============================] - 0s 4ms/step - loss: 474.2117 - huber_fn: 37.8320
Epoch 3/3
13/13 [==============================] - 0s 3ms/step - loss: 456.2298 - huber_fn: 36.9550
如果损失函数和评价指标使用同一个函数，你可能会看到结果有一些略微的不同.这是因为计算的操作顺序不完全相同，所以可能存以微小的错误。特别是如何使用 sample weights和class weights那么，结果将更加不一样。
- fit()方法会一直跟踪记录所有batch的平均的损失，从训练开始起就这样。每个batch的损失值是由每个样本的损失，如果这个过程有样本或分类加权，会算上去的，求和然后除以样本数得来的。
- 而评价指标是从训练开始，记录所有单个样本的损失，然后然后算平均，再加权算
precision = tf.keras.metrics.Precision()
precision([0,1,1,1,0,1,0,1],[1,1,0,1,0,1,0,1])
<tf.Tensor: shape=(), dtype=float32, numpy=0.8>
precision([0,1,0,0,1,0,1,1],[1,0,1,1,0,0,0,0])
<tf.Tensor: shape=(), dtype=float32, numpy=0.5>
precision.variables
[<tf.Variable 'true_positives:0' shape=(1,) dtype=float32, numpy=array([4.], dtype=float32)>,<tf.Variable 'false_positives:0' shape=(1,) dtype=float32, numpy=array([4.], dtype=float32)>]
precision.reset_states()
接着改一下
class HuberMetric(tf.keras.metrics.Metric):def __init__(self,threshold=1.0,**kwargs):super().__init__(**kwargs)self.threshold = thresholdself.huber_fn = create_huber(threshold)self.total = self.add_weight("total",initializer="zeros")self.count = self.add_weight("count",initializer="zeros")def update_state(self,y_true,y_pred,sample_weight=None):sample_metric = self.huber_fn(y_true,y_pred)self.total.assign_add(tf.reduce_sum(sample_metric))self.count.assign_add(tf.cast(tf.size(y_true),tf.float32))def result(self):return self.total/self.countdef get_config(self):base_config = super().get_config()all_config = {**base_config,'threshold':self.threshold}return all_config
m = HuberMetric(2.0)
# total = 2*|2-10|-2^2/2=14.0
# count = 1
# result = 14.0/1.0=14.0
m(tf.constant([2.0]),tf.constant([10.0]))
<tf.Tensor: shape=(), dtype=float32, numpy=14.0>
# total =total+ (|1.0-0|^2)/2 + (2*|5-9.25|-2^2/2)=14.0+7.0=21.0
# count = count +2=3
# result = total/count=21/3=7
m(tf.constant([[0.],[5]]),tf.constant([[1.],[9.25]]))
<tf.Tensor: shape=(), dtype=float32, numpy=7.0>
m.result()
<tf.Tensor: shape=(), dtype=float32, numpy=7.0>
m.variables
[<tf.Variable 'total:0' shape=() dtype=float32, numpy=21.0>,<tf.Variable 'count:0' shape=() dtype=float32, numpy=3.0>]
m.reset_states()
m.variables
[<tf.Variable 'total:0' shape=() dtype=float32, numpy=0.0>,<tf.Variable 'count:0' shape=() dtype=float32, numpy=0.0>]
检查我们自定义的hubermetric运行良好
tf.random.set_seed(42)
input_shape = x_train.shape[1:]
model = tf.keras.Sequential([tf.keras.layers.Dense(30,activation='relu',kernel_initializer='he_normal',input_shape=input_shape),tf.keras.layers.Dense(1)]
)
model.compile(loss=create_huber(2.0),optimizer='adam',metrics=[HuberMetric(2.0)])
history=model.fit(x_train_scaled,y_train,epochs=3)
Epoch 1/3
13/13 [==============================] - 0s 3ms/step - loss: 38.6856 - huber_metric_6: 38.6856
Epoch 2/3
13/13 [==============================] - 0s 3ms/step - loss: 37.8080 - huber_metric_6: 37.8080
Epoch 3/3
13/13 [==============================] - 0s 2ms/step - loss: 36.9100 - huber_metric_6: 36.9100
model.save("my_model_with_a_custom_metric")
INFO:tensorflow:Assets written to: my_model_with_a_custom_metric/assets
new_model = tf.keras.models.load_model("my_model_with_a_custom_metric/")
history1=new_model.fit(x_train_scaled,y_train,epochs=3)
---------------------------------------------------------------------------ValueError Traceback (most recent call last)/tmp/ipykernel_21219/1845581953.py in <module>
----> 1 new_model = tf.keras.models.load_model("my_model_with_a_custom_metric/")2 history1=newmodel.fit(x_train_scaled,y_train,epochs=3)~/anaconda3/envs/tf/lib/python3.7/site-packages/keras/utils/traceback_utils.py in error_handler(*args, **kwargs)65 except Exception as e: # pylint: disable=broad-except66 filtered_tb = _process_traceback_frames(e.__traceback__)
---> 67 raise e.with_traceback(filtered_tb) from None68 finally:69 del filtered_tb~/anaconda3/envs/tf/lib/python3.7/site-packages/keras/saving/saved_model/load.py in revive_custom_object(identifier, metadata)993 else:994 raise ValueError(
--> 995 f'Unable to restore custom object of type {identifier}. '996 f'Please make sure that any custom layers are included in the '997 f'`custom_objects` arg when calling `load_model()` and make sure that 'ValueError: Unable to restore custom object of type _tf_keras_metric. Please make sure that any custom layers are included in the `custom_objects` arg when calling `load_model()` and make sure that all layers implement `get_config` and `from_config`.
new_model = tf.keras.models.load_model("my_model_with_a_custom_metric/",custom_objects={'huber_fn':create_huber(2.0),'HuberMetric':HuberMetric})
history1=new_model.fit(x_train_scaled,y_train,epochs=3)
Epoch 1/3
13/13 [==============================] - 1s 3ms/step - loss: 35.9728 - huber_metric_6: 35.9728
Epoch 2/3
13/13 [==============================] - 0s 3ms/step - loss: 35.0231 - huber_metric_6: 35.0231
Epoch 3/3
13/13 [==============================] - 0s 3ms/step - loss: 34.0567 - huber_metric_6: 34.0567
model.metrics包括损失和评价指标，所以model.meterics[-1]是我们自定义的hubermetric
model.metrics
[<keras.metrics.base_metric.Mean at 0x7fb95c7a7250>,<__main__.HuberMetric at 0x7fb95c7a2f90>]
tf.keras.metrics.serialize(model.metrics[-1])
{'class_name': 'HuberMetric','config': {'name': 'huber_metric_6', 'dtype': 'float32', 'threshold': 2.0}}
这个类的更简单的定义方式可以是：
class HuberMetric(tf.keras.metrics.Mean):def __init__(self,threshold=1.0,name='HuberMetric',dtype=None):super().__init__(name='HuberMetric',dtype=None)self.threshold = thresholdself.huber_fn = create_huber(threshold)def update_state(self,y_true,y_pred,sample_weight=None):sample_metric = self.huber_fn(y_true,y_pred)super(HuberMetric,self).update_state(sample_metric,sample_weight)def get_config(self):base_config = super().get_config()all_config = {**base_config,'threshold':self.threshold}return all_config
这种定义方式可以更好的处理输入形状和sample weights
tf.random.set_seed(42)
input_shape = x_train.shape[1:]
model = tf.keras.Sequential([tf.keras.layers.Dense(30,activation='relu',kernel_initializer='he_normal',input_shape=input_shape),tf.keras.layers.Dense(1)]
)
model.compile(loss=tf.keras.losses.Huber(2.0),optimizer='adam',weighted_metrics=[HuberMetric(2.0)]) np.random.seed(42)
sample_weight = np.random.rand(len(y_train))
history=model.fit(x_train_scaled,y_train,epochs=3,sample_weight=sample_weight)
Epoch 1/3
13/13 [==============================] - 1s 4ms/step - loss: 19.2427 - HuberMetric: 38.8675
Epoch 2/3
13/13 [==============================] - 0s 3ms/step - loss: 18.8211 - HuberMetric: 38.0161
Epoch 3/3
13/13 [==============================] - 0s 3ms/step - loss: 18.3968 - HuberMetric: 37.1590
(history.history['loss'][0],history.history['HuberMetric'][0]*sample_weight.mean())
(19.242660522460938, 19.242658563073878)
model.save("my_model_with_a_custom_metric2")
INFO:tensorflow:Assets written to: my_model_with_a_custom_metric2/assets
new_model = tf.keras.models.load_model("my_model_with_a_custom_metric2/")
history1=new_model.fit(x_train_scaled,y_train,epochs=3)
---------------------------------------------------------------------------ValueError Traceback (most recent call last)/tmp/ipykernel_20639/4045318839.py in <module>
----> 1 new_model = tf.keras.models.load_model("my_model_with_a_custom_metric2/")2 history1=new_model.fit(x_train_scaled,y_train,epochs=3)~/anaconda3/envs/tf/lib/python3.7/site-packages/keras/utils/traceback_utils.py in error_handler(*args, **kwargs)65 except Exception as e: # pylint: disable=broad-except66 filtered_tb = _process_traceback_frames(e.__traceback__)
---> 67 raise e.with_traceback(filtered_tb) from None68 finally:69 del filtered_tb~/anaconda3/envs/tf/lib/python3.7/site-packages/keras/saving/saved_model/load.py in revive_custom_object(identifier, metadata)993 else:994 raise ValueError(
--> 995 f'Unable to restore custom object of type {identifier}. '996 f'Please make sure that any custom layers are included in the '997 f'`custom_objects` arg when calling `load_model()` and make sure that 'ValueError: Unable to restore custom object of type _tf_keras_metric. Please make sure that any custom layers are included in the `custom_objects` arg when calling `load_model()` and make sure that all layers implement `get_config` and `from_config`.
new_model = tf.keras.models.load_model("my_model_with_a_custom_metric2/",custom_objects={"HuberMetric":HuberMetric})
history1=new_model.fit(x_train_scaled,y_train,epochs=3)
Epoch 1/3
13/13 [==============================] - 0s 3ms/step - loss: 36.0534 - HuberMetric: 36.0534
Epoch 2/3
13/13 [==============================] - 0s 3ms/step - loss: 35.1053 - HuberMetric: 35.1053
Epoch 3/3
13/13 [==============================] - 0s 3ms/step - loss: 34.1408 - HuberMetric: 34.1408
5 关于serialize deserialize get register_keras_serializable的使用
这三个函数的使用还有一个要配合的是tf.keras.utils.register_keras_serializable.
通过一些代码我们来研究一下这三个函数的作用.
其中register_keras_serializable相当重要，可能使用户在使用tf.keras.models.load_model(custom_objects={…}),custom_objects不用由用户额外提供。也看到custom_objects是一个字典，key是我们自定义函数的名称。
5.1 serialize
maxnorm = tf.keras.constraints.MaxNorm(3,0)
output=tf.keras.constraints.serialize(maxnorm)
print(f"type:{type(output)} content:{output}")
print("get_config:",maxnorm.get_config())
type:<class 'dict'> content:{'class_name': 'MaxNorm', 'config': {'max_value': 3, 'axis': 0}}
get_config: {'max_value': 3, 'axis': 0}
可以看到serialize是比get_config反回的内空还要多，其中class_name的内容就是custom_object字典的key,但自定义的层的class_name就是自定义的函数或类的名称，关于类的查看5.4节
def relu_my(input):return tf.nn.relu(input)
output1 = tf.keras.constraints.serialize(relu_my)
output2 = tf.keras.activations.serialize(relu_my)
print(f"output1:{output1} output2:{output2}")
output1:relu_my output2:relu_my
5.2 deserialize
这个是个serialize相反的一对，可以从serialize的结果中反向获得要定义的对像，大致是这么个意思
maxnorm1=tf.keras.constraints.deserialize(output)
print(maxnorm1.get_config())
{'max_value': 3, 'axis': 0}
maxnorm1是和maxnorm有相同参数的一个新的层，也就是说我们可以能过deserialize重新获得这个层的定义
5.3 get
get是为了使用字符串来获得要定义的层，比如maxnorm的serialize的output,可以看到class_name
maxnorm2 = tf.keras.constraints.get('MaxNorm')
print(maxnorm2.get_config())
{'max_value': 2, 'axis': 0}
maxnorm2是一个拥有初始化参数的层，所以get的作用有限
5.4 register_keras_serializable
对比使用和不使用，及使用不同方式时的结果.首先查看不注册的情况下：
class NonNegative(tf.keras.constraints.Constraint):def __call__(self,w):return w*tf.cast(tf.math.greater_equal(w,0),w.dtype)def get_config(self):base_config=super().get_config()return base_config
non = NonNegative()
out = tf.keras.constraints.serialize(non)
print(out)
{'class_name': 'NonNegative', 'config': {}}
print(tf.keras.constraints.get('NonNegative'))
---------------------------------------------------------------------------ValueError Traceback (most recent call last)/tmp/ipykernel_19639/1015723810.py in <module>
----> 1 print(tf.keras.constraints.get('NonNegative'))~/anaconda3/envs/tf/lib/python3.7/site-packages/keras/constraints.py in get(identifier)341 elif isinstance(identifier, str):342 config = {'class_name': str(identifier), 'config': {}}
--> 343 return deserialize(config)344 elif callable(identifier):345 return identifier~/anaconda3/envs/tf/lib/python3.7/site-packages/keras/constraints.py in deserialize(config, custom_objects)329 module_objects=globals(),330 custom_objects=custom_objects,
--> 331 printable_module_name='constraint')332 333 ~/anaconda3/envs/tf/lib/python3.7/site-packages/keras/utils/generic_utils.py in deserialize_keras_object(identifier, module_objects, custom_objects, printable_module_name)663 config = identifier664 (cls, cls_config) = class_and_config_for_serialized_keras_object(
--> 665 config, module_objects, custom_objects, printable_module_name)666 667 # If this object has already been loaded (i.e. it's shared between multiple~/anaconda3/envs/tf/lib/python3.7/site-packages/keras/utils/generic_utils.py in class_and_config_for_serialized_keras_object(config, module_objects, custom_objects, printable_module_name)561 if cls is None:562 raise ValueError(
--> 563 f'Unknown {printable_module_name}: {class_name}. Please ensure this '564 'object is passed to the `custom_objects` argument. See '565 'https://www.tensorflow.org/guide/keras/save_and_serialize'ValueError: Unknown constraint: NonNegative. Please ensure this object is passed to the `custom_objects` argument. See https://www.tensorflow.org/guide/keras/save_and_serialize#registering_the_custom_object for details.
可以看到无法使用get这个方法
print(tf.keras.constraints.deserialize(out))
---------------------------------------------------------------------------ValueError Traceback (most recent call last)/tmp/ipykernel_19639/3405952053.py in <module>
----> 1 print(tf.keras.constraints.deserialize(out))~/anaconda3/envs/tf/lib/python3.7/site-packages/keras/constraints.py in deserialize(config, custom_objects)329 module_objects=globals(),330 custom_objects=custom_objects,
--> 331 printable_module_name='constraint')332 333 ~/anaconda3/envs/tf/lib/python3.7/site-packages/keras/utils/generic_utils.py in deserialize_keras_object(identifier, module_objects, custom_objects, printable_module_name)663 config = identifier664 (cls, cls_config) = class_and_config_for_serialized_keras_object(
--> 665 config, module_objects, custom_objects, printable_module_name)666 667 # If this object has already been loaded (i.e. it's shared between multiple~/anaconda3/envs/tf/lib/python3.7/site-packages/keras/utils/generic_utils.py in class_and_config_for_serialized_keras_object(config, module_objects, custom_objects, printable_module_name)561 if cls is None:562 raise ValueError(
--> 563 f'Unknown {printable_module_name}: {class_name}. Please ensure this '564 'object is passed to the `custom_objects` argument. See '565 'https://www.tensorflow.org/guide/keras/save_and_serialize'ValueError: Unknown constraint: NonNegative. Please ensure this object is passed to the `custom_objects` argument. See https://www.tensorflow.org/guide/keras/save_and_serialize#registering_the_custom_object for details.
也无法使用deserialize。要想使用，需要register_keras_serializable
接着进行使用注册
@tf.keras.utils.register_keras_serializable()
class NonNegative(tf.keras.constraints.Constraint):def __call__(self,w):return w*tf.cast(tf.math.greater_equal(w,0),w.dtype)def get_config(self):base_config=super().get_config()return base_config
non = NonNegative()
out = tf.keras.constraints.serialize(non)
print(out)
{'class_name': 'Custom>NonNegative', 'config': {}}
tf.keras.utils.register_keras_serializable()里边有两个参数，我们接着看,其中package默认参数是Custom
@tf.keras.utils.register_keras_serializable(package="cc",name=None)
class NonNegative(tf.keras.constraints.Constraint):def __call__(self,w):return w*tf.cast(tf.math.greater_equal(w,0),w.dtype)def get_config(self):base_config=super().get_config()return base_config
non = NonNegative()
out = tf.keras.constraints.serialize(non)
print(out)
{'class_name': 'cc>NonNegative', 'config': {}}
@tf.keras.utils.register_keras_serializable(name="ccc")
class NonNegative(tf.keras.constraints.Constraint):def __call__(self,w):return w*tf.cast(tf.math.greater_equal(w,0),w.dtype)def get_config(self):base_config=super().get_config()return base_config
non = NonNegative()
out = tf.keras.constraints.serialize(non)
print(out)
{'class_name': 'Custom>ccc', 'config': {}}
@tf.keras.utils.register_keras_serializable(package='',name="dd")
class NonNegative(tf.keras.constraints.Constraint):def __call__(self,w):return w*tf.cast(tf.math.greater_equal(w,0),w.dtype)def get_config(self):base_config=super().get_config()return base_config
non = NonNegative()
out = tf.keras.constraints.serialize(non)
print(out)
{'class_name': '>dd', 'config': {}}
@tf.keras.utils.register_keras_serializable(package='ok',name="dd")
class NonNegative(tf.keras.constraints.Constraint):def __call__(self,w):return w*tf.cast(tf.math.greater_equal(w,0),w.dtype)def get_config(self):base_config=super().get_config()return base_config
non = NonNegative()
out = tf.keras.constraints.serialize(non)
print(out)
{'class_name': 'ok>dd', 'config': {}}
print(tf.keras.constraints.get('ok>dd'))
print(non)
print(tf.keras.constraints.deserialize(out))
<__main__.NonNegative object at 0x7fcfcf681dd0>
<__main__.NonNegative object at 0x7fd08328d2d0>
<__main__.NonNegative object at 0x7fcfcf6813d0>
总结来说就是package name两个参数分别决定了class_name中’>'这个符号的前边和后边部分，而使用register_keras_serializable可以方便用户在使用自定义的东西时，不需要在加载模型时tf.keras.models.load_model 中传入custom_objects,这对模型部署是极不安全和便利的。以上1到4节部分的内容,都可以加上这个，就不用写custom_objects了
有多种serialize,deserialize,get的组合：tf.keras.layers,tf.keras.initializers,tf.keras.regularizers,tf.keras.losses,tf.keras.constriants等，具体的要上官方文档查看一下
import tensorflow as tf
2022-07-28 20:38:53.365900: I tensorflow/core/util/util.cc:169] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
对于其它普通的类或函数也可以进行序列化
@tf.keras.utils.register_keras_serializable(package="test")
class TestSample:def __call__(self,shape=None,dtype='float32'):print("test")def get_config(self):return{}
test= TestSample()
print(test())
print(tf.keras.utils.serialize_keras_object(test))
test
None
{'class_name': 'test>TestSample', 'config': {}}
@tf.keras.utils.register_keras_serializable(package='test1')
def test(shape=None):print("test1")return None
print(tf.keras.utils.serialize_keras_object(test))
test1>test
Tensorflow模型各部分自定义方式相关推荐
- [TensorFlow深度学习入门]实战七·简便方法实现TensorFlow模型参数保存与加载（ckpt方式）
[TensorFlow深度学习入门]实战七·简便方法实现TensorFlow模型参数保存与加载(ckpt方式) 个人网站–> http://www.yansongsong.cn TensorFl ...
- 打成jar包_keras, tensorflow模型部署通过jar包部署到spark环境攻略
这是个我想干很久的事情了.之前研究tensorflow on spark, DL4j 都没有成功.所以这里首先讲一下我做这件事情的流程.模型的部署,首先你得有一个模型.这里假设你有了一个keras模型 ...
- transformers库的使用【二】tokenizer的使用，模型的保存自定义
使用标记器(tokenizer) 在之前提到过,标记器(tokenizer)是用来对文本进行预处理的一个工具. 首先,标记器会把输入的文档进行分割,将一个句子分成单个的word(或者词语的一部分,或者 ...
- DeepR —训练TensorFlow模型进行生产
Authors Guillaume Genthial, Romain Beaumont, Denis Kuzin, Amine Benhalloum 作家 Guillaume Genthial , R ...
- Tensorflow模型持久化与恢复
Tensorflow模型 简单点说,一个tensorflow模型包含了神经网络的结构(graph)和通过训练得到的一系列神经网络的参数. 神经网络的结构(graph)即神经网络的节点(nodes)及其 ...
- Tensorflow模型优化 端侧部署 tflite 量化 稀疏 quantization sparsity
Tensorflow模型优化 文章目录 Tensorflow模型优化 为什么要模型优化 端侧部署 模型优化方式 Quantization 工具包支持 32 bit float ->16 bit ...
- 用MindStudio进行TensorFlow模型开发流程详解
相对应的视频教学可以在B站进行观看 https://www.bilibili.com/video/BV18a411S7pr 一.MindStudio简介与安装 1.MindStudio简介 Mind ...
- 干货 | tensorflow模型导出与OpenCV DNN中使用
点击上方"小白学视觉",选择加"星标"或"置顶" 重磅干货,第一时间送达 本文转自|OpenCV学堂 OpenCV DNN模块 Deep N ...
- 简单完整地讲解tensorflow模型的保存和恢复
http://blog.csdn.net/liangyihuai/article/details/78515913 在本教程主要讲到: 1. 什么是Tensorflow模型? 2. 如何保存Tenso ...
最新文章
- Dumpzilla工具第615行bug的解决办法
- [HOW TO]-centos环境搭建opengrok笔记
- MACOSX下查看某个端口被哪个程序占用及杀进程方法
- java 状态机_Java 数据持久化系列之池化技术
- php 闭包不能序列化,php如何序列化/存储闭包(Closure)
- Ubuntu安装Yafu方法及资料
- 老师发的周报写法，之前自己太随意没有规则了QAQ
- remote access between two linuxs
- Facebook CTF 2019 Products Manager
- mysql同张表关联多次查询_MySQL多次单表查询和多表联合查询
- 通过计算机名查找当前域用户名,局域网中怎样通过IP查找计算机名
- 全自动苹果CMS火车头采集器，苹果CMS火车头发布插件
- vue+播放直播视频流（websocket的流文件）
- 【Linux】目录权限和默认权限
- 关于H.264 x264 h264 AVC1
- iview table 横向拖动表格内容滚动
- Qt平台编写的五子棋单机版
- 云呐数据备份|什么是离线磁带设备
- Python实现一个论文下载器
- 荣耀20android版本10什么意思,手机 篇一：荣耀10的老用户要不要换荣耀20Pro