如何插入8bit量化节点(tensorflow)

tf流图graph基础知识

默认图

创建显式图

创建多个图

调用tf伪量化接口

插入kernel、层间量化节点

tf流图graph基础知识

默认图

import tensorflow as tf
import numpy as npa = tf.constant(123)
print(a.graph)
print(tf.get_default_graph())

输出：

<tensorflow.python.framework.ops.Graph object at 0x7f32df766668>
<tensorflow.python.framework.ops.Graph object at 0x7f32df766668>

当tensorflow库被加载时，即使用户没有显示地创建一个图，他也会自动创建一个图对象，并将其作为默认的额数据流图

创建显式图

import tensorflow as tf
import numpy as npg= tf.Graph() #创建了一个图对象g
with g.as_default(): #返回一个上下文管理器，使得当前图对象称为当前默认图对象a = tf.constant(123)print(a.graph) #a.graph：获得a所在的图print(tf.get_default_graph()) #get_default_graph：获取当前图对象的句柄

输出：

<tensorflow.python.framework.ops.Graph object at 0x7f71a7a2b710>
<tensorflow.python.framework.ops.Graph object at 0x7f71a7a2b710>

创建多个图

import tensorflow as tf
import numpy as npg1 = tf.Graph()
g2 = tf.Graph()with g1.as_default():a = tf.constant(123)print(a.graph)print(tf.get_default_graph())with g2.as_default():b = tf.multiply(2, 3)print(b.graph)print(tf.get_default_graph())

输出：

<tensorflow.python.framework.ops.Graph object at 0x7f570354fcc0>
<tensorflow.python.framework.ops.Graph object at 0x7f570354fcc0>
<tensorflow.python.framework.ops.Graph object at 0x7f5684813860>
<tensorflow.python.framework.ops.Graph object at 0x7f5684813860>

调用tf伪量化接口

class Quantizationint8(object):#初始化量化参数#n为量化bit数8或16，d是实际小数点位数def __init__(self, n, d):d = float(d)self._quant_min = -(2 ** (n - 1) - 1) * (2 ** d)self._quant_max = (2 ** (n - 1) - 1) * (2 ** d)self._num_bits = nself._narrow = True#利用tf接口函数计算伪量化值，并返回def __call__(self, inputs):return array_ops.fake_quant_with_min_max_vars(inputs, self._quant_min, self._quant_max,num_bits=self._num_bits, narrow_range=self._narrow)

tf接口释义

可参考：https://blog.csdn.net/weixin_36670529/article/details/100560469：

插入kernel、层间量化节点

#g为静态图句柄，producer为全精度浮点数，quant_tensor为上面伪量化值节点的输出
def insert_slim_quant_op(g, producer, name, quant_tensor):Tar_op= []for (op_name, op) in g._nodes_by_name.items():if producer in op.inputs._inputs and name in op_name:Tar_op.append(op)assert len(Tar_op) != 0, "do not have the Tar_op of node {}\n".format(producer.name)with variable_scope.variable_scope(name + '/SlimQuant'):graph_editor.reroute_ts([quant_tensor], [producer], can_modify=Tar_op)