文章目录

Tensorflow训练maskrcnn
- 安装cuda cudnn Anaconda
- Anaconda环境配置
- - 换源(清华源，清华源下载更快)
  - 新建虚拟环境
  - 进入虚拟环境
  - 安装tensorflow
  - 安装环境
- 安装TensorFlow Object Detection API
- - 下载TensorFlow模型源码
  - 将object detection的源码导入Anaconda
  - 编译proto文件
  - 装载research模块
  - 添加Slim环境变量
  - COCO api 安装
- 准备数据集
- - 打标数据集
  - 将标定样本生成为.record格式文件
  - - 在data文件夹下创建create_tf_record.py如下：
    - 在data文件夹下创建read_pbtxt_file.py如下
    - 在data文件夹下创建string_int_label_map_pb2.py如下
  - 生成.record数据
- 训练样本数据
- - 下载预训练模型
  - 编辑 pipeline_config 训练配置
  - 训练模型
  - 转换模型
  - 测试
- 参考文献

Tensorflow训练maskrcnn

安装cuda cudnn Anaconda

Anaconda环境配置

换源(清华源，清华源下载更快)

清华大学开源软件镜像站

使用命令：conda config --set show_channel_urls yes在用户目录下生成.condarc
直接替换为下面的内容即可

channels:- conda-forge- https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/- https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/- https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/r/- https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/pro/- https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/msys2/- https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/- https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/msys2/- https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/bioconda/- https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/menpo/- https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/pytorch/- https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/simpleitk/- defaults
show_channel_urls: true
ssl_verify: true

或者：

conda config --set show_channel_urls yes
conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/
conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/
conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/menpo/
conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/bioconda/
conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/msys2/
conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/pytorch/

新建虚拟环境

conda create --name tensorflow1_14_0_python3_7 python=3.7

进入虚拟环境

conda activate tensorflow1_14_0_python3_7

安装tensorflow

#CPU环境选这个
conda install tensorflow=1.14.0
#GPU环境选这个
conda install tensorflow-gpu=1.14.0

测试TensorFlow是否正常，如果不正常则卸载TensorFlow，用pip安装

python
import tensorflow as tf
print(tf.test.is_gpu_available())

pip安装TensorFlow命令:

pip install tensorflow-gpu==1.14.0 -i https://pypi.tuna.tsinghua.edu.cn/simple

安装环境

conda install opencv ipython cython pillow

安装TensorFlow Object Detection API

下载TensorFlow模型源码

TensorFlow模型源码

将object detection的源码导入Anaconda

将模型源码解压，将research文件夹解压到 (安装位置\Anaconda3\Lib\site-packages)下

编译proto文件

目的：将 Anaconda3\Lib\site-packages\research\object_detection\protos 路径下的.proto文件编译为.py文件

GitHub上下载protobuf包
选win32.zip的文件，下载下来
解压后将压缩包内（/bin）目录下的protoc.exe文件放到（research/）目录下

在（research/）目录下按住shift键，点击右键打开powershell窗口，输入命令：

>>>.\protoc.exe .\object_detection\protos\*.proto --python_out=.

编译成功不会显示其他信息，进入（\research\object_detection\protos）目录，发现所有.proto文件已被编译为.py文件。

装载research模块

打开Anconda Prompt，转到（\research）目录下，输入命令:

python setup.py install

会下载很多东西，比较慢

添加Slim环境变量

在（\Anaconda3\Lib\site-packages\）目录下新建tensorflow_model.pth，输入：

F:\ProgramData\Anaconda3\envs\tensorflow1_14_0_python3_7\Lib\site-packages\research
F:\ProgramData\Anaconda3\envs\tensorflow1_14_0_python3_7\Lib\site-packages\research\slim

路径用自己的路径

若环境变量不起作用，后面会出现找不到slim的错误。 转到slim路径下，
修改BUILD为BUILD1，然后运行  ```python setup.py install ```

COCO api 安装

ubuntu下可以直接安装，windows下要找修改过的

本来 COCO 对 Windows 是不支持的。不过为了支持 Windows ，有人对 COCO 做了一些修改。下面是 COCO 在 GitHub 上源码地址信息：

COCO 地址
支持 Windows 的 COCO 地址

并切换到 *\cocoapi-master\PythonAPI 目录。运行以下指令(win)

python setup.py build_ext install

成功后会在当前环境下的 *\Lib\site-packages\ 下生成一个pycocotools…文件夹

准备数据集

打标数据集

可以用labelme打标数据集
conda install labelme
编辑*.pbtxt类别定义

item {id: 1name: 'apple'
}item {id: 2name: 'banana'
}item {id: 3name: 'orange'
}

文件结构如下：

|data
----|datasets
--------|train
------------|image
------------|json--------|val
------------|image
------------|json
--------|label.pbtxt

将标定样本生成为.record格式文件

在data文件夹下创建create_tf_record.py如下：

#!/usr/bin/env python3
# -*- coding: utf-8 -*-"""Convert raw dataset to TFRecord for object_detection.Please note that this tool only applies to labelme's annotations(json file).Example usage:python3 create_tf_record.py \--images_dir=your absolute path to read images.--annotations_json_dir=your path to annotaion json files.--label_map_path=your path to label_map.pbtxt--output_path=your path to write .record.
"""import cv2
import glob
import hashlib
import io
import json
import numpy as np
import os
import PIL.Image
import tensorflow as tfimport read_pbtxt_fileflags = tf.app.flagsflags.DEFINE_string('images_dir', None, 'Path to images directory.')
flags.DEFINE_string('annotations_json_dir', 'datasets/annotations', 'Path to annotations directory.')
flags.DEFINE_string('label_map_path', None, 'Path to label map proto.')
flags.DEFINE_string('output_path', None, 'Path to the output tfrecord.')FLAGS = flags.FLAGSdef int64_feature(value):return tf.train.Feature(int64_list=tf.train.Int64List(value=[value]))def int64_list_feature(value):return tf.train.Feature(int64_list=tf.train.Int64List(value=value))def bytes_feature(value):return tf.train.Feature(bytes_list=tf.train.BytesList(value=[value]))def bytes_list_feature(value):return tf.train.Feature(bytes_list=tf.train.BytesList(value=value))def float_list_feature(value):return tf.train.Feature(float_list=tf.train.FloatList(value=value))def create_tf_example(annotation_dict, label_map_dict=None):"""Converts image and annotations to a tf.Example proto.Args:annotation_dict: A dictionary containing the following keys:['height', 'width', 'filename', 'sha256_key', 'encoded_jpg','format', 'xmins', 'xmaxs', 'ymins', 'ymaxs', 'masks','class_names'].label_map_dict: A dictionary maping class_names to indices.Returns:example: The converted tf.Example.Raises:ValueError: If label_map_dict is None or is not containing a class_name."""if annotation_dict is None:return Noneif label_map_dict is None:raise ValueError('`label_map_dict` is None')height = annotation_dict.get('height', None)width = annotation_dict.get('width', None)filename = annotation_dict.get('filename', None)sha256_key = annotation_dict.get('sha256_key', None)encoded_jpg = annotation_dict.get('encoded_jpg', None)image_format = annotation_dict.get('format', None)xmins = annotation_dict.get('xmins', None)xmaxs = annotation_dict.get('xmaxs', None)ymins = annotation_dict.get('ymins', None)ymaxs = annotation_dict.get('ymaxs', None)masks = annotation_dict.get('masks', None)class_names = annotation_dict.get('class_names', None)labels = []for class_name in class_names:label = label_map_dict.get(class_name, 'None')if label is None:raise ValueError('`label_map_dict` is not containing {}.'.format(class_name))labels.append(label)encoded_masks = []for mask in masks:pil_image = PIL.Image.fromarray(mask.astype(np.uint8))output_io = io.BytesIO()pil_image.save(output_io, format='PNG')encoded_masks.append(output_io.getvalue())feature_dict = {'image/height': int64_feature(height),'image/width': int64_feature(width),'image/filename': bytes_feature(filename.encode('utf8')),'image/source_id': bytes_feature(filename.encode('utf8')),'image/key/sha256': bytes_feature(sha256_key.encode('utf8')),'image/encoded': bytes_feature(encoded_jpg),'image/format': bytes_feature(image_format.encode('utf8')),'image/object/bbox/xmin': float_list_feature(xmins),'image/object/bbox/xmax': float_list_feature(xmaxs),'image/object/bbox/ymin': float_list_feature(ymins),'image/object/bbox/ymax': float_list_feature(ymaxs),'image/object/mask': bytes_list_feature(encoded_masks),'image/object/class/label': int64_list_feature(labels)}example = tf.train.Example(features=tf.train.Features(feature=feature_dict))return exampledef _get_annotation_dict(images_dir, annotation_json_path):  """Get boundingboxes and masks.Args:images_dir: Path to images directory.annotation_json_path: Path to annotated json file corresponding tothe image. The json file annotated by labelme with keys:['lineColor', 'imageData', 'fillColor', 'imagePath', 'shapes','flags'].Returns:annotation_dict: A dictionary containing the following keys:['height', 'width', 'filename', 'sha256_key', 'encoded_jpg','format', 'xmins', 'xmaxs', 'ymins', 'ymaxs', 'masks','class_names'].
#
#    Raises:
#        ValueError: If images_dir or annotation_json_path is not exist."""
#    if not os.path.exists(images_dir):
#        raise ValueError('`images_dir` is not exist.')
#
#    if not os.path.exists(annotation_json_path):
#        raise ValueError('`annotation_json_path` is not exist.')if (not os.path.exists(images_dir) ornot os.path.exists(annotation_json_path)):return Nonewith open(annotation_json_path, 'r') as f:json_text = json.load(f)shapes = json_text.get('shapes', None)if shapes is None:return Noneimage_relative_path = json_text.get('imagePath', None)if image_relative_path is None:return Noneimage_name = image_relative_path.split('/')[-1]image_path = os.path.join(images_dir, image_name)image_format = image_name.split('.')[-1].replace('jpg', 'jpeg')if not os.path.exists(image_path):return Nonewith tf.gfile.GFile(image_path, 'rb') as fid:encoded_jpg = fid.read()image = cv2.imread(image_path)height = image.shape[0]width = image.shape[1]key = hashlib.sha256(encoded_jpg).hexdigest()xmins = []xmaxs = []ymins = []ymaxs = []masks = []class_names = []hole_polygons = []for mark in shapes:class_name = mark.get('label')class_names.append(class_name)polygon = mark.get('points')polygon = np.array(polygon,dtype=np.int)if class_name == 'hole':hole_polygons.append(polygon)else:mask = np.zeros(image.shape[:2])cv2.fillPoly(mask, [polygon], 1)masks.append(mask)# Boundingboxx = polygon[:, 0]y = polygon[:, 1]xmin = np.min(x)xmax = np.max(x)ymin = np.min(y)ymax = np.max(y)xmins.append(float(xmin) / width)xmaxs.append(float(xmax) / width)ymins.append(float(ymin) / height)ymaxs.append(float(ymax) / height)# Remove holes in maskfor mask in masks:mask = cv2.fillPoly(mask, hole_polygons, 0)annotation_dict = {'height': height,'width': width,'filename': image_name,'sha256_key': key,'encoded_jpg': encoded_jpg,'format': image_format,'xmins': xmins,'xmaxs': xmaxs,'ymins': ymins,'ymaxs': ymaxs,'masks': masks,'class_names': class_names}return annotation_dictdef main(_):if not os.path.exists(FLAGS.images_dir):raise ValueError('`images_dir` is not exist.')if not os.path.exists(FLAGS.annotations_json_dir):raise ValueError('`annotations_json_dir` is not exist.')if not os.path.exists(FLAGS.label_map_path):raise ValueError('`label_map_path` is not exist.')label_map = read_pbtxt_file.get_label_map_dict(FLAGS.label_map_path)writer = tf.python_io.TFRecordWriter(FLAGS.output_path)num_annotations_skiped = 0annotations_json_path = os.path.join(FLAGS.annotations_json_dir, '*.json')for i, annotation_file in enumerate(glob.glob(annotations_json_path)):if i % 100 == 0:print('On image %d', i)annotation_dict = _get_annotation_dict(FLAGS.images_dir, annotation_file)if annotation_dict is None:num_annotations_skiped += 1continuetf_example = create_tf_example(annotation_dict, label_map)writer.write(tf_example.SerializeToString())print('Successfully created TFRecord to {}.'.format(FLAGS.output_path))if __name__ == '__main__':tf.app.run()

在data文件夹下创建read_pbtxt_file.py如下

#!/usr/bin/env python3
# -*- coding: utf-8 -*-"""A tool to read .pbtxt file.See Details at:TensorFlow models/research/object_detetion/protos/string_int_label_pb2.pyTensorFlow models/research/object_detection/utils/label_map_util.py
"""import tensorflow as tffrom google.protobuf import text_formatimport string_int_label_map_pb2def load_pbtxt_file(path):"""Read .pbtxt file.Args: path: Path to StringIntLabelMap proto text file (.pbtxt file).Returns:A StringIntLabelMapProto.Raises:ValueError: If path is not exist."""if not tf.gfile.Exists(path):raise ValueError('`path` is not exist.')with tf.gfile.GFile(path, 'r') as fid:pbtxt_string = fid.read()pbtxt = string_int_label_map_pb2.StringIntLabelMap()try:text_format.Merge(pbtxt_string, pbtxt)except text_format.ParseError:pbtxt.ParseFromString(pbtxt_string)return pbtxtdef get_label_map_dict(path):"""Reads a .pbtxt file and returns a dictionary.Args:path: Path to StringIntLabelMap proto text file.Returns:A dictionary mapping class names to indices."""pbtxt = load_pbtxt_file(path)result_dict = {}for item in pbtxt.item:result_dict[item.name] = item.idreturn result_dict

在data文件夹下创建string_int_label_map_pb2.py如下

# Generated by the protocol buffer compiler.  DO NOT EDIT!
# source: object_detection/protos/string_int_label_map.protoimport sys
_b=sys.version_info[0]<3 and (lambda x:x) or (lambda x:x.encode('latin1'))
from google.protobuf import descriptor as _descriptor
from google.protobuf import message as _message
from google.protobuf import reflection as _reflection
from google.protobuf import symbol_database as _symbol_database
# @@protoc_insertion_point(imports)_sym_db = _symbol_database.Default()DESCRIPTOR = _descriptor.FileDescriptor(name='object_detection/protos/string_int_label_map.proto',package='object_detection.protos',syntax='proto2',serialized_options=None,serialized_pb=_b('\n2object_detection/protos/string_int_label_map.proto\x12\x17object_detection.protos\"G\n\x15StringIntLabelMapItem\x12\x0c\n\x04name\x18\x01 \x01(\t\x12\n\n\x02id\x18\x02 \x01(\x05\x12\x14\n\x0c\x64isplay_name\x18\x03 \x01(\t\"Q\n\x11StringIntLabelMap\x12<\n\x04item\x18\x01 \x03(\x0b\x32..object_detection.protos.StringIntLabelMapItem')
)_STRINGINTLABELMAPITEM = _descriptor.Descriptor(name='StringIntLabelMapItem',full_name='object_detection.protos.StringIntLabelMapItem',filename=None,file=DESCRIPTOR,containing_type=None,fields=[_descriptor.FieldDescriptor(name='name', full_name='object_detection.protos.StringIntLabelMapItem.name', index=0,number=1, type=9, cpp_type=9, label=1,has_default_value=False, default_value=_b("").decode('utf-8'),message_type=None, enum_type=None, containing_type=None,is_extension=False, extension_scope=None,serialized_options=None, file=DESCRIPTOR),_descriptor.FieldDescriptor(name='id', full_name='object_detection.protos.StringIntLabelMapItem.id', index=1,number=2, type=5, cpp_type=1, label=1,has_default_value=False, default_value=0,message_type=None, enum_type=None, containing_type=None,is_extension=False, extension_scope=None,serialized_options=None, file=DESCRIPTOR),_descriptor.FieldDescriptor(name='display_name', full_name='object_detection.protos.StringIntLabelMapItem.display_name', index=2,number=3, type=9, cpp_type=9, label=1,has_default_value=False, default_value=_b("").decode('utf-8'),message_type=None, enum_type=None, containing_type=None,is_extension=False, extension_scope=None,serialized_options=None, file=DESCRIPTOR),],extensions=[],nested_types=[],enum_types=[],serialized_options=None,is_extendable=False,syntax='proto2',extension_ranges=[],oneofs=[],serialized_start=79,serialized_end=150,
)_STRINGINTLABELMAP = _descriptor.Descriptor(name='StringIntLabelMap',full_name='object_detection.protos.StringIntLabelMap',filename=None,file=DESCRIPTOR,containing_type=None,fields=[_descriptor.FieldDescriptor(name='item', full_name='object_detection.protos.StringIntLabelMap.item', index=0,number=1, type=11, cpp_type=10, label=3,has_default_value=False, default_value=[],message_type=None, enum_type=None, containing_type=None,is_extension=False, extension_scope=None,serialized_options=None, file=DESCRIPTOR),],extensions=[],nested_types=[],enum_types=[],serialized_options=None,is_extendable=False,syntax='proto2',extension_ranges=[],oneofs=[],serialized_start=152,serialized_end=233,
)_STRINGINTLABELMAP.fields_by_name['item'].message_type = _STRINGINTLABELMAPITEM
DESCRIPTOR.message_types_by_name['StringIntLabelMapItem'] = _STRINGINTLABELMAPITEM
DESCRIPTOR.message_types_by_name['StringIntLabelMap'] = _STRINGINTLABELMAP
_sym_db.RegisterFileDescriptor(DESCRIPTOR)StringIntLabelMapItem = _reflection.GeneratedProtocolMessageType('StringIntLabelMapItem', (_message.Message,), dict(DESCRIPTOR = _STRINGINTLABELMAPITEM,__module__ = 'object_detection.protos.string_int_label_map_pb2'# @@protoc_insertion_point(class_scope:object_detection.protos.StringIntLabelMapItem)))
_sym_db.RegisterMessage(StringIntLabelMapItem)StringIntLabelMap = _reflection.GeneratedProtocolMessageType('StringIntLabelMap', (_message.Message,), dict(DESCRIPTOR = _STRINGINTLABELMAP,__module__ = 'object_detection.protos.string_int_label_map_pb2'# @@protoc_insertion_point(class_scope:object_detection.protos.StringIntLabelMap)))
_sym_db.RegisterMessage(StringIntLabelMap)# @@protoc_insertion_point(module_scope)

生成.record数据

#cd 到 data 路径下 python create_tf_record.py  --images_dir=./datasets/train/image  --annotations_json_dir=./datasets/train/json  --label_map_path=./datasets/label.pbtxt  --output_path=../output/train.recordpython create_tf_record.py  --images_dir=./datasets/val/image  --annotations_json_dir=./datasets/val/json  --label_map_path=./datasets/label.pbtxt  --output_path=../output/val.record

训练样本数据

新建 mask_rcnn_inception_v2_coco 文件夹

下载预训练模型

在Tensorflow detection model zoo 下载 COCO-trained models
mask_rcnn_inception_v2_coco

解压到 mask_rcnn_inception_v2_coco 文件夹下

编辑 pipeline_config 训练配置

*/research/object_detection/samples/configs/下的 mask_rcnn_inception_v2_coco.config 拷贝一份放到 mask_rcnn_inception_v2_coco 文件夹下 并将编码格式转为UTF-8

修改以下参数: 注意斜杠用正斜杠

model {faster_rcnn {num_classes: 1                                          #类别个数 (不含背景)image_resizer {keep_aspect_ratio_resizer {min_dimension: 224                                  #resize尺寸max_dimension: 224                                  #resize尺寸}}

train_config: {batch_size: 1                                                       #训练的batchsize

gradient_clipping_by_norm: 10.0fine_tune_checkpoint: "F:/deeplearning/tensorflow_maskrcnn/mask_rcnn_inception_v2_coco/mask_rcnn_inception_v2_coco_2018_01_28/model.ckpt"  #训练模板路径from_detection_checkpoint: true# Note: The below line limits the training process to 200K steps, which we# empirically found to be sufficient enough to train the pets dataset. This# effectively bypasses the learning rate schedule (the learning rate will# never decay). Remove the below line to train indefinitely.num_steps: 10000                                          #训练次数

train_input_reader: {tf_record_input_reader {input_path: "F:/deeplearning/tensorflow_maskrcnn/output/train.record"        #训练集record文件}label_map_path: "F:/deeplearning/tensorflow_maskrcnn/data/datasets/label.pbtxt"                  #类别文件load_instance_masks: truemask_type: PNG_MASKS
}

eval_input_reader: {tf_record_input_reader {input_path: "F:/deeplearning/tensorflow_maskrcnn/output/val.record"         #测试集record文件}label_map_path: "F:/deeplearning/tensorflow_maskrcnn/data/datasets/label.pbtxt"                   #类别文件load_instance_masks: truemask_type: PNG_MASKSshuffle: falsenum_readers: 1
}

训练模型

python ./research/object_detection/model_main.py --model_dir=./output --pipeline_config_path=./mask_rcnn_inception_v2_coco/mask_rcnn_inception_v2_coco.config

转换模型

python ./research/object_detection/export_inference_graph.py --input_type image_tensor  --pipeline_config_path ./mask_rcnn_inception_v2_coco/mask_rcnn_inception_v2_coco.config --trained_checkpoint_prefix ./output/model.ckpt-10000 --output_directory ./output

测试

python tf_text_graph_mask_rcnn.py  --input  ../output/frozen_inference_graph.pb  --output ../output/mask_rcnn.pbtxt  --config ../mask_rcnn_inception_v2_coco/mask_rcnn_inception_v2_coco.config
python mask_rcnn_predict.py

参考文献

https://blog.csdn.net/zlase/article/details/78734138

https://www.jianshu.com/p/27e4dc070761

https://www.pianshen.com/article/1872263299/