水表的数字表盘分割

一、安装必要的库
二、模型训练
- 1.准备表盘数据集
- 2. 模型训练
- - 2.1 配置GPU
  - 2.2 定义图像预处理流程transforms
  - 2.3 定义数据集Dataset
  - 2.4 模型开始训练
- 3.模型导出
三、封装Module
- 1.模型转换
- 2.补充代码
- 3.模型测试
四、总结与升华
个人简介

教你将对水表进行语义分割的模型贡献到PaddleHub

前不久，飞桨官方在AI Studio上挑选了45个优质项目，优质项目链接：https://shimo.im/sheets/CqQvXq3JhGqCxdXv/MODOC

开发者将上面的优质项目转换成PaddleHub模型即可获得相应奖励，目前还剩不少优质项目没有转换，大家赶紧冲冲冲！

参考资料：

水表的数字表盘分割体验 - PaddleX -DeepLabV3+语义分割
手把手带你将Paddlex模型部署为PaddleHub
【PaddleHub模型贡献】一行代码实现从彩色图提取素描线稿

一、安装必要的库

!pip install paddlex -i https://mirror.baidu.com/pypi/simple
!pip install --upgrade paddlepaddle-gpu -i https://pypi.tuna.tsinghua.edu.cn/simple
!pip install --upgrade paddlehub==2.0.1 -i https://pypi.tuna.tsinghua.edu.cn/simple

二、模型训练

项目作者使用PaddleX做的语义分割，因为作者没有直接公开训练好的模型，所以这里我们先按照作者的思路复现模型。

1.准备表盘数据集

!unzip -oq /home/aistudio/data/data73852/water.zip

2. 模型训练

2.1 配置GPU

# 设置使用0号GPU卡（如无GPU，执行此代码后仍然会使用CPU训练模型）
import matplotlib
matplotlib.use('Agg')
import os
os.environ['CUDA_VISIBLE_DEVICES'] = '0'
import paddlex as pdx

2.2 定义图像预处理流程transforms

定义数据处理流程，其中训练和测试需分别定义，训练过程包括了部分测试过程中不需要的数据增强操作，如在本示例中，训练过程使用了RandomHorizontalFlip和RandomPaddingCrop两种数据增强方式，更多图像预处理流程transforms的使用可参见paddlex.seg.transforms。

from paddlex.seg import transforms
train_transforms = transforms.Compose([transforms.RandomHorizontalFlip(),transforms.Resize(target_size=512),transforms.RandomPaddingCrop(crop_size=500),transforms.Normalize()
])
eval_transforms = transforms.Compose([transforms.Resize(512),transforms.Normalize()
])

2.3 定义数据集Dataset

实例分割使用SegDataset格式的数据集，因此采用pdx.datasets.SegDataset来加载数据集，该接口的介绍可参见文档pdx.datasets.SegDataset。

train_dataset = pdx.datasets.SegDataset(data_dir='water',file_list='water/train.txt',label_list='water/class_names.txt',transforms=train_transforms,shuffle=True)
eval_dataset = pdx.datasets.SegDataset(data_dir='water',file_list='water/val.txt',label_list='water/class_names.txt',transforms=eval_transforms)

2021-03-11 14:54:48 [INFO]   150 samples in file water/train.txt
2021-03-11 14:54:48 [INFO]  11 samples in file water/val.txt

2.4 模型开始训练

使用本数据集在P40上训练，如有GPU，模型的训练过程预估为13分钟左右；如无GPU，则预估为5小时左右。更多训练模型的参数可参见文档paddlex.seg.DeepLabv3p。模型训练过程每间隔save_interval_epochs轮会保存一次模型在save_dir目录下，同时在保存的过程中也会在验证数据集上计算相关指标，具体相关日志参见文档。

num_classes = len(train_dataset.labels)
model = pdx.seg.DeepLabv3p(num_classes=num_classes)
model.train(num_epochs=40,train_dataset=train_dataset,train_batch_size=4,eval_dataset=eval_dataset,learning_rate=0.01,save_interval_epochs=1,# pretrain_weights='output/deeplab4/best_model',save_dir='output/water')

最后一轮的输出如下所示：

2021-03-11 15:02:56 [INFO]   [TRAIN] Epoch=40/40, Step=1/37, loss=0.010831, lr=0.000362, time_each_step=0.18s, eta=0:0:10
2021-03-11 15:02:56 [INFO]  [TRAIN] Epoch=40/40, Step=3/37, loss=0.010944, lr=0.000344, time_each_step=0.2s, eta=0:0:10
2021-03-11 15:02:57 [INFO]  [TRAIN] Epoch=40/40, Step=5/37, loss=0.009099, lr=0.000326, time_each_step=0.22s, eta=0:0:10
2021-03-11 15:02:57 [INFO]  [TRAIN] Epoch=40/40, Step=7/37, loss=0.011186, lr=0.000308, time_each_step=0.24s, eta=0:0:10
2021-03-11 15:02:57 [INFO]  [TRAIN] Epoch=40/40, Step=9/37, loss=0.008269, lr=0.00029, time_each_step=0.25s, eta=0:0:10
2021-03-11 15:02:58 [INFO]  [TRAIN] Epoch=40/40, Step=11/37, loss=0.011792, lr=0.000272, time_each_step=0.25s, eta=0:0:10
2021-03-11 15:02:58 [INFO]  [TRAIN] Epoch=40/40, Step=13/37, loss=0.010976, lr=0.000254, time_each_step=0.26s, eta=0:0:9
2021-03-11 15:02:58 [INFO]  [TRAIN] Epoch=40/40, Step=15/37, loss=0.01399, lr=0.000236, time_each_step=0.26s, eta=0:0:9
2021-03-11 15:02:58 [INFO]  [TRAIN] Epoch=40/40, Step=17/37, loss=0.009998, lr=0.000217, time_each_step=0.26s, eta=0:0:8
2021-03-11 15:02:58 [INFO]  [TRAIN] Epoch=40/40, Step=19/37, loss=0.012266, lr=0.000198, time_each_step=0.26s, eta=0:0:8
2021-03-11 15:02:58 [INFO]  [TRAIN] Epoch=40/40, Step=21/37, loss=0.011713, lr=0.00018, time_each_step=0.13s, eta=0:0:5
2021-03-11 15:02:58 [INFO]  [TRAIN] Epoch=40/40, Step=23/37, loss=0.010291, lr=0.00016, time_each_step=0.11s, eta=0:0:5
2021-03-11 15:02:58 [INFO]  [TRAIN] Epoch=40/40, Step=25/37, loss=0.010211, lr=0.000141, time_each_step=0.09s, eta=0:0:4
2021-03-11 15:02:59 [INFO]  [TRAIN] Epoch=40/40, Step=27/37, loss=0.02097, lr=0.000121, time_each_step=0.08s, eta=0:0:4
2021-03-11 15:02:59 [INFO]  [TRAIN] Epoch=40/40, Step=29/37, loss=0.008198, lr=0.000101, time_each_step=0.07s, eta=0:0:3
2021-03-11 15:02:59 [INFO]  [TRAIN] Epoch=40/40, Step=31/37, loss=0.010346, lr=8.1e-05, time_each_step=0.06s, eta=0:0:3
2021-03-11 15:02:59 [INFO]  [TRAIN] Epoch=40/40, Step=33/37, loss=0.009331, lr=6e-05, time_each_step=0.06s, eta=0:0:3
2021-03-11 15:02:59 [INFO]  [TRAIN] Epoch=40/40, Step=35/37, loss=0.01259, lr=3.8e-05, time_each_step=0.06s, eta=0:0:3
2021-03-11 15:02:59 [INFO]  [TRAIN] Epoch=40/40, Step=37/37, loss=0.013072, lr=1.4e-05, time_each_step=0.06s, eta=0:0:3
2021-03-11 15:02:59 [INFO]  [TRAIN] Epoch 40 finished, loss=0.011522, lr=0.000195 .
2021-03-11 15:02:59 [INFO]  Start to evaluating(total_samples=11, total_steps=3)...
100%|██████████| 3/3 [00:02<00:00,  1.00it/s]
2021-03-11 15:03:02 [INFO]  [EVAL] Finished, Epoch=40, miou=0.814756, category_iou=[0.99168644 0.63782582], oacc=0.991806, category_acc=[0.99431391 0.84710874], kappa=0.774722, category_F1-score=[0.99582587 0.77886893] .
2021-03-11 15:03:03 [INFO]  Model saved in output/water/epoch_40.
2021-03-11 15:03:03 [INFO]  Current evaluated best model in eval_dataset is epoch_35, miou=0.8284633456567256

3.模型导出

模型训练时会自动保存模型参数，我们需要把训练模型导出成可预测模型。

!paddlex --export_inference --model_dir=output/water/best_model --save_dir=./inference_model

/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/setuptools/depends.py:2: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative usesimport imp
W0311 15:49:28.613981   782 device_context.cc:362] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 11.0, Runtime API Version: 10.1
W0311 15:49:28.618839   782 device_context.cc:372] device: 0, cuDNN Version: 7.6.
2021-03-11 15:49:32 [INFO]  Model[DeepLabv3p] loaded.
2021-03-11 15:49:32 [INFO]  Model for inference deploy saved in ./inference_model.

三、封装Module

下面正式开始模型转换！

1.模型转换

PaddleX模型可以快速转换成PaddleHub模型，只需要用下面这一句命令即可：

!hub convert --model_dir inference_model \--module_name WatermeterSegmentation \--module_version 1.0.0 \--output_dir outputs

转换成功后的模型保存在outputs文件夹下，我们解压一下：

!gzip -dfq /home/aistudio/outputs/WatermeterSegmentation.tar.gz
!tar -xf /home/aistudio/outputs/WatermeterSegmentation.tar

2.补充代码

刚刚转换的模型其实已经是PaddleHub的Module了，但是原项目中，作者做了一些图片的裁剪等操作，把数字提取出来了，因此，我们需要把这部分代码补充进去。

完整的module.py文件内容如下：

from __future__ import absolute_import
from __future__ import divisionimport os
import cv2
import argparse
import base64
import paddlex as pdxfrom math import *
import time, math, reimport numpy as np
import paddlehub as hub
from paddlehub.module.module import moduleinfo, runnable, servingdef base64_to_cv2(b64str):data = base64.b64decode(b64str.encode('utf8'))data = np.fromstring(data, np.uint8)data = cv2.imdecode(data, cv2.IMREAD_COLOR)return datadef cv2_to_base64(image):# return base64.b64encode(image)data = cv2.imencode('.jpg', image)[1]return base64.b64encode(data.tostring()).decode('utf8')def read_images(paths):images = []for path in paths:images.append(cv2.imread(path))return images'''旋转图像并剪裁'''
def rotate(img,  # 图片pt1, pt2, pt3, pt4,imgOutSrc):# print(pt1,pt2,pt3,pt4)withRect = math.sqrt((pt4[0] - pt1[0]) ** 2 + (pt4[1] - pt1[1]) ** 2)  # 矩形框的宽度heightRect = math.sqrt((pt1[0] - pt2[0]) ** 2 + (pt1[1] - pt2[1]) **2)# print("矩形的宽度",withRect, "矩形的高度", heightRect)angle = acos((pt4[0] - pt1[0]) / withRect) * (180 / math.pi)  # 矩形框旋转角度# print("矩形框旋转角度", angle)if withRect > heightRect:if pt4[1]>pt1[1]:# print("顺时针旋转")passelse:# print("逆时针旋转")angle=-angleelse:# print("逆时针旋转")angle=90 - angleheight = img.shape[0]  # 原始图像高度width = img.shape[1]   # 原始图像宽度rotateMat = cv2.getRotationMatrix2D((width / 2, height / 2), angle, 1)  # 按angle角度旋转图像heightNew = int(width * fabs(sin(radians(angle))) + height * fabs(cos(radians(angle))))widthNew = int(height * fabs(sin(radians(angle))) + width * fabs(cos(radians(angle))))rotateMat[0, 2] += (widthNew - width) / 2rotateMat[1, 2] += (heightNew - height) / 2imgRotation = cv2.warpAffine(img, rotateMat, (widthNew, heightNew), borderValue=(255, 255, 255))# cv2.imwrite("imgRotation.jpg", imgRotation) # 旋转后图像的四点坐标[[pt1[0]], [pt1[1]]] = np.dot(rotateMat, np.array([[pt1[0]], [pt1[1]], [1]]))[[pt3[0]], [pt3[1]]] = np.dot(rotateMat, np.array([[pt3[0]], [pt3[1]], [1]]))[[pt2[0]], [pt2[1]]] = np.dot(rotateMat, np.array([[pt2[0]], [pt2[1]], [1]]))[[pt4[0]], [pt4[1]]] = np.dot(rotateMat, np.array([[pt4[0]], [pt4[1]], [1]]))# 处理反转的情况if pt2[1]>pt4[1]:pt2[1],pt4[1]=pt4[1],pt2[1]if pt1[0]>pt3[0]:pt1[0],pt3[0]=pt3[0],pt1[0]imgOut = imgRotation[int(pt2[1]):int(pt4[1]), int(pt1[0]):int(pt3[0])]cv2.imwrite(imgOutSrc, imgOut) # 裁减得到的旋转矩形框@moduleinfo(name='WatermeterSegmentation',type='CV/semantic_segmentatio',author='郑博培、彭兆帅',author_email='2733821739@qq.com',summary='Digital dial segmentation of water meter',version='1.0.0')
class MODULE(hub.Module):def _initialize(self, **kwargs):self.default_pretrained_model_path = os.path.join(self.directory, 'assets')self.model = pdx.deploy.Predictor(self.default_pretrained_model_path,**kwargs)def predict(self,images=None,paths=None,data=None,batch_size=1,use_gpu=False,**kwargs):all_data = images if images is not None else read_images(paths)total_num = len(all_data)loop_num = int(np.ceil(total_num / batch_size))res = []for iter_id in range(loop_num):batch_data = list()handle_id = iter_id * batch_sizefor image_id in range(batch_size):try:batch_data.append(all_data[handle_id + image_id])except IndexError:breakout = self.model.batch_predict(batch_data, **kwargs)res.extend(out)return resdef cutPic(self, picUrl):# seg = hub.Module(name='WatermeterSegmentation')image_name = picUrlim = cv2.imread(image_name)result = self.predict(images=[im])# 将多边形polygon转矩形contours, hier = cv2.findContours(result[0]['label_map'], cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE) print(type(contours[0]))n = 0m = 0for index,contour in enumerate(contours):if len(contour) > n:n = len(contour)m = indeximage = cv2.imread(image_name)# 获取最小的矩形rect = cv2.minAreaRect(contours[m])box = np.int0(cv2.boxPoints(rect))# 获取到矩形的四个点tmp = cv2.drawContours(image, [box], 0, (0, 0, 255), 3)imgOutSrc = 'result.jpg'rotate(image, box[0], box[1], box[2], box[3], imgOutSrc)res = []res.append(imgOutSrc)return res@servingdef serving_method(self, images, **kwargs):"""Run as a service."""images_decode = [base64_to_cv2(image) for image in images]results = self.predict(images_decode, **kwargs)res = []for result in results:if isinstance(result, dict):# result_new = dict()for key, value in result.items():if isinstance(value, np.ndarray):result[key] = cv2_to_base64(value)elif isinstance(value, np.generic):result[key] = np.asscalar(value)elif isinstance(result, list):for index in range(len(result)):for key, value in result[index].items():if isinstance(value, np.ndarray):result[index][key] = cv2_to_base64(value)elif isinstance(value, np.generic):result[index][key] = np.asscalar(value)else:raise RuntimeError('The result cannot be used in serving.')res.append(result)return res@runnabledef run_cmd(self, argvs):"""Run as a command."""self.parser = argparse.ArgumentParser(description="Run the {} module.".format(self.name),prog='hub run {}'.format(self.name),usage='%(prog)s',add_help=True)self.arg_input_group = self.parser.add_argument_group(title="Input options", description="Input data. Required")self.arg_config_group = self.parser.add_argument_group(title="Config options",description="Run configuration for controlling module behavior, not required.")self.add_module_config_arg()self.add_module_input_arg()args = self.parser.parse_args(argvs)results = self.predict(paths=[args.input_path],use_gpu=args.use_gpu)return resultsdef add_module_config_arg(self):"""Add the command config options."""self.arg_config_group.add_argument('--use_gpu',type=bool,default=False,help="whether use GPU or not")def add_module_input_arg(self):"""Add the command input options."""self.arg_input_group.add_argument('--input_path', type=str, help="path to image.")if __name__ == '__main__':module = MODULE(directory='./new_model')images = [cv2.imread('./cat.jpg'), cv2.imread('./cat.jpg'), cv2.imread('./cat.jpg')]res = module.predict(images=images)

3.模型测试

首先安装我们刚刚写好的Module：

!hub install WatermeterSegmentation

/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/setuptools/depends.py:2: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative usesimport imp
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/__init__.py:107: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop workingfrom collections import MutableMapping
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/rcsetup.py:20: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop workingfrom collections import Iterable, Mapping
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/colors.py:53: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop workingfrom collections import Sized
[32m[2021-03-11 16:42:50,225] [    INFO][0m - Successfully uninstalled WatermeterSegmentation[0m
[32m[2021-03-11 16:42:50,441] [    INFO][0m - Successfully installed WatermeterSegmentation-1.0.0[0m
[0m

模型调用：

import cv2
import paddlehub as hubseg = hub.Module(name='WatermeterSegmentation')
meterSegmentation')
res = seg.cutPic(picUrl="water/images/val/20200521105032.png")

[2021-03-11 17:13:36,113] [ WARNING] - The _initialize method in HubModule will soon be deprecated, you can use the __init__() to handle the initialization of the object<class 'numpy.ndarray'>

预测结果如下。

输入图片：

最终将截取的图片显示效果如下：

四、总结与升华

总的来说，将PaddleX的模型转换成PaddleHub模型其实还是非常容易去实现的，我这次算了一下时长，基本上半天可以搞定。

个人简介

北京联合大学机器人学院自动化专业 2018级本科生郑博培
百度飞桨开发者技术专家 PPDE
百度飞桨官方帮帮团、答疑团成员
深圳柴火创客空间认证会员
百度大脑智能对话训练师
阿里云人工智能、DevOps助理工程师

我在AI Studio上获得至尊等级，点亮9个徽章，来互关呀！！！

https://aistudio.baidu.com/aistudio/personalcenter/thirdview/147378

【PaddleHub模型贡献】一行代码实现水表的数字表盘分割相关推荐

C#中一行代码实现18位数字时间戳转换为DateTime
场景存取的时间戳数据为: 636728674347302002 怎样将其转换为DateTime时间. 目前大多数的策略是,转换成string,然后 DateTime dateTimeStart = ...
java 水表识别_水表识别 --数字的分割
#将水表截取下来的数字进行识别 from cv2 import cv2 import numpy as np from matplotlib import pyplot as plt from PI ...
基于飞桨PaddlePaddle实现素描线稿提取——仅需一行代码即可实现模型调用
一行代码实现从彩色图提取素描线稿一.前言:初识PaddleHub 预训练模型(Pre-Trained Models) 二.构建Module的大框架 1.将模型文件存放在infer_model下 2. ...
PaddleHub新增100+优质模型，少量代码快速搭建AI魔镜带你对话未来！
你是否也曾经幻想能和未来的自己对话,问问未来的自己过的怎么样,当初的梦想都实现了吗?现在PaddleHub可以帮你轻松实现,先快速看下效果. 这个项目中用到了PaddleHub中的最新模型,分别用到了 ...
PyTorch Hub发布！一行代码调用最潮模型，图灵奖得主强推
文章来源:量子位原文地址:https://mp.weixin.qq.com/s/lS3YiXzYyY6-XNTFyH_GHg 如有兴趣可以**点击加入极市CV专业微信群**,获取更多高质量干货为了 ...
PyTorch Hub发布获Yann LeCun强推！一行代码调用经典模型
作者 | Team PyTorch 译者 | Monanfei 责编 | 夕颜出品 | AI科技大本营(ID: rgznai100) 导读:6月11日,Facebook PyTorch 团队推出了全 ...
一行代码不用写，就可以训练、测试、使用模型，这个star量1.5k的项目帮你做到...
机器之心报道机器之心编辑部 igel 是 GitHub 上的一个热门工具,基于 scikit-learn 构建,支持 sklearn 的所有机器学习功能,如回归.分类和聚类.用户无需编写一行代码即可 ...
pytorch 获取模型参数_剑指TensorFlow，PyTorch Hub官方模型库一行代码复现主流模型...
选自PyTorch 机器之心编译参与:思源.一鸣经典预训练模型.新型前沿研究模型是不是比较难调用?PyTorch 团队今天发布了模型调用神器 PyTorch Hub,只需一行代码,BERT.GPT ...

【PaddleHub模型贡献】一行代码实现水表的数字表盘分割