42、使用mmrotate中k3det进行旋转目标检测，并进行mnn部署和ncnn部署
2024-05-08 12:32:33
基本思想：仍然是身份证分割，因为上一个篇博客的效果不好，所以操刀改mm系列的框架，并进行ncnn和mnn的c++的部署开发
mmcv_full 1.6.1＋mmrotate v0.3.2测试没有问题
mmcv_full1.4.６+mmrotate v0.3.0版本如果报错 ，需要删掉/home/ubuntu/v0.3.0/mmrotate/mmrotate/models/dense_head/oriented_reppoints_head.py报错提示的引用包chamfer_distance
mmrotate源码和mnn源码使用20220813之前日期的测试没有问题,
第一步下载源码
ubuntu@ubuntu-Super-Server:~/sxj731533730$ git clone https://github.com/open-mmlab/mmrotate.gitCloning into 'mmrotate'...
remote: Enumerating objects: 23529, done.
remote: Total 23529 (delta 0), reused 0 (delta 0), pack-reused 23529
Receiving objects: 100% (23529/23529), 35.29 MiB | 3.54 MiB/s, done.
Resolving deltas: 100% (16443/16443), done.ubuntu@ubuntu-Super-Server:~/sxj731533730$ cd mmrotate/
ubuntu@ubuntu-Super-Server:~/sxj731533730/mmrotate$ pip3 install -r requirements.txt第二步：下载模型和测试
ubuntu@ubuntu-Super-Server:~/sxj731533730/mmrotaten$ mkdir model
ubuntu@ubuntu-Super-Server:~/sxj731533730/mmrotate$ cd model/
ubuntu@ubuntu-Super-Server:~/sxj731533730/mmrotate/model$ cat ../configs/kfiou/README.md
ubuntu@ubuntu-Super-Server:~/sxj731533730/mmrotate/model$ wget https://download.openmmlab.com/mmrotate/v0.1.0/kfiou/r3det_kfiou_ln_r50_fpn_1x_dota_oc/r3det_kfiou_ln_r50_fpn_1x_dota_oc-8e7f049d.pth
#ubuntu@ubuntu-Super-Server:~/sxj731533730/mmrotate/model$ axel -n 100 https://download.openmmlab.com/mmrotate/v0.1.0/kfiou/r3det_kfiou_ln_r50_fpn_1x_dota_oc/r3det_kfiou_ln_r50_fpn_1x_dota_oc-8e7f049d.pth
ubuntu@ubuntu-Super-Server:~/sxj731533730/mmrotate/model$ pip install mmcv-full==1.4.6 -f https://download.openmmlab.com/mmcv/dist/cu113/torch1.10.0/index.html然后修改一下测试程序
ubuntu@ubuntu-Super-Server:~/sxj731533730/mmrotate/demo$ python3 image_demo.py demo.jpg ../configs/kfiou/r3det_kfiou_ln_
r50_fpn_1x_dota_oc.py ../model/r3det_kfiou_ln_r50_fpn_1x_dota_oc-8e7f049d.pth --output_file sxj731533730.jpg
第三步：首先开始造数据集，比如我只标注了四张身份证图片，且为4个坐标点图 一个反面 三个反面 一个正面 三个正面的 （正面的图就不上了）
标注方式参考官网解释 https://captain-whu.github.io/DOTA/index.html
x1, y1, x2, y2, x3, y3, x4, y4：四边形的四个顶点的坐标 顶点按顺时针顺序排列，第一个起点为左上第一个点使用的labelme 标注的，也可以使用rolabelimg标注~
使用旋转代码，进行生成批量角度图片
# -*- coding: utf-8 -*-
import os
import sys
import json
import io
import random
import re
import cv2
import numpy as np
from random import choice
import mathsource_path = r'A'
destination_path = r'B'
angle=[]
for item in range(0,360,15):angle.append(item)for angle_item in angle:article_info = {}data_json = json.loads(json.dumps(article_info))data_json['version'] = '5.0.1'data_json['flags'] = {}data_json["lineColor"] = [0,255,0,128]data_json["fillColor"] = [255,0,0,128]def file_name(file_dir):L = []for root, dirs, files in os.walk(file_dir):for file in files:if os.path.splitext(file)[1] == '.json':L.append(os.path.join(root, file))return Ldef rotation_point(img, angle,pts):cols = img.shape[1]rows = img.shape[0]M = cv2.getRotationMatrix2D((cols / 2, rows / 2), angle, 1)heightNew = int(cols * math.fabs(math.sin(math.radians(angle))) + rows * math.fabs(math.cos(math.radians(angle))))widthNew = int(rows * math.fabs(math.sin(math.radians(angle))) + cols * math.fabs(math.cos(math.radians(angle))))M[0, 2] += (widthNew - cols) / 2M[1, 2] += (heightNew - rows) / 2img = cv2.warpAffine(img, M, (widthNew, heightNew))pts = cv2.transform(np.asarray(pts, dtype=np.float64).reshape((-1, 1, 2)), M)return img,ptsfor name in enumerate(file_name(source_path)):shape_json = []m_path = name[1]dir = os.path.dirname(m_path)file_json = io.open(m_path, 'r', encoding='utf-8')json_data = file_json.read()data = json.loads(json_data)data_json['imageData'] = Nonedata_name = data['imagePath']data_path = os.path.join(dir ,data_name)object_name = os.path.splitext(data['imagePath'])[0]data_new_json_name =Nonelist_point=[]for i in range(len(data['shapes'])):m_name_0 = data['shapes'][i]['label']print('m_name_0=', m_name_0)item_point = []for j in range(len(data['shapes'][i]['points'])):print(data['shapes'][i]['points'][j][0],data['shapes'][i]['points'][j][1])item_point.append([data['shapes'][i]['points'][j][0],data['shapes'][i]['points'][j][1]])data_json_fill_color=Nonedata_json_rec = data['shapes'][i]['shape_type']img = cv2.imread(data_path)im_rotate,item_point = rotation_point(img, angle_item,np.asarray(item_point))item_point=np.squeeze(item_point).tolist()print(item_point)(filename, extension) = os.path.splitext(data_name)data_new_picture_name = os.path.join(destination_path ,filename+ ".".join([str(angle_item), "jpg"]))data_new_json_name = os.path.join(destination_path , filename+".".join([str(angle_item), "json"]))data_json['imagePath'] = filename + ".".join([str(angle_item) , "jpg"])cv2.imwrite(data_new_picture_name, im_rotate)im_rotate = cv2.imread(data_new_picture_name)data_json['imageWidth'] = im_rotate.shape[1]data_json['imageHeight'] = im_rotate.shape[0]shape_json_item = {"label": m_name_0,"points": item_point, "shape_type": data_json_rec}shape_json.append(shape_json_item)data_json['shapes'] = shape_jsondata_info = json.dumps(data_json, ensure_ascii=False)fp = open(data_new_json_name, "w+")json.dump(data_info, fp, ensure_ascii=False, indent=4)fp.close()fp = open(data_new_json_name, "r")for x in fp.readlines():y = x.replace("\\\"", "\"")z = y.replace("\"{", "{")w = z.replace("}\"", "}")fp.close()fp = open(data_new_json_name, "w+")fp.write(w)fp.close()只贴两张
第四步：训练我们自己的数据集
ubuntu@ubuntu-Super-Server:~/sxj731533730/mmrotate$ mkdir -p kfiouDataSets里面是标准的jpg和json（labelme标注的格式）
ubuntu@ubuntu-Super-Server:~/sxj731533730/mmrotate/kfiouDataSets$ tree -L 1├── trainDataset #里面含有jpg和对用json文件└── valDataset #里面含有jpg和对用json文件└── testDataset #里面含有jpg和对用json文件3 directories, 0 files
生成 DOTA数据集（参考附录作者修改了下面脚本） labelme2dota.py
生成方式格式 每个图片目标对应一个txt 解释权 https://captain-whu.github.io/DOTA/index.html
x1, y1, x2, y2, x3, y3, x4, y4, category, difficult
x1, y1, x2, y2, x3, y3, x4, y4, category, difficult
category: 标签名字
difficult：表示标签检测的难易程度 （1表示困难，0表示不困难）转换脚本 labelme2dota ,我担心顺时针排序代码写的有问题，也做了反向转换验证,因为 mmrotate 代码里训练需要png 图片 还是不动代码了 改图片格式吧
import json
import os
from glob import glob
import argparse
import numpy as np
import shutil
from PIL import Image
import cv2# convert labelme json to DOTA txt format
# convert DOTA json to lableme txt format
def custombasename(fullname):return os.path.basename(os.path.splitext(fullname)[0])def order_points_new(pts): # clock -https://zhuanlan.zhihu.com/p/10643062# sort the points based on their x-coordinatesxSorted = pts[np.argsort(pts[:, 0]), :]# grab the left-most and right-most points from the sorted# x-roodinate pointsleftMost = xSorted[:2, :]rightMost = xSorted[2:, :]if leftMost[0, 1] != leftMost[1, 1]:leftMost = leftMost[np.argsort(leftMost[:, 1]), :]else:leftMost = leftMost[np.argsort(leftMost[:, 0])[::-1], :](tl, bl) = leftMostif rightMost[0, 1] != rightMost[1, 1]:rightMost = rightMost[np.argsort(rightMost[:, 1]), :]else:rightMost = rightMost[np.argsort(rightMost[:, 0])[::-1], :](tr, br) = rightMost# return the coordinates in top-left, top-right,# bottom-right, and bottom-left orderreturn np.array([tl, tr, br, bl], dtype="float32")parser = argparse.ArgumentParser(formatter_class=argparse.ArgumentDefaultsHelpFormatter
)
parser.add_argument('--input_dir', default=r'C:\Users\PHILIPS\Desktop\source', type=str,help='input annotated directory')
parser.add_argument('--output_images', default=r'C:\Users\PHILIPS\Desktop\datasets\images', type=str,help='input annotated directory')
parser.add_argument('--output_dir', default=r'C:\Users\PHILIPS\Desktop\datasets\labelTxt', type=str, help='output dataset directory')
parser.add_argument('--verify_dir', default=r'C:\Users\PHILIPS\Desktop\datasets\verify', type=str,help='verify dataset directory')
parser.add_argument('--verify', default=True, type=bool, help='verify')
parser.add_argument('--labels', default=r'labels.txt', type=str, help='labels annotated directory')
args = parser.parse_args()if not os.path.exists(args.output_dir):os.makedirs(args.output_dir)
if not os.path.exists(args.output_images):os.makedirs(args.output_images)
print('Creating dataset:', args.output_dir)file_list = glob(os.path.join(args.input_dir, ".".join(["*", "json"])))for i in range(len(file_list)):with open(file_list[i]) as f:label_str = f.read()label_dict = json.loads(label_str) # json文件读入dict# 输出 txt 文件的路径out_file = os.path.join(args.output_dir, ".".join([custombasename(file_list[i]), 'txt']))# 写入 poly 四点坐标 和 labelfout = open(out_file, 'w')out_str = ''# np.array(box, dtype="int")for shape_dict in label_dict['shapes']:points = shape_dict['points']item_points = []for p in points:item_points.append([p[0], p[1]])item_points = order_points_new(np.array(item_points, dtype="float"))for p in item_points.tolist():out_str += (str(p[0]) + ' ' + str(p[1]) + ' ')out_str += shape_dict['label'] + ' 0\n'fout.write(out_str)fout.close()print('%d/%d' % (i + 1, len(file_list)))print("labelme2dota...")
if args.verify:if not os.path.exists(args.verify_dir):os.makedirs(args.verify_dir)txt_list = glob(os.path.join(args.output_dir, ".".join(["*", "txt"])))for i in range(len(txt_list)):(filepath, tempfilename) = os.path.split(txt_list[i])(filename, extension) = os.path.splitext(tempfilename)sourcePath = Noneimage_filename = Noneif os.path.exists(os.path.join(args.input_dir, ".".join([filename, "jpg"]))):sourcePath = os.path.join(args.input_dir, ".".join([filename, "jpg"]))image_filename = ".".join([filename, "png"])elif os.path.exists(os.path.join(args.input_dir, ".".join([filename, "png"]))):sourcePath = os.path.join(args.input_dir, ".".join([filename, "png"]))image_filename = ".".join([filename, "png"])if sourcePath is None:print("check photo type")continuetargetPath = os.path.join(args.verify_dir,image_filename)targetpng = os.path.join(args.output_images, image_filename)shutil.copy(sourcePath, targetPath)shutil.copy(sourcePath, targetpng)img = Image.open(sourcePath)imgSize = img.size # 大小/尺寸w = img.width # 图片的宽h = img.height # 图片的高data = {}data['imagePath'] = image_filenamedata['flags'] = {}data['imageWidth'] = wdata['imageHeight'] = hdata['imageData'] = Nonedata['version'] = "5.0.1"data["shapes"] = []with open(txt_list[i]) as f:label_str = f.readlines()for label_item in label_str:line_char = label_item.split("\n")[0].split(' ')points = [[eval(line_char[0]), eval(line_char[1])], [eval(line_char[2]), eval(line_char[3])],[eval(line_char[4]), eval(line_char[5])], [eval(line_char[6]), eval(line_char[7])]]itemData = {'points': []}itemData['points'].extend(points)itemData["flag"] = {}itemData["group_id"] = NoneitemData["shape_type"] = "polygon"itemData["label"] = line_char[-2]data["shapes"].append(itemData)jsonName = ".".join([filename, "json"])jsonPath = os.path.join(args.verify_dir, jsonName)with open(jsonPath, "w") as f:json.dump(data, f)print(jsonName)print("dota2labelme...")
执行命令,最好在本地用该脚本，熟悉一下转换方式
ubuntu@ubuntu-Super-Server:~/sxj731533730/mmrotate/kfiouDataSets$ sudo vim labels.txt
ubuntu@ubuntu-Super-Server:~/sxj731533730/mmrotate/kfiouDataSets$ cat labels.txt
cardubuntu@ubuntu-Super-Server:~/sxj731533730/mmrotate$ python3 labelme2dota.py --input_dir kfiouDataSets/trainDataset/ --output_dir kfiouDataSets/train --verify false --labels kfiouDataSets/labels.txt
ubuntu@ubuntu-Super-Server:~/sxj731533730/mmrotate$ python3 labelme2dota.py --input_dir kfiouDataSets/valDataset/ --output_dir kfiouDataSets/val --verify false --labels kfiouDataSets/labels.txt
ubuntu@ubuntu-Super-Server:~/sxj731533730/mmrotate$ python3 labelme2dota.py --input_dir kfiouDataSets/testDataset/ --output_dir kfiouDataSets/test --verify false --labels kfiouDataSets/labels.txt划分训练和验证 测试集
import os
from glob import glob
import shutil
import randomann_txt=r"C:\Users\PHILIPS\Desktop\dest"
train_annfile=r"C:\Users\PHILIPS\Desktop\train_annfile"
test_annfile=r"C:\Users\PHILIPS\Desktop\test_annfile"
val_annfile=r"C:\Users\PHILIPS\Desktop\val_annfile"train_num=0.8
test_num=0.2
val_num=0.2if not os.path.exists(train_annfile):os.makedirs(train_annfile)if not os.path.exists(test_annfile):os.makedirs(test_annfile)
if not os.path.exists(val_annfile):os.makedirs(val_annfile)txt_list = glob(os.path.join(ann_txt, ".".join(["*", "txt"])))
train_list = random.sample(txt_list, int(train_num*len(txt_list)))
val_list = random.sample(txt_list, int(val_num*len(txt_list)))
test_list = random.sample(txt_list, int(test_num*len(txt_list)))for idx,item in enumerate(txt_list):source_file_txt=os.path.join(ann_txt,item)if item in train_list:shutil.copy(source_file_txt, train_annfile)elif item in val_list:shutil.copy(source_file_txt, val_annfile)else:shutil.copy(source_file_txt, test_annfile)print("copy txt into file")
print("complish")整个目录结构
ubuntu@ubuntu-Super-Server:~/sxj731533730/mmrotate$ tree -L 1
.
├── CITATION.cff
├── configs
├── demo
├── docker
├── docs
├── generate.py
├── kfiouDataSets
├── LICENSE
├── MANIFEST.in
├── mmrotate
├── model
├── model-index.yml
├── README.md
├── README_zh-CN.md
├── requirements
├── requirements.txt
├── resources
├── setup.cfg
├── setup.py
├── tests
└── tools11 directories, 10 files修改配置文件参数和增加参数,目录
ubuntu@ubuntu-Super-Server:~/sxj731533730/mmrotate/kfiouDataSets$ tree -L 1
.
├── labels.txt
├── source
├── test_annfile
├── train_annfile
└── val_annfile4 directories, 1 file修改配置文件
home/ubuntu/sxj731533730/mmrotate/configs/_base_/datasets/dotav1.py变更数据集的根目录和训练集 测试集 验证集的目录
# dataset settings
dataset_type = 'DOTADataset'
data_root = '/home/ubuntu/sxj731533730/mmrotate/kfiouDataSets/'
。。。。。
data = dict(samples_per_gpu=2,workers_per_gpu=2,train=dict(type=dataset_type,#classes=('card',),ann_file=data_root + 'train_annfile/',img_prefix=data_root + 'source/',pipeline=train_pipeline),val=dict(type=dataset_type,#classes=('card',),ann_file=data_root + 'val_annfile/',img_prefix=data_root + 'source/',pipeline=test_pipeline),test=dict(type=dataset_type,#classes=('card',),ann_file=data_root + 'test_annfile/',img_prefix=data_root + 'source/',pipeline=test_pipeline))修改一下标签
ubuntu@ubuntu-Super-Server:~/sxj731533730/mmrotate$ sudo vim mmrotate/datasets/dota.pyclass DOTADataset(CustomDataset):"""DOTA dataset for detection.Args:ann_file (str): Annotation file path.pipeline (list[dict]): Processing pipeline.version (str, optional): Angle representations. Defaults to 'oc'.difficulty (bool, optional): The difficulty threshold of GT."""'''CLASSES = ('plane', 'baseball-diamond', 'bridge', 'ground-track-field','small-vehicle', 'large-vehicle', 'ship', 'tennis-court','basketball-court', 'storage-tank', 'soccer-ball-field','roundabout', 'harbor', 'swimming-pool', 'helicopter')'''CLASSES=('card',) # need to add ,修改配置文件
ubuntu@ubuntu-Super-Server:~/sxj731533730/mmrotate$ sudo vim configs/r3det/r3det_r50_fpn_1x_dota_oc.py
ubuntu@ubuntu-Super-Server:~/sxj731533730/mmrotate$ sudo vim configs/kfiou/r3det_kfiou_ln_r50_fpn_1x_dota_oc.pynum_classes=1 # 建立一个存储日志和模型的文件夹
ubuntu@ubuntu-Super-Server:~/sxj731533730/mmrotate$ mkdir run第五步：开始训练模型
ubuntu@ubuntu-Super-Server:~/sxj731533730/mmrotate$ python3 tools/train.py configs/kfiou/r3det_kfiou_ln_r50_fpn_1x_dota_oc.py --work-dir=run训练的过程
2022-03-22 06:21:11,014 - mmrotate - INFO - workflow: [('train', 1)], max: 12 epochs
2022-03-22 06:21:11,014 - mmrotate - INFO - Checkpoints will be saved to /home/ubuntu/sxj731533730/mmrotate/run by HardDiskBackend.
2022-03-22 06:21:35,543 - mmrotate - INFO - Epoch [1][50/330] lr: 9.967e-04, eta: 0:31:57, time: 0.491, data_time: 0.050, memory: 3442, s0.loss_cls: 1.1651, s0.loss_bbox: 6.7060, sr0.loss_cls: 1.1758, sr0.loss_bbox: 6.2110, loss: 15.2579, grad_norm: 8.4750
2022-03-22 06:21:57,744 - mmrotate - INFO - Epoch [1][100/330] lr: 1.163e-03, eta: 0:30:03, time: 0.444, data_time: 0.005, memory: 3442, s0.loss_cls: 0.9773, s0.loss_bbox: 6.1366, sr0.loss_cls: 0.4372, sr0.loss_bbox: 5.9701, loss: 13.5212, grad_norm: 26.8998
2022-03-22 06:22:19,983 - mmrotate - INFO - Epoch [1][150/330] lr: 1.330e-03, eta: 0:29:11, time: 0.445, data_time: 0.005, memory: 3442, s0.loss_cls: 0.3552, s0.loss_bbox: 5.9312, sr0.loss_cls: 0.2589, sr0.loss_bbox: 5.9402, loss: 12.4856, grad_norm: 18.0854
2022-03-22 06:22:42,092 - mmrotate - INFO - Epoch [1][200/330] lr: 1.497e-03, eta: 0:28:32, time: 0.442, data_time: 0.005, memory: 3442, s0.loss_cls: 0.2930, s0.loss_bbox: 5.8729, sr0.loss_cls: 0.2286, sr0.loss_bbox: 5.9325, loss: 12.3271, grad_norm: 15.2396
2022-03-22 06:23:04,421 - mmrotate - INFO - Epoch [1][250/330] lr: 1.663e-03, eta: 0:28:02, time: 0.447, data_time: 0.005, memory: 3442, s0.loss_cls: 0.2843, s0.loss_bbox: 5.8343, sr0.loss_cls: 0.2799, sr0.loss_bbox: 5.8648, loss: 12.2633, grad_norm: 12.3675
2022-03-22 06:23:26,901 - mmrotate - INFO - Epoch [1][300/330] lr: 1.830e-03, eta: 0:27:37, time: 0.450, data_time: 0.005, memory: 3442, s0.loss_cls: 0.2462, s0.loss_bbox: 5.7725, sr0.loss_cls: 0.0950, sr0.loss_bbox: 5.8509, loss: 11.9646, grad_norm: 9.6189
2022-03-22 06:24:04,773 - mmrotate - INFO - Epoch [2][50/330] lr: 2.097e-03, eta: 0:25:11, time: 0.491, data_time: 0.049, memory: 3442, s0.loss_cls: 0.1637, s0.loss_bbox: 5.6981, sr0.loss_cls: 0.0575, sr0.loss_bbox: 5.8003, loss: 11.7196, grad_norm: 6.5400
......
2022-03-23 01:48:42,354 - mmrotate - INFO -
+-------+-----+------+--------+-------+
| class | gts | dets | recall | ap |
+-------+-----+------+--------+-------+
| card | 50 | 53 | 1.000 | 1.000 |
+-------+-----+------+--------+-------+
| mAP | | | | 1.000 |
+-------+-----+------+--------+-------+
2022-03-23 01:48:42,385 - mmrotate - INFO - Exp name: r3det_kfiou_ln_r50_fpn_1x_dota_oc.py
2022-03-23 01:48:42,386 - mmrotate - INFO - Epoch(val) [12][28] mAP: 1.0000第l六步：测试一下 我只训练12次而已
ubuntu@ubuntu-Super-Server:~/sxj731533730/mmrotate$ python3 demo/image_demo.py kfiouDataSets/source/cap_output_00_03_30_30210.jpg configs/kfiou/r3det_kfiou_ln_r50_fpn_1x_dota_oc.py run/epoch_12.pth demo/sxj731533730.jpg将身份证生成分割图片写入本地显示 （还未训练好，继续训练中）
第七步：转mmrotate模型到onnx过程中遇到了涉及到mmcv函数，好难啊，官方暂时没提供转换脚本，只能抄袭mmdetection~
（1）先修改一下
mmrotate/models/detectors/r3det.pydef forward_dummy(self, img):
.....
return outs,rois #返回两个变量（2）补充一个函数，用于onnx的输出结果和测试
def simple_test_onnx(self, img_meta,outs,rois, rescale=False):"""Test function without test time augmentation.Args:imgs (list[torch.Tensor]): List of multiple imagesimg_metas (list[dict]): List of image information.rescale (bool, optional): Whether to rescale the results.Defaults to False.Returns:list[list[np.ndarray]]: BBox results of each image and classes. \The outer list corresponds to each image. The inner list \corresponds to each class."""bbox_inputs = outs + (img_meta, self.test_cfg, rescale)print(self.refine_head[-1])bbox_list = self.refine_head[-1].get_bboxes(*bbox_inputs, rois=rois)bbox_results = [rbbox2result(det_bboxes, det_labels,self.refine_head[-1].num_classes)for det_bboxes, det_labels in bbox_list]return bbox_results（3）修改一下mmcv的源码，否则无法转成onnx模型，读一下源代码，因为只做前向传播推理，可以简单处理一下，以转模型成功为基准
D:\Python39\Lib\site-packages\mmcv\ops\rotated_feature_align.py
或者linux 路径 ubuntu@ubuntu:~/.local/lib/python3.8/site-packages/mmcv/ops$ sudo vim rotated_feature_align.py原mmcv源代码
def rotated_feature_align(features,best_rbboxes,spatial_scale=1 / 8,points=1):return RotatedFeatureAlignFunction.apply(features, best_rbboxes,spatial_scale, points)mmcv修改后，记得转完模型，还原回去。。。别影响训练 实际把forward 函数的backward删掉了
def rotated_feature_align(features,best_rbboxes,spatial_scale=1 / 8,points=1):spatial_scale = spatial_scalepoints = pointsassert points in [1, 5]output = torch.zeros_like(features)ext_module.rotated_feature_align_forward(features,best_rbboxes,output,spatial_scale=spatial_scale,points=points)return output然后使用转换脚本进行convertonnx.py
import mmcv
import numpy as np
import torch
from mmcv.ops import RoIPool
from mmcv.parallel import collate, scatter
from mmcv.runner import load_checkpointfrom mmdet.core import get_classes
from mmdet.datasets import replace_ImageToTensor
from mmdet.datasets.pipelines import Compose
from mmdet.models import build_detector
from argparse import ArgumentParserfrom mmdet.apis import inference_detector, init_detector
import torch
# import trochvision
import torch.utils.data
import argparse
import onnxruntime
import os
import cv2
import numpy as np
from onnxruntime.datasets import get_exampleimport mmrotate # noqa: F401device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")def getConfig(imgs, model):if isinstance(imgs, (list, tuple)):is_batch = Trueelse:imgs = [imgs]is_batch = Falsecfg = model.cfgdevice = next(model.parameters()).device # model deviceif isinstance(imgs[0], np.ndarray):cfg = cfg.copy()# set loading pipeline typecfg.data.test.pipeline[0].type = 'LoadImageFromWebcam'cfg.data.test.pipeline = replace_ImageToTensor(cfg.data.test.pipeline)test_pipeline = Compose(cfg.data.test.pipeline)datas = []for img in imgs:# prepare dataif isinstance(img, np.ndarray):# directly add imgdata = dict(img=img)else:# add information into dictdata = dict(img_info=dict(filename=img), img_prefix=None)# build the data pipelinedata = test_pipeline(data)datas.append(data)data = collate(datas, samples_per_gpu=len(imgs))# just get the actual data from DataContainerdata['img_metas'] = [img_metas.data[0] for img_metas in data['img_metas']]img_metas = data['img_metas'][0]return img_metasdef torch2onnx(args, model):img_metas = getConfig(args.img, model)model.forward = model.forward_dummymodel.to(device)input_names = ["input"] # 模型输入的name# output_names = ["output","prob"] # 模型输出的namefrom torch.autograd import Variableinput = cv2.imread(args.img)(new_h,new_w)=(1024,1024) # 配置文件h, w = input.shape[:2]img_mmcv_scale_factor = max(new_h, new_w) / max(h, w)(new_h, new_w)=(img_mmcv_scale_factor*h,img_mmcv_scale_factor*w)h_scale = new_h / hw_scale = new_w / wscale_factor = np.array([w_scale, h_scale, w_scale, h_scale,1],dtype=np.float32)inputr = cv2.resize(input, (int(new_w), int(new_h))) # hwc bgrinput = cv2.cvtColor(inputr, cv2.COLOR_BGR2RGB) # hwc rgb# [0.485, 0.456, 0.406], [0.229, 0.224, 0.225])input = np.transpose(input, (2, 0, 1)).astype(np.float32) # chw rgb# input=input/255.0print("befor the input[0,0,0]:{}".format(input[0, 0, 0]))print("the size of input[0,...] is {}".format(input[0, ...].shape))print("the size of input[1,...] is {}".format(input[1, ...].shape))print("the size of input[2,...] is {}".format(input[2, ...].shape))input[0, ...] = (input[0, ...] - 123.675) / 58.395 # lainput[1, ...] = (input[1, ...] - 116.28) / 57.12input[2, ...] = (input[2, ...] - 103.53) / 57.375# 'mean': array([123.675, 116.28, 103.53], dtype=float32), 'std': array([58.395, 57.12, 57.375], dtype=float32)print("after input[0,0,0]:{}".format(input[0, 0, 0]))now_image1 = Variable(torch.from_numpy(input))dummy_input = now_image1.unsqueeze(0).to(device)# vutils.save_image(dummy_input, r'G:\mmrotate\mmrotate\models\detectors\dummy_input.jpg', normalize=True)print("====", dummy_input, type(dummy_input))torch.onnx.export(model, dummy_input, args.onnx_model_path, input_names=input_names,export_params=True,keep_initializers_as_inputs=True,do_constant_folding=True,verbose=False,opset_version=11)example_model = get_example(args.onnx_model_path)session = onnxruntime.InferenceSession(example_model)input_name = session.get_inputs()[0].nameresult = session.run([], {input_name: dummy_input.data.cpu().numpy()})print("outs=", result)a0 = torch.tensor(result[0]).to(device)a1 = torch.tensor(result[1]).to(device)a2 = torch.tensor(result[2]).to(device)a3 = torch.tensor(result[3]).to(device)a4 = torch.tensor(result[4]).to(device)a = [a0, a1, a2, a3, a4]b0 = torch.tensor(result[5]).to(device)b1 = torch.tensor(result[6]).to(device)b2 = torch.tensor(result[7]).to(device)b3 = torch.tensor(result[8]).to(device)b4 = torch.tensor(result[9]).to(device)b = [b0, b1, b2, b3, b4]outs = (a, b)c0 = torch.tensor(result[10]).to(device)b0_scale_factor = c0.new_tensor(scale_factor)b0_scale_factor= c0 / b0_scale_factorc0 = b0_scale_factorc1 = torch.tensor(result[11]).to(device)c2 = torch.tensor(result[12]).to(device)c3 = torch.tensor(result[13]).to(device)c4 = torch.tensor(result[14]).to(device)c = [c0, c1, c2, c3, c4]c = [c[idx] / item.new_tensor(scale_factor) for idx,item in enumerate(c)]rois = [c]result_ = model.simple_test_onnx(img_metas, outs, rois, False)model.show_result(args.img, result_[0], score_thr=args.score_thr, out_file=args.onnxoutput)def main():"""Test a single image."""parser = ArgumentParser()parser.add_argument('--img', default=r"F:\mmrotate\demo\demo.jpg",help='Image file')parser.add_argument('--config', default=r"F:\mmrotate\configs\kfiou\r3det_kfiou_ln_r50_fpn_1x_dota_oc.py",help='Config file')parser.add_argument('--checkpoint', default=r"F:\mmrotate\model\r3det_kfiou_ln_r50_fpn_1x_dota_oc-8e7f049d.pth",help='Checkpoint file')parser.add_argument('--onnx_model_path',default=r"F:\mmrotate\model\r3det_kfiou_ln_r50_fpn_1x_dota_oc-8e7f049d.onnx",help='onnx_model_path')parser.add_argument('--device', default='cuda:0', help='Device used for inference')parser.add_argument('--output', default=r'ptsxj731533730.jpg', help='Output image')parser.add_argument('--onnxoutput', default=r'onnxsxj731533730.jpg', help='Output image')parser.add_argument('--score-thr', type=float, default=0.3, help='bbox score threshold')args = parser.parse_args()# build the model from a config file and a checkpoint filemodel = init_detector(args.config, args.checkpoint, device=args.device)# test a single imageresult = inference_detector(model, args.img)print("result=", result)# show the resultsmodel.show_result(args.img, result, score_thr=args.score_thr, out_file=args.output)torch2onnx(args, model)#D:\Python39\Lib\site-packages\mmcv\opsif __name__ == '__main__':main()第八步、比对一下数据; pth 模型输出 ,存在差异，但是结果图是正确的~
onnx模型测试图
第九步、先转mnn模型，测试，
D:\MNN\buildMinGW>python -m onnxsim G:\mmrotate\model\r3det_kfiou_ln_r50_fpn_1x_dota_oc-8e7f049d.onnx G:\mmrotate\model\r3det_kfiou_ln_r50_fpn_1x_dota_oc-8e7f049d_sim.onnx
Simplifying...
Checking 0/3...
Checking 1/3...
Checking 2/3...
Ok!D:\MNN\buildMinGW>MNNConvert -f ONNX --modelFile G:\mmrotate\model\r3det_kfiou_ln_r50_fpn_1x_dota_oc-8e7f049d_sim.onnx --MNNModel G:\mmrotate\model\r3det_kfiou_ln_r50_fpn_1x_dota_oc-8e7f049d_sim.mnn --bizCode MNN
Start to Convert Other Model Format To MNN Model...
[11:38:18] D:\MNN\tools\converter\source\onnx\onnxConverter.cpp:30: ONNX Model ir version: 6
Start to Optimize the MNN Net...
inputTensors : [ input, ]
outputTensors: [ 1233, 1406, 1579, 1752, 1925, 1962, 1963, 1980, 1981, 1998, 1999, 2016, 2017, 2034, 2035, ]
Converted Success!先完成第一步，数据对上
py的onnx输出结果
测试代码 ，使用mnn的数据输出 只用前5个数据比对一下就好
#include <iostream>
#include <ostream>
#include <random>
#include <vector>
#include <opencv2/opencv.hpp>#include <iostream>
#include<opencv2/core.hpp>
#include<opencv2/imgproc.hpp>
#include<opencv2/highgui.hpp>
#include<MNN/Interpreter.hpp>
#include<MNN/ImageProcess.hpp>#define IMAGE_TARGET 1024int main(int argc, char **argv) {std::vector<std::vector<float>> vec_hm, vec_wh, vec_reg, vec_cls_theta, vec_hmax;cv::Mat img = cv::imread("G:\\mmrotate\\demo\\demo.jpg");int img_width = img.cols;int img_height = img.rows;std::vector<float> meanVals ={ - 123.675f*(1.0f / 58.395), -116.28f*(1.0f / 57.12) , -103.53f*(1.0f / 57.375)};;std::vector<float> normVals= { 1.0f / 58.395,1.0f / 57.12,1.0f / 57.375};cv::Mat img_color;cv::cvtColor(img.clone(),img_color,cv::COLOR_BGR2RGB);cv::Mat img_resized;cv::resize(img_color, img_resized, cv::Size(IMAGE_TARGET, IMAGE_TARGET));auto net = std::shared_ptr<MNN::Interpreter>(MNN::Interpreter::createFromFile("G:\\mmrotate\\model\\r3det_kfiou_ln_r50_fpn_1x_dota_oc-8e7f049d_sim.mnn"));//创建解释器std::cout << "Interpreter created" << std::endl;MNN::ScheduleConfig config;config.numThread = 8;config.type = MNN_FORWARD_CPU;auto session = net->createSession(config);//创建sessionstd::cout << "session created" << std::endl;auto inTensor = net->getSessionInput(session, NULL);auto outTensor = net->getSessionInput(session, NULL);auto _Tensor = MNN::Tensor::create<float>({1,3,IMAGE_TARGET,IMAGE_TARGET}, NULL, MNN::Tensor::CAFFE);if(_Tensor->elementSize()!=3*IMAGE_TARGET*IMAGE_TARGET){std::cout<<_Tensor->elementSize()<<" "<<img_resized.channels()*img_resized.cols*img_resized.rows<<std::endl;std::cout<<"input shape not equal image shape"<<std::endl;return -1;}std::vector<cv::Mat> rgbChannels(3);cv::split(img_resized, rgbChannels);for (auto i = 0; i < rgbChannels.size(); i++) {rgbChannels[i].convertTo(rgbChannels[i], CV_32FC1, normVals[i], meanVals[i]);for(int j=0;j<rgbChannels[i].rows;j++) {for (int k = 0; k < rgbChannels[i].cols; k++) {_Tensor->host<float>()[i*IMAGE_TARGET*IMAGE_TARGET+j*IMAGE_TARGET+k] =rgbChannels[i].at<float>(j, k);//printf("%f ",rgbChannels[i].at<float>(j, k)); //debug 比對圖片數據}}}inTensor->copyFromHostTensor(_Tensor);//推理net->runSession(session);std::cout<<std::endl;auto rois_net_a0= net->getSessionOutput(session, "1962");MNN::Tensor rois_tensor_a0(rois_net_a0, rois_net_a0->getDimensionType());rois_net_a0->copyToHostTensor(&rois_tensor_a0);auto rois_ptr_a0 = rois_tensor_a0.host<float>();std::cout<<"1962 = "<<rois_tensor_a0.height()<<" "<<rois_tensor_a0.width()<<" "<<rois_tensor_a0.channel()<<" "<<rois_tensor_a0.elementSize()<<std::endl;for (int i = 0; i < rois_tensor_a0.elementSize(); i++) {if(i==5) break;std::cout<<rois_ptr_a0[i]<<" ";}std::cout<<std::endl;auto rois_net_a1= net->getSessionOutput(session, "1980");MNN::Tensor rois_tensor_a1(rois_net_a1, rois_net_a1->getDimensionType());rois_net_a1->copyToHostTensor(&rois_tensor_a1);auto rois_ptr_a1 = rois_tensor_a1.host<float>();std::cout<<"1980 = "<<rois_tensor_a1.height()<<" "<<rois_tensor_a1.width()<<" "<<rois_tensor_a1.channel()<<" "<<rois_tensor_a1.elementSize()<<std::endl;for (int i = 0; i < rois_tensor_a1.elementSize(); i++) {if(i==5) break;std::cout<<rois_ptr_a1[i]<<" ";}std::cout<<std::endl;auto rois_net_a2= net->getSessionOutput(session, "1998");MNN::Tensor rois_tensor_a2(rois_net_a2, rois_net_a2->getDimensionType());rois_net_a2->copyToHostTensor(&rois_tensor_a2);auto rois_ptr_a2 = rois_tensor_a2.host<float>();std::cout<<"1998 = "<<rois_tensor_a2.height()<<" "<<rois_tensor_a2.width()<<" "<<rois_tensor_a2.channel()<<" "<<rois_tensor_a2.elementSize()<<std::endl;for (int i = 0; i < rois_tensor_a2.elementSize(); i++) {if(i==5) break;std::cout<<rois_ptr_a2[i]<<" ";}std::cout<<std::endl;auto rois_net_a3= net->getSessionOutput(session, "2016");MNN::Tensor rois_tensor_a3(rois_net_a3, rois_net_a3->getDimensionType());rois_net_a3->copyToHostTensor(&rois_tensor_a3);auto rois_ptr_a3 = rois_tensor_a3.host<float>();std::cout<<"2017 = "<<rois_tensor_a3.height()<<" "<<rois_tensor_a3.width()<<" "<<rois_tensor_a3.channel()<<" "<<rois_tensor_a3.elementSize()<<std::endl;for (int i = 0; i < rois_tensor_a3.elementSize(); i++) {if(i==5) break;std::cout<<rois_ptr_a3[i]<<" ";}std::cout<<std::endl;auto rois_net_a4= net->getSessionOutput(session, "2034");MNN::Tensor rois_tensor_a4(rois_net_a4, rois_net_a4->getDimensionType());rois_net_a4->copyToHostTensor(&rois_tensor_a4);auto rois_ptr_a4 = rois_tensor_a4.host<float>();std::cout<<"2034 = "<<rois_tensor_a4.height()<<" "<<rois_tensor_a4.width()<<" "<<rois_tensor_a4.channel()<<" "<<rois_tensor_a4.elementSize()<<std::endl;for (int i = 0; i < rois_tensor_a4.elementSize(); i++) {if(i==5) break;std::cout<<rois_ptr_a4[i]<<" ";}std::cout<<std::endl;auto rois_net_b0= net->getSessionOutput(session, "1963");MNN::Tensor rois_tensor_b0(rois_net_b0, rois_net_b0->getDimensionType());rois_net_b0->copyToHostTensor(&rois_tensor_b0);auto rois_ptr_b0 = rois_tensor_b0.host<float>();std::cout<<"1963 = "<<rois_tensor_b0.height()<<" "<<rois_tensor_b0.width()<<" "<<rois_tensor_b0.channel()<<" "<<rois_tensor_b0.elementSize()<<std::endl;for (int i = 0; i < rois_tensor_b0.elementSize(); i++) {if(i==5) break;std::cout<<rois_ptr_b0[i]<<" ";}std::cout<<std::endl;auto rois_net_b1= net->getSessionOutput(session, "1981");MNN::Tensor rois_tensor_b1(rois_net_b1, rois_net_b1->getDimensionType());rois_net_b1->copyToHostTensor(&rois_tensor_b1);auto rois_ptr_b1 = rois_tensor_b1.host<float>();std::cout<<"1981 = "<<rois_tensor_b1.height()<<" "<<rois_tensor_b1.width()<<" "<<rois_tensor_b1.channel()<<" "<<rois_tensor_b1.elementSize()<<std::endl;for (int i = 0; i < rois_tensor_b1.elementSize(); i++) {if(i==5) break;std::cout<<rois_ptr_b1[i]<<" ";}std::cout<<std::endl;auto rois_net_b2= net->getSessionOutput(session, "1999");MNN::Tensor rois_tensor_b2(rois_net_b2, rois_net_b2->getDimensionType());rois_net_b2->copyToHostTensor(&rois_tensor_b2);auto rois_ptr_b2 = rois_tensor_b2.host<float>();std::cout<<"1999 = "<<rois_tensor_b2.height()<<" "<<rois_tensor_b2.width()<<" "<<rois_tensor_b2.channel()<<" "<<rois_tensor_b2.elementSize()<<std::endl;for (int i = 0; i < rois_tensor_b2.elementSize(); i++) {if(i==5) break;std::cout<<rois_ptr_b2[i]<<" ";}std::cout<<std::endl;auto rois_net_b3= net->getSessionOutput(session, "2017");MNN::Tensor rois_tensor_b3(rois_net_b3, rois_net_b3->getDimensionType());rois_net_b3->copyToHostTensor(&rois_tensor_b3);auto rois_ptr_b3 = rois_tensor_b3.host<float>();std::cout<<"2017 = "<<rois_tensor_b3.height()<<" "<<rois_tensor_b3.width()<<" "<<rois_tensor_b3.channel()<<" "<<rois_tensor_b3.elementSize()<<std::endl;for (int i = 0; i < rois_tensor_b3.elementSize(); i++) {if(i==5) break;std::cout<<rois_ptr_b3[i]<<" ";}std::cout<<std::endl;auto rois_net_b4= net->getSessionOutput(session, "2035");MNN::Tensor rois_tensor_b4(rois_net_b4, rois_net_b4->getDimensionType());rois_net_b4->copyToHostTensor(&rois_tensor_b4);auto rois_ptr_b4 = rois_tensor_b4.host<float>();std::cout<<"2035 = "<<rois_tensor_b4.height()<<" "<<rois_tensor_b4.width()<<" "<<rois_tensor_b4.channel()<<" "<<rois_tensor_b4.elementSize()<<std::endl;for (int i = 0; i < rois_tensor_b4.elementSize(); i++) {if(i==5) break;std::cout<<rois_ptr_b4[i]<<" ";}std::cout<<std::endl;auto rois_net_c0= net->getSessionOutput(session, "1233");MNN::Tensor rois_tensor_c0(rois_net_c0, rois_net_c0->getDimensionType());rois_net_c0->copyToHostTensor(&rois_tensor_c0);auto rois_ptr_c0 = rois_tensor_c0.host<float>();std::cout<<"1233 = "<<rois_tensor_c0.height()<<" "<<rois_tensor_c0.width()<<" "<<rois_tensor_c0.channel()<<" "<<rois_tensor_c0.elementSize()<<std::endl;for (int i = 0; i < rois_tensor_c0.elementSize(); i++) {if(i==5) break;std::cout<<rois_ptr_c0[i]<<" ";}std::cout<<std::endl;auto rois_net_c1= net->getSessionOutput(session, "1406");MNN::Tensor rois_tensor_c1(rois_net_c1, rois_net_c1->getDimensionType());rois_net_c1->copyToHostTensor(&rois_tensor_c1);auto rois_ptr_c1 = rois_tensor_c1.host<float>();std::cout<<"1406 = "<<rois_tensor_c1.height()<<" "<<rois_tensor_c1.width()<<" "<<rois_tensor_c1.channel()<<" "<<rois_tensor_c1.elementSize()<<std::endl;for (int i = 0; i < rois_tensor_c1.elementSize(); i++) {if(i==5) break;std::cout<<rois_ptr_c1[i]<<" ";}std::cout<<std::endl;auto rois_net_c2= net->getSessionOutput(session, "1579");MNN::Tensor rois_tensor_c2(rois_net_c2, rois_net_c2->getDimensionType());rois_net_c2->copyToHostTensor(&rois_tensor_c2);auto rois_ptr_c2 = rois_tensor_c2.host<float>();std::cout<<"1579 = "<<rois_tensor_c2.height()<<" "<<rois_tensor_c2.width()<<" "<<rois_tensor_c2.channel()<<" "<<rois_tensor_c2.elementSize()<<std::endl;for (int i = 0; i < rois_tensor_c2.elementSize(); i++) {if(i==5) break;std::cout<<rois_ptr_c2[i]<<" ";}std::cout<<std::endl;auto rois_net_c3= net->getSessionOutput(session, "1752");MNN::Tensor rois_tensor_c3(rois_net_c3, rois_net_c3->getDimensionType());rois_net_c3->copyToHostTensor(&rois_tensor_c3);auto rois_ptr_c3 = rois_tensor_c3.host<float>();std::cout<<"1752 = "<<rois_tensor_c3.height()<<" "<<rois_tensor_c3.width()<<" "<<rois_tensor_c3.channel()<<" "<<rois_tensor_c3.elementSize()<<std::endl;for (int i = 0; i < rois_tensor_c3.elementSize(); i++) {if(i==5) break;std::cout<<rois_ptr_c3[i]<<" ";}std::cout<<std::endl;auto rois_net_c4= net->getSessionOutput(session, "1925");MNN::Tensor rois_tensor_c4(rois_net_c4, rois_net_c4->getDimensionType());rois_net_c4->copyToHostTensor(&rois_tensor_c4);auto rois_ptr_c4 = rois_tensor_c4.host<float>();std::cout<<"1925 = "<<rois_tensor_c4.height()<<" "<<rois_tensor_c4.width()<<" "<<rois_tensor_c4.channel()<<" "<<rois_tensor_c4.elementSize()<<std::endl;for (int i = 0; i < rois_tensor_c4.elementSize(); i++) {if(i==5) break;std::cout<<rois_ptr_c4[i]<<" ";}// 写的尽量不依赖MNN ，还要进行NCNN的转化//cv::imshow("image", img);// cv::imwrite("image.jpg", img);// cv::waitKey(0);return 0;
}测试结果
F:\untitled12\cmake-build-debug\untiled2.exe
Interpreter created
session created1962 = 128 128 15 245760
-4.39439 -5.24617 -5.18704 -5.12905 -4.93742
1980 = 64 64 15 61440
-4.21322 -4.90304 -4.8548 -4.88889 -4.98375
1998 = 32 32 15 15360
-4.14835 -4.77178 -4.73817 -4.63989 -4.54071
2017 = 16 16 15 3840
-4.25359 -5.01801 -5.07386 -5.17409 -5.20806
2034 = 8 8 15 960
-4.23796 -4.98971 -4.95354 -4.98515 -5.0024
1963 = 128 128 5 81920
-0.0302005 0.00219762 0.00354933 0.00171571 0.00321476
1981 = 64 64 5 20480
-0.0328103 0.0025497 -0.000417736 0.00785263 0.0128756
1999 = 32 32 5 5120
-0.0317866 0.00656122 0.0161173 0.0156595 0.0134422
2017 = 16 16 5 1280
-0.0194252 0.00759413 0.00278557 -0.00387253 -0.00116776
2035 = 8 8 5 320
-0.0312997 0.00519017 0.00391664 0.00136454 -0.000162137
1233 = 1 1 5 81920
-2.13289 -0.66669 21.7076 11.6278 -0.0438018
1406 = 1 1 5 20480
-0.9342 -1.13472 102.071 55.0583 -0.0763953
1579 = 1 1 5 5120
-6.1721 -4.72425 64.9247 79.673 0.716497
1752 = 1 1 5 1280
3.56645 -1.43359 217.386 93.2715 -0.111636
1925 = 1 1 5 320
-14.4982 -8.48447 415.77 186.24 -0.0219311
Process finished with exit code 0
第十步、完整的代码测试 模型
链接：https://pan.baidu.com/s/1lYkzuR85vLGixgtuhFceAA
提取码：u700
我使用的window11+mingw32+clion测试
cmakelists.txt
cmake_minimum_required(VERSION 3.10)
project(untiled2)set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -g -fopenmp -w")
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -g -fopenmp -w")
set(CMAKE_CXX_STANDARD 11)include_directories(${CMAKE_SOURCE_DIR}/include)
include_directories(${CMAKE_SOURCE_DIR}/include/MNN)set(OpenCV_DIR "D:\\Opencv440\\buildMinGW")#改为mingw-bulid的位置
set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} "${CMAKE_SOURCE_DIR}/cmake/")find_package(OpenCV REQUIRED)add_library(libmnn-static STATIC IMPORTED cmake-build-debug/main.cpp cmake-build-debug/main.h)
set_target_properties(libmnn-static PROPERTIES IMPORTED_LOCATION ${CMAKE_SOURCE_DIR}/lib/libMNN.a)add_executable(untiled2 polyiou.cpp main.cpp )target_link_libraries(untiled2 ${OpenCV_LIBS} libmnn-static)
测试目录
代码自己优化吧 其中polyio.h抄40、使用BBAVectors-Oriented-Object-Detection 进行旋转目标检测，并使用mnn和ncnn进行部署_sxj731533730的博客-CSDN博客
#include <iostream>
#include <ostream>
#include <random>
#include <vector>
#include <opencv2/opencv.hpp>#include<MNN/Interpreter.hpp>
#include<MNN/ImageProcess.hpp>
#include "polyiou.h"#define IMAGE_TARGET 1024struct Ploy {float p1_0;float p1_1;float p2_0;float p2_1;float p3_0;float p3_1;float p4_0;float p4_1;Ploy() : p1_0(0), p1_1(0), p2_0(0), p2_1(0), p3_0(0), p3_1(0), p4_0(0), p4_1(0) {}Ploy(float p1_0, float p1_1, float p2_0, float p2_1, float p3_0, float p3_1, float p4_0, float p4_1) : p1_0(p1_0),p1_1(p1_1),p2_0(p2_0),p2_1(p2_1),p3_0(p3_0),p3_1(p3_1),p4_0(p4_0),p4_1(p4_1) {}
};struct BBox {int idx;float conf;Ploy ploy;BBox() : idx(0), conf(0), ploy(0, 0, 0, 0, 0, 0, 0, 0) {}BBox(int idx, float conf, Ploy ploy) : idx(idx), conf(conf), ploy(ploy) {}
};double mysoftmax(double num) {return exp(num);
}double mysigmoid(double num) {return 1 / (1 + exp(-num));
}template<typename T>
void softmax(const typename ::std::vector<T> &v, typename ::std::vector<T> &s) {double sum = 0.0;transform(v.begin(), v.end(), s.begin(), mysoftmax);sum = accumulate(s.begin(), s.end(), sum);for (size_t i = 0; i < s.size(); ++i)s.at(i) /= sum;
}template<typename T>
void sigmoid(const typename ::std::vector<T> &v, typename ::std::vector<T> &s) {transform(v.begin(), v.end(), s.begin(), mysigmoid);}void cls_score_softmax(std::vector<float> &vec_cls_scores, std::vector<float> &item_cls_scores) {item_cls_scores = vec_cls_scores;softmax(vec_cls_scores, item_cls_scores);}void cls_score_sigmoid(std::vector<float> &vec_cls_scores, std::vector<float> &item_cls_scores) {item_cls_scores = vec_cls_scores;sigmoid(vec_cls_scores, item_cls_scores);}void dim_permute(std::vector<float> vec_cls_scores, std::vector<float> &item_cls_score, int w_h_dim) {for (int i = 0; i < w_h_dim; i++) {for (int j = i; j < vec_cls_scores.size(); j += w_h_dim) {item_cls_score.push_back(vec_cls_scores[j]);}}
}void process_top_k(std::vector<float> scores, int top_K, std::vector<float> &scores_K, std::vector<int> &index_K) {std::vector<int> idx(scores.size());std::iota(idx.begin(), idx.end(), 0);std::sort(idx.begin(), idx.end(),[&scores](int index_1, int index_2) { return scores[index_1] > scores[index_2]; });
// 获取K值int k_num = std::min<int>(scores.size(), top_K);int idx_j = 0;for (int j = 0; j < k_num; ++j) {idx_j = idx[j];index_K.push_back(idx_j);scores_K.push_back(scores[idx_j]);}}float norm_angle(float angle, std::string angle_range) {if (angle_range == "oc") {return angle;} else if (angle_range == "le135") {return fmod((angle + M_PI / 4), M_PI) - M_PI / 4;} else if (angle_range == "le90") {return fmod((angle + (M_PI) / 2), M_PI) - M_PI / 2;}std::cout << "Not yet implemented." << std::endl;return 0;
}void bbox_coder_decode(std::vector<float> top_k_anchors,std::vector<float> top_k_bbox_pred,std::vector<float> &bbox_coder,int bbox_preds_channels,int img_width,int img_height,float wh_ratio_clip = 16.0f / 1000.0f,bool add_ctr_clamp = false,int ctr_clamp = 32,bool proj_xy = false,std::string angle_range = "oc",bool edge_swap = false) {// G:\mmrotate\mmrotate\core\bbox\coder\delta_xywh_rbbox_coder.pyfloat dx_width;float dy_height;float dw;float dh;float gx;float gy;float ga;for (int i = 0; i < top_k_bbox_pred.size(); i += bbox_preds_channels) {float dx = top_k_bbox_pred[i];float dy = top_k_bbox_pred[i + 1];float dw = top_k_bbox_pred[i + 2];float dh = top_k_bbox_pred[i + 3];float da = top_k_bbox_pred[i + 4];float px = top_k_anchors[i];float py = top_k_anchors[i + 1];float pw = top_k_anchors[i + 2];float ph = top_k_anchors[i + 3];float pa = top_k_anchors[i + 4];float dx_width = pw * dx;float dy_height = ph * dy;float max_ratio = abs(log(wh_ratio_clip));if (add_ctr_clamp) {dx_width = (dx_width >= -ctr_clamp) ? dx_width : -ctr_clamp;dx_width = (dx_width <= ctr_clamp) ? dx_width : ctr_clamp;dy_height = (dy_height >= -ctr_clamp) ? dy_height : -ctr_clamp;dy_height = (dy_height <= ctr_clamp) ? dy_height : ctr_clamp;dw = (dw <= max_ratio) ? dw : max_ratio;dh = (dh <= max_ratio) ? dh : max_ratio;} else {dw = (dw <= max_ratio) ? dw : max_ratio;dw = (dw >= -max_ratio) ? dw : -max_ratio;dh = (dh <= max_ratio) ? dh : max_ratio;dh = (dh >= -max_ratio) ? dh : -max_ratio;}float gw = pw * exp(dw);float gh = ph * exp(dh);if (proj_xy) {gx = dx * pw * cos(pa) - dy * ph * sin(pa) + px;gy = dx * pw * sin(pa) + dy * ph * cos(pa) + py;} else {gx = px + dx_width;gy = py + dy_height;ga = norm_angle(pa + da, angle_range);}// if max_shape is not None:{gx = (gx >= 0) ? gx : 0;gx = (gx <= img_width - 1) ? gx : img_width - 1;gy = (gy >= 0) ? gy : 0;gy = (gy <= img_height - 1) ? gy : img_height - 1;}if (edge_swap) {float w_regular = gw > gh ? gw : gh;float h_regular = gw > gh ? gh : gw;float theta_regular = gw > gh ? ga : ga + M_PI / 2;theta_regular = norm_angle(theta_regular, angle_range);bbox_coder.push_back(gx);bbox_coder.push_back(gy);bbox_coder.push_back(gw);bbox_coder.push_back(gh);bbox_coder.push_back(ga);} else {bbox_coder.push_back(gx);bbox_coder.push_back(gy);bbox_coder.push_back(gw);bbox_coder.push_back(gh);bbox_coder.push_back(ga);}}}// 进行iou计算
float iou(Ploy &r1, Ploy &r2) {std::vector<double> p;p.push_back(r1.p1_0);p.push_back(r1.p1_1);p.push_back(r1.p2_0);p.push_back(r1.p2_1);p.push_back(r1.p3_0);p.push_back(r1.p3_1);p.push_back(r1.p4_0);p.push_back(r1.p4_1);std::vector<double> q;q.push_back(r2.p1_0);q.push_back(r2.p1_1);q.push_back(r2.p2_0);q.push_back(r2.p2_1);q.push_back(r2.p3_0);q.push_back(r2.p3_1);q.push_back(r2.p4_0);q.push_back(r2.p4_1);double iou = iou_poly(p, q);p.clear();std::vector<double>().swap(p);q.clear();std::vector<double>().swap(q);return iou;}// 进行nms计算
void single_class_non_max_suppression(std::vector<Ploy> ploys, std::vector<float> confs, std::vector<Ploy> &ans,std::vector<int> &keep_idx, float conf_thresh, float iou_thresh) {if (ploys.size() == 0) {return;}std::vector<BBox> bboxes;BBox bbox;for (int i = 0; i < (int) ploys.size(); ++i) {bboxes.push_back(BBox(i, confs[i], ploys[i]));}// 对bbox的conf进行降序排序sort(bboxes.begin(), bboxes.end(), [&](const BBox &a, const BBox &b) {return a.conf > b.conf;});while (!bboxes.empty()) {bbox = bboxes[0];if (bbox.conf < conf_thresh) {break;}keep_idx.emplace_back(bbox.idx);bboxes.erase(bboxes.begin());// 让conf最高的bbox与其他剩余的bbox进行iou计算int size = bboxes.size();for (int i = 0; i < size; ++i) {float iou_ans = iou(bbox.ploy, bboxes[i].ploy);if (iou_ans > iou_thresh) {bboxes.erase(bboxes.begin() + i);size = bboxes.size();i = i - 1;}}}for (const int number : keep_idx) {ans.push_back(ploys[number]);}}void convet_point(std::vector<float> mutil_point,float &p1_0,float &p1_1,float &p2_0,float &p2_1,float &p3_0,float &p3_1,float &p4_0,float &p4_1) {float xc = mutil_point[0];float yc = mutil_point[1];float w = mutil_point[2];float h = mutil_point[3];float ag = mutil_point[4];float wx = w / 2 * cos(ag);float wy = w / 2 * sin(ag);float hx = -h / 2 * sin(ag);float hy = h / 2 * cos(ag);p1_0 = xc - wx - hx;p1_1 = yc - wy - hy;p2_0 = xc + wx - hx;p2_1 = yc + wy - hy;p3_0 = xc + wx + hx;p3_1 = yc + wy + hy;p4_0 = xc - wx + hx;p4_1 = yc - wy + hy;
}void ext_module_nms_rotated(std::vector<std::vector<float>> dim_mlvl_bbox,std::vector<float> dim_mlvl_score,std::vector<int> order_K,std::vector<int> &keep_idx,float nms_iou_thr,bool multi_label = false) {// nms.pystd::cout << dim_mlvl_bbox.size() << " " << " " << dim_mlvl_score.size() << std::endl;std::vector<Ploy> pts0_item;std::vector<Ploy> ans;for (int i = 0; i < dim_mlvl_bbox.size(); i++) {float p1_0 = 0, p1_1 = 0, p2_0 = 0, p2_1 = 0, p3_0 = 0, p3_1 = 0, p4_0 = 0, p4_1 = 0;convet_point(dim_mlvl_bbox[i], p1_0, p1_1, p2_0, p2_1, p3_0, p3_1, p4_0, p4_1);pts0_item.push_back(Ploy(p1_0, p1_1, p2_0, p2_1, p3_0, p3_1, p4_0, p4_1));}single_class_non_max_suppression(pts0_item, dim_mlvl_score, ans, keep_idx, 0.3, nms_iou_thr);pts0_item.clear();std::vector<Ploy>().swap(pts0_item);ans.clear();std::vector<Ploy>().swap(ans);}void nms_rotated(std::vector<std::vector<float>> dim_mlvl_bbox,std::vector<float> dim_mlvl_score,std::vector<int> &keep_inds,float nms_iou_thr,bool clockwise = true) {if (dim_mlvl_bbox[0].size() == 0) {return;}std::vector<int> order_K;std::vector<float> scores_K;process_top_k(dim_mlvl_score, dim_mlvl_score.size(), scores_K, order_K);ext_module_nms_rotated(dim_mlvl_bbox, dim_mlvl_score, order_K, keep_inds, nms_iou_thr);
}void multiclass_nms_rotated(std::vector<std::vector<float>> multi_bbox,std::vector<std::vector<float>> mlvl_score,std::vector<int> mlvl_data_shape,std::vector<std::vector<float>> &keep_box,std::vector<float> &keep_score,std::vector<int> &keep_label,float score_thr = 0.05,float nms_iou_thr = 0.1,int max_num = 2000,bool score_factors = false,bool return_inds = false) {std::vector<std::vector<float>> dim_mlvl_bbox;std::vector<float> temp_dim_mlvl_bbox;std::vector<std::vector<float>> dets_;std::vector<float> dim_mlvl_score;std::vector<int> dim_mlvl_label;std::vector<int> dim_mlvl_inds;float max_coordinate = -1;int num_classes = mlvl_score[0].size() - 1;if (multi_bbox[0].size() > 5) { ;} else { ;}for (int i = 0; i < mlvl_score.size(); i++) {int length = mlvl_score[i].size() - 1;for (int j = 0; j < length; j++) {if (mlvl_score[i][j] > score_thr) {dim_mlvl_score.push_back(mlvl_score[i][j]);dim_mlvl_inds.push_back(i * length + j);dim_mlvl_label.push_back(j);auto item = multi_bbox[(i * length + j) / num_classes];auto maxPosition = max_element(item.begin(), item.end());max_coordinate = std::max(max_coordinate, item[maxPosition - item.begin()]);dim_mlvl_bbox.push_back(item);}}}if (dim_mlvl_bbox.size() == 0) {for (int i = 0; i < dim_mlvl_bbox.size(); i++) {for (int j = 0; j < dim_mlvl_bbox[i].size(); j++) {temp_dim_mlvl_bbox.push_back(dim_mlvl_bbox[i][j]);}temp_dim_mlvl_bbox.push_back(dim_mlvl_score[i]);dets_.push_back(temp_dim_mlvl_bbox);temp_dim_mlvl_bbox.clear();std::vector<float>().swap(temp_dim_mlvl_bbox);}if (return_inds) {return; //dets, labels, inds} else {return;//dets, labels, inds}}std::vector<int> keep;nms_rotated(dim_mlvl_bbox, dim_mlvl_score, keep, nms_iou_thr);std::cout << "kip_dim= " << keep.size() << std::endl;if (max_num > 0) {if (keep.size() > max_num) {keep.resize(max_num);}}for (int i = 0; i < keep.size(); i++) {keep_box.push_back(dim_mlvl_bbox[keep[i]]);keep_score.push_back(dim_mlvl_score[keep[i]]);keep_label.push_back(dim_mlvl_label[keep[i]]);}std::cout << keep_box.size() << " " << keep_score.size() << " " << keep_label.size() << std::endl;dim_mlvl_bbox.clear();std::vector<std::vector<float>>().swap(dim_mlvl_bbox);temp_dim_mlvl_bbox.clear();std::vector<float>().swap(temp_dim_mlvl_bbox);dets_.clear();std::vector<std::vector<float>>().swap(dets_);dim_mlvl_score.clear();std::vector<float>().swap(dim_mlvl_score);dim_mlvl_label.clear();std::vector<int>().swap(dim_mlvl_label);dim_mlvl_inds.clear();std::vector<int>().swap(dim_mlvl_inds);}void get_bboxes_single(std::vector<std::vector<float>> vec_cls_scores,std::vector<std::vector<float>> vec_bbox_preds,std::vector<std::vector<float>> vec_rois,std::vector<int> scores_shape_0,std::vector<std::vector<float>> &keep_box,std::vector<float> &keep_score,std::vector<int> &keep_label,int img_width,int img_height,int nms_pre,int cls_out_channels,int bbox_preds_channels,int rois_channels,bool rescale,bool with_nms,bool use_sigmoid_cls,float scale_factor[],float score_thr,float iou_thr,int max_pre_img) {std::vector<std::vector<float>> mlvl_bboxes;std::vector<std::vector<float>> mlvl_scores;std::vector<float> item_cls_score;std::vector<float> item_bbox_pred;std::vector<float> item_s_cls_score;std::vector<int> topk_inds;std::vector<float> scores_K;std::vector<float> max_scores;std::vector<float> top_k_anchors;std::vector<float> top_k_bbox_pred;std::vector<float> top_k_scores;std::vector<float> bbox_coder;std::vector<std::vector<float>> mlvl_bbox;std::vector<std::vector<float>> mlvl_score;std::vector<int> mlvl_data_shape;std::vector<float> item_temp;for (int i = 0; i < vec_cls_scores.size(); i++) {dim_permute(vec_cls_scores[i], item_cls_score, vec_cls_scores[i].size() / cls_out_channels); //1,2,0dim_permute(vec_bbox_preds[i], item_bbox_pred, vec_bbox_preds[i].size() / bbox_preds_channels); //1,2,0//item_cls_score 16384 15 仍然拉伸成1维数据// item_bbox_pred 5 仍然拉伸成1维数据if (use_sigmoid_cls) {cls_score_sigmoid(item_cls_score, item_s_cls_score);} else {cls_score_softmax(item_cls_score, item_s_cls_score);}if (nms_pre > 0 && scores_shape_0[i] > nms_pre) {if (use_sigmoid_cls) {for (int j = 0; j < item_s_cls_score.size() / cls_out_channels; j++) {auto maxPosition = max_element(item_s_cls_score.begin() + j * cls_out_channels,item_s_cls_score.begin() + (j + 1) * cls_out_channels);max_scores.push_back(item_s_cls_score[maxPosition - item_s_cls_score.begin()]);}process_top_k(max_scores, nms_pre, scores_K, topk_inds);for (int m = 0; m < topk_inds.size(); m++) {int index = topk_inds[m];for (int n = 0; n < rois_channels; n++) {top_k_anchors.push_back(vec_rois[i][index * rois_channels + n]);}for (int n = 0; n < bbox_preds_channels; n++) {top_k_bbox_pred.push_back(item_bbox_pred[index * bbox_preds_channels + n]);}for (int n = 0; n < cls_out_channels; n++) {top_k_scores.push_back(item_s_cls_score[index * cls_out_channels + n]);}}topk_inds.clear();std::vector<int>().swap(topk_inds);scores_K.clear();std::vector<float>().swap(scores_K);max_scores.clear();std::vector<float>().swap(max_scores);}mlvl_data_shape.push_back(nms_pre);} else {top_k_anchors = vec_rois[i];top_k_bbox_pred = item_bbox_pred;top_k_scores = item_s_cls_score;mlvl_data_shape.push_back(scores_shape_0[i]);}bbox_coder_decode(top_k_anchors, top_k_bbox_pred, bbox_coder, bbox_preds_channels, img_width, img_height);mlvl_bbox.push_back(bbox_coder);mlvl_score.push_back(top_k_scores);top_k_anchors.clear();std::vector<float>().swap(top_k_anchors);bbox_coder.clear();std::vector<float>().swap(bbox_coder);top_k_scores.clear();std::vector<float>().swap(top_k_scores);top_k_bbox_pred.clear();std::vector<float>().swap(top_k_bbox_pred);item_cls_score.clear();std::vector<float>().swap(item_cls_score);item_bbox_pred.clear();std::vector<float>().swap(item_bbox_pred);item_s_cls_score.clear();std::vector<float>().swap(item_s_cls_score);}mlvl_bboxes = mlvl_bbox;mlvl_scores = mlvl_score;if (rescale) {mlvl_bboxes.clear();std::vector<std::vector<float>>().swap(mlvl_bboxes);// angle should not be rescaledfor (int i = 0; i < mlvl_bbox.size(); i++) {for (int j = 0; j < mlvl_bbox[i].size(); j += bbox_preds_channels) {for (int k = 0; k < bbox_preds_channels - 1; k++) {item_temp.push_back(mlvl_bbox[i][j + k] / scale_factor[k]);}item_temp.push_back(mlvl_bbox[i][j + bbox_preds_channels - 1]);}mlvl_bboxes.push_back(item_temp);item_temp.clear();std::vector<float>().swap(item_temp);}}if (use_sigmoid_cls) {mlvl_scores.clear();std::vector<std::vector<float>>().swap(mlvl_scores);for (int i = 0; i < mlvl_score.size(); i++) {for (int j = 0; j < mlvl_score[i].size(); j += cls_out_channels) {for (int k = 0; k < cls_out_channels; k++) {item_temp.push_back(mlvl_score[i][j + k]);};item_temp.push_back(0);}mlvl_scores.push_back(item_temp);item_temp.clear();std::vector<float>().swap(item_temp);}cls_out_channels = cls_out_channels + 1;}mlvl_bbox.clear();std::vector<std::vector<float>>().swap(mlvl_bbox);//cat datafor (int i = 0; i < mlvl_bboxes.size(); i++) {for (int j = 0; j < mlvl_bboxes[i].size(); j += bbox_preds_channels) {for (int k = 0; k < bbox_preds_channels; k++) {item_temp.push_back(mlvl_bboxes[i][j + k]);}mlvl_bbox.push_back(item_temp);item_temp.clear();std::vector<float>().swap(item_temp);}}mlvl_score.clear();std::vector<std::vector<float>>().swap(mlvl_score);for (int i = 0; i < mlvl_scores.size(); i++) {for (int j = 0; j < mlvl_scores[i].size(); j += cls_out_channels) { // 扩展了0维空间for (int k = 0; k < cls_out_channels; k++) {item_temp.push_back(mlvl_scores[i][j + k]);}mlvl_score.push_back(item_temp);item_temp.clear();std::vector<float>().swap(item_temp);}}if (with_nms) {multiclass_nms_rotated(mlvl_bbox,mlvl_score,mlvl_data_shape,keep_box,keep_score,keep_label,score_thr,iou_thr,max_pre_img);} else {// 引用已经回传数据}item_cls_score.clear(); //有点习惯了 释放，其实局部变量没有必要std::vector<float>().swap(item_cls_score);item_bbox_pred.clear();std::vector<float>().swap(item_bbox_pred);item_s_cls_score.clear();std::vector<float>().swap(item_s_cls_score);topk_inds.clear();std::vector<int>().swap(topk_inds);scores_K.clear();std::vector<float>().swap(scores_K);max_scores.clear();std::vector<float>().swap(max_scores);top_k_anchors.clear();std::vector<float>().swap(top_k_anchors);top_k_bbox_pred.clear();std::vector<float>().swap(top_k_bbox_pred);top_k_scores.clear();std::vector<float>().swap(top_k_scores);bbox_coder.clear();std::vector<float>().swap(bbox_coder);mlvl_bbox.clear();std::vector<std::vector<float>>().swap(mlvl_bbox);mlvl_score.clear();std::vector<std::vector<float>>().swap(mlvl_score);mlvl_data_shape.clear();std::vector<int>().swap(mlvl_data_shape);mlvl_bboxes.clear();std::vector<std::vector<float>>().swap(mlvl_bboxes);mlvl_scores.clear();std::vector<std::vector<float>>().swap(mlvl_scores);
}int main(int argc, char **argv) {cv::Mat img = cv::imread("F:\\mmrotate\\demo\\demo.jpg");const char *classes[] = {"plane", "baseball-diamond", "bridge", "ground-track-field","small-vehicle", "large-vehicle", "ship", "tennis-court","basketball-court", "storage-tank", "soccer-ball-field","roundabout", "harbor", "swimming-pool", "helicopter" };std::vector<float> meanVals = {-123.675f * (1.0f / 58.395), -116.28f * (1.0f / 57.12), -103.53f * (1.0f / 57.375)};;std::vector<float> normVals = {1.0f / 58.395, 1.0f / 57.12, 1.0f / 57.375};cv::Mat img_color;cv::cvtColor(img.clone(), img_color, cv::COLOR_BGR2RGB);int new_h=1024.0;int new_w=1024.0;int img_width = img.cols;int img_height = img.rows;float img_mmcv_scale_factor = std::max(new_w*1.0f, new_h*1.0f) / std::max(img_height*1.0f, img_width*1.0f);new_h=img_mmcv_scale_factor*img_height;new_w=img_mmcv_scale_factor*img_width;float h_scale = new_h / (img_height*1.0f);float w_scale = new_w / (img_width*1.0f);float scale_factor[] = {w_scale, h_scale, w_scale, h_scale};cv::Mat img_resized;cv::resize(img_color, img_resized, cv::Size(new_w, new_h));auto net = std::shared_ptr<MNN::Interpreter>(MNN::Interpreter::createFromFile("F:\\mmrotate\\model\\r3det_kfiou_ln_r50_fpn_1x_dota_oc-8e7f049d_sim.mnn"));//创建解释器std::cout << "Interpreter created" << std::endl;MNN::ScheduleConfig config;config.numThread = 8;config.type = MNN_FORWARD_CPU;auto session = net->createSession(config);//创建sessionstd::cout << "session created" << std::endl;auto inTensor = net->getSessionInput(session, NULL);auto outTensor = net->getSessionInput(session, NULL);auto _Tensor = MNN::Tensor::create<float>({1, 3, new_w, new_h}, NULL, MNN::Tensor::CAFFE);if (_Tensor->elementSize() != 3 * new_w * new_h) {std::cout << _Tensor->elementSize() << " " << img_resized.channels() * img_resized.cols * img_resized.rows<< std::endl;std::cout << "input shape not equal image shape" << std::endl;return -1;}std::vector<cv::Mat> rgbChannels(3);cv::split(img_resized, rgbChannels);for (auto i = 0; i < rgbChannels.size(); i++) {rgbChannels[i].convertTo(rgbChannels[i], CV_32FC1, normVals[i], meanVals[i]);for (int j = 0; j < rgbChannels[i].rows; j++) {for (int k = 0; k < rgbChannels[i].cols; k++) {_Tensor->host<float>()[i * new_w * new_h + j * new_w +k] = rgbChannels[i].at<float>(j, k);//printf("%f ",rgbChannels[i].at<float>(j, k)); //debug 比對圖片數據}}}inTensor->copyFromHostTensor(_Tensor);//推理net->runSession(session);char* cls_id0="1962";char* cls_id1="1980";char* cls_id2="1998";char* cls_id3="2016";char* cls_id4="2034";char* bbox_id0="1963";char* bbox_id1="1981";char *bbox_id2="1999";char *bbox_id3="2017";char *bbox_id4="2035";char* rois_id0="1233";char* rois_id1="1406";char *rois_id2="1579";char *rois_id3="1752";char *rois_id4="1925";/*char* cls_id0="1967";//"1962";char* cls_id1="1985";//"1980";char* cls_id2="2003";//"1998";char* cls_id3="2021";//"2016";char* cls_id4="2039";//"2034";char* bbox_id0="1968";//"1963";char* bbox_id1="1986";//"1981";char *bbox_id2="2004";//"1999";char *bbox_id3="2022";//"2017";char *bbox_id4="2040";//"2035";char* rois_id0="1238";//"1233";char* rois_id1="1411";//"1406";char *rois_id2="1584";//"1579";char *rois_id3="1757";//"1752";char *rois_id4="1930";//"1925";
*/std::vector<std::vector<float>> vec_cls_scores;std::vector<int> scores_shape_0;std::vector<float> vec_temp;int cls_out_channels = 0;std::cout << std::endl;auto cls_scores_a0 = net->getSessionOutput(session, cls_id0);MNN::Tensor cls_scores_tensor_a0(cls_scores_a0, cls_scores_a0->getDimensionType());cls_scores_a0->copyToHostTensor(&cls_scores_tensor_a0);auto cls_scores_ptr_a0 = cls_scores_tensor_a0.host<float>();std::cout << std::string(cls_id0) << cls_scores_tensor_a0.height() << " " << cls_scores_tensor_a0.width() << " "<< cls_scores_tensor_a0.channel() << " " << cls_scores_tensor_a0.elementSize() << std::endl;cls_out_channels = cls_scores_tensor_a0.channel();for (int i = 0; i < cls_scores_tensor_a0.elementSize(); i++) {vec_temp.push_back(cls_scores_ptr_a0[i]);}scores_shape_0.push_back(cls_scores_tensor_a0.height() * cls_scores_tensor_a0.width());vec_cls_scores.push_back(vec_temp);vec_temp.clear();std::vector<float>().swap(vec_temp);std::cout << std::endl;auto cls_scores_a1 = net->getSessionOutput(session, cls_id1);MNN::Tensor cls_scores_tensor_a1(cls_scores_a1, cls_scores_a1->getDimensionType());cls_scores_a1->copyToHostTensor(&cls_scores_tensor_a1);auto cls_scores_ptr_a1 = cls_scores_tensor_a1.host<float>();std::cout << std::string(cls_id1) << cls_scores_tensor_a1.height() << " " << cls_scores_tensor_a1.width() << " "<< cls_scores_tensor_a1.channel() << " " << cls_scores_tensor_a1.elementSize() << std::endl;for (int i = 0; i < cls_scores_tensor_a1.elementSize(); i++) {vec_temp.push_back(cls_scores_ptr_a1[i]);}scores_shape_0.push_back(cls_scores_tensor_a1.height() * cls_scores_tensor_a1.width());vec_cls_scores.push_back(vec_temp);vec_temp.clear();std::vector<float>().swap(vec_temp);std::cout << std::endl;auto cls_scores_net_a2 = net->getSessionOutput(session, cls_id2);MNN::Tensor cls_scores_tensor_a2(cls_scores_net_a2, cls_scores_net_a2->getDimensionType());cls_scores_net_a2->copyToHostTensor(&cls_scores_tensor_a2);auto cls_scores_ptr_a2 = cls_scores_tensor_a2.host<float>();std::cout << std::string(cls_id2) << cls_scores_tensor_a2.height() << " " << cls_scores_tensor_a2.width() << " "<< cls_scores_tensor_a2.channel() << " " << cls_scores_tensor_a2.elementSize() << std::endl;for (int i = 0; i < cls_scores_tensor_a2.elementSize(); i++) {vec_temp.push_back(cls_scores_ptr_a2[i]);}scores_shape_0.push_back(cls_scores_tensor_a2.height() * cls_scores_tensor_a2.width());vec_cls_scores.push_back(vec_temp);vec_temp.clear();std::vector<float>().swap(vec_temp);std::cout << std::endl;auto cls_scores_net_a3 = net->getSessionOutput(session, cls_id3);MNN::Tensor cls_scores_tensor_a3(cls_scores_net_a3, cls_scores_net_a3->getDimensionType());cls_scores_net_a3->copyToHostTensor(&cls_scores_tensor_a3);auto rois_ptr_a3 = cls_scores_tensor_a3.host<float>();std::cout << std::string(cls_id3)<< cls_scores_tensor_a3.height() << " " << cls_scores_tensor_a3.width() << " "<< cls_scores_tensor_a3.channel() << " " << cls_scores_tensor_a3.elementSize() << std::endl;for (int i = 0; i < cls_scores_tensor_a3.elementSize(); i++) {vec_temp.push_back(rois_ptr_a3[i]);}scores_shape_0.push_back(cls_scores_tensor_a3.height() * cls_scores_tensor_a3.width());vec_cls_scores.push_back(vec_temp);vec_temp.clear();std::vector<float>().swap(vec_temp);std::cout << std::endl;auto cls_scores_net_a4 = net->getSessionOutput(session, cls_id4);MNN::Tensor cls_scores_tensor_a4(cls_scores_net_a4, cls_scores_net_a4->getDimensionType());cls_scores_net_a4->copyToHostTensor(&cls_scores_tensor_a4);auto cls_scores_ptr_a4 = cls_scores_tensor_a4.host<float>();std::cout << std::string(cls_id4) << cls_scores_tensor_a4.height() << " " << cls_scores_tensor_a4.width() << " "<< cls_scores_tensor_a4.channel() << " " << cls_scores_tensor_a4.elementSize() << std::endl;for (int i = 0; i < cls_scores_tensor_a4.elementSize(); i++) {vec_temp.push_back(cls_scores_ptr_a4[i]);}scores_shape_0.push_back(cls_scores_tensor_a4.height() * cls_scores_tensor_a4.width());vec_cls_scores.push_back(vec_temp);vec_temp.clear();std::vector<float>().swap(vec_temp);std::vector<std::vector<float>> vec_bbox_preds;int bbox_preds_channels = 0;std::cout << std::endl;auto bbox_preds_net_b0 = net->getSessionOutput(session, bbox_id0);MNN::Tensor bbox_preds_tensor_b0(bbox_preds_net_b0, bbox_preds_net_b0->getDimensionType());bbox_preds_net_b0->copyToHostTensor(&bbox_preds_tensor_b0);auto rois_ptr_b0 = bbox_preds_tensor_b0.host<float>();std::cout << std::string(bbox_id0) << bbox_preds_tensor_b0.height() << " " << bbox_preds_tensor_b0.width() << " "<< bbox_preds_tensor_b0.channel() << " " << bbox_preds_tensor_b0.elementSize() << std::endl;bbox_preds_channels = bbox_preds_tensor_b0.channel();for (int i = 0; i < bbox_preds_tensor_b0.elementSize(); i++) {vec_temp.push_back(rois_ptr_b0[i]);}vec_bbox_preds.push_back(vec_temp);vec_temp.clear();std::vector<float>().swap(vec_temp);std::cout << std::endl;auto vec_bbox_net_b1 = net->getSessionOutput(session, bbox_id1);MNN::Tensor vec_bbox_tensor_b1(vec_bbox_net_b1, vec_bbox_net_b1->getDimensionType());vec_bbox_net_b1->copyToHostTensor(&vec_bbox_tensor_b1);auto vec_bbox_ptr_b1 = vec_bbox_tensor_b1.host<float>();std::cout << std::string(bbox_id1) << vec_bbox_tensor_b1.height() << " " << vec_bbox_tensor_b1.width() << " "<< vec_bbox_tensor_b1.channel() << " " << vec_bbox_tensor_b1.elementSize() << std::endl;for (int i = 0; i < vec_bbox_tensor_b1.elementSize(); i++) {vec_temp.push_back(vec_bbox_ptr_b1[i]);}vec_bbox_preds.push_back(vec_temp);vec_temp.clear();std::vector<float>().swap(vec_temp);std::cout << std::endl;auto vec_bbox_net_b2 = net->getSessionOutput(session, bbox_id2);MNN::Tensor vec_bbox_tensor_b2(vec_bbox_net_b2, vec_bbox_net_b2->getDimensionType());vec_bbox_net_b2->copyToHostTensor(&vec_bbox_tensor_b2);auto rois_ptr_b2 = vec_bbox_tensor_b2.host<float>();std::cout << std::string(bbox_id2) << vec_bbox_tensor_b2.height() << " " << vec_bbox_tensor_b2.width() << " "<< vec_bbox_tensor_b2.channel() << " " << vec_bbox_tensor_b2.elementSize() << std::endl;for (int i = 0; i < vec_bbox_tensor_b2.elementSize(); i++) {vec_temp.push_back(rois_ptr_b2[i]);}vec_bbox_preds.push_back(vec_temp);vec_temp.clear();std::vector<float>().swap(vec_temp);std::cout << std::endl;auto bbox_preds_net_b3 = net->getSessionOutput(session, bbox_id3);MNN::Tensor bbox_preds_tensor_b3(bbox_preds_net_b3, bbox_preds_net_b3->getDimensionType());bbox_preds_net_b3->copyToHostTensor(&bbox_preds_tensor_b3);auto bbox_preds_ptr_b3 = bbox_preds_tensor_b3.host<float>();std::cout << std::string(bbox_id3) << bbox_preds_tensor_b3.height() << " " << bbox_preds_tensor_b3.width() << " "<< bbox_preds_tensor_b3.channel() << " " << bbox_preds_tensor_b3.elementSize() << std::endl;for (int i = 0; i < bbox_preds_tensor_b3.elementSize(); i++) {vec_temp.push_back(bbox_preds_ptr_b3[i]);}vec_bbox_preds.push_back(vec_temp);vec_temp.clear();std::vector<float>().swap(vec_temp);std::cout << std::endl;auto bbox_preds_net_b4 = net->getSessionOutput(session, bbox_id4);MNN::Tensor bbox_preds_tensor_b4(bbox_preds_net_b4, bbox_preds_net_b4->getDimensionType());bbox_preds_net_b4->copyToHostTensor(&bbox_preds_tensor_b4);auto bbox_preds_ptr_b4 = bbox_preds_tensor_b4.host<float>();std::cout << std::string(bbox_id4) << bbox_preds_tensor_b4.height() << " " << bbox_preds_tensor_b4.width() << " "<< bbox_preds_tensor_b4.channel() << " " << bbox_preds_tensor_b4.elementSize() << std::endl;for (int i = 0; i < bbox_preds_tensor_b4.elementSize(); i++) {vec_temp.push_back(bbox_preds_ptr_b4[i]);}vec_bbox_preds.push_back(vec_temp);vec_temp.clear();std::vector<float>().swap(vec_temp);std::vector<std::vector<float>> vec_rois;int rois_channels = 0;std::cout << std::endl;auto rois_net_c0 = net->getSessionOutput(session, rois_id0);MNN::Tensor rois_tensor_c0(rois_net_c0, rois_net_c0->getDimensionType());rois_net_c0->copyToHostTensor(&rois_tensor_c0);auto rois_ptr_c0 = rois_tensor_c0.host<float>();std::cout << std::string(rois_id0) << rois_tensor_c0.height() << " " << rois_tensor_c0.width() << " "<< rois_tensor_c0.channel() << " " << rois_tensor_c0.elementSize() << std::endl;rois_channels = rois_tensor_c0.channel();for (int i = 0; i < rois_tensor_c0.elementSize(); i++) {vec_temp.push_back(rois_ptr_c0[i]);}vec_rois.push_back(vec_temp);vec_temp.clear();std::vector<float>().swap(vec_temp);std::cout << std::endl;auto rois_net_c1 = net->getSessionOutput(session, rois_id1);MNN::Tensor rois_tensor_c1(rois_net_c1, rois_net_c1->getDimensionType());rois_net_c1->copyToHostTensor(&rois_tensor_c1);auto rois_ptr_c1 = rois_tensor_c1.host<float>();std::cout << std::string(rois_id1) << rois_tensor_c1.height() << " " << rois_tensor_c1.width() << " "<< rois_tensor_c1.channel() << " " << rois_tensor_c1.elementSize() << std::endl;for (int i = 0; i < rois_tensor_c1.elementSize(); i++) {vec_temp.push_back(rois_ptr_c1[i]);}vec_rois.push_back(vec_temp);vec_temp.clear();std::vector<float>().swap(vec_temp);std::cout << std::endl;auto rois_net_c2 = net->getSessionOutput(session, rois_id2);MNN::Tensor rois_tensor_c2(rois_net_c2, rois_net_c2->getDimensionType());rois_net_c2->copyToHostTensor(&rois_tensor_c2);auto rois_ptr_c2 = rois_tensor_c2.host<float>();std::cout << std::string(rois_id2) << rois_tensor_c2.height() << " " << rois_tensor_c2.width() << " "<< rois_tensor_c2.channel() << " " << rois_tensor_c2.elementSize() << std::endl;for (int i = 0; i < rois_tensor_c2.elementSize(); i++) {vec_temp.push_back(rois_ptr_c2[i]);}vec_rois.push_back(vec_temp);vec_temp.clear();std::vector<float>().swap(vec_temp);std::cout << std::endl;auto rois_net_c3 = net->getSessionOutput(session, rois_id3);MNN::Tensor rois_tensor_c3(rois_net_c3, rois_net_c3->getDimensionType());rois_net_c3->copyToHostTensor(&rois_tensor_c3);auto rois_ptr_c3 = rois_tensor_c3.host<float>();std::cout << std::string(rois_id3) << rois_tensor_c3.height() << " " << rois_tensor_c3.width() << " "<< rois_tensor_c3.channel() << " " << rois_tensor_c3.elementSize() << std::endl;for (int i = 0; i < rois_tensor_c3.elementSize(); i++) {vec_temp.push_back(rois_ptr_c3[i]);}vec_rois.push_back(vec_temp);vec_temp.clear();std::vector<float>().swap(vec_temp);std::cout << std::endl;auto rois_net_c4 = net->getSessionOutput(session, rois_id4);MNN::Tensor rois_tensor_c4(rois_net_c4, rois_net_c4->getDimensionType());rois_net_c4->copyToHostTensor(&rois_tensor_c4);auto rois_ptr_c4 = rois_tensor_c4.host<float>();std::cout << std::string(rois_id4)<< rois_tensor_c4.height() << " " << rois_tensor_c4.width() << " "<< rois_tensor_c4.channel() << " " << rois_tensor_c4.elementSize() << std::endl;for (int i = 0; i < rois_tensor_c4.elementSize(); i++) {vec_temp.push_back(rois_ptr_c4[i]);}vec_rois.push_back(vec_temp);vec_temp.clear();std::vector<float>().swap(vec_temp);// 写的尽量不依赖MNN ，还要进行NCNN的转化std::vector<int> keep;std::vector<std::vector<float>> keep_box;std::vector<float> keep_score;std::vector<int> keep_label;std::vector<Ploy> keep_box_dets;int nms_pre = 2000;int min_bbox_size = 0;float score_thr = 0.05;float iou_thr = 0.1;int max_pre_img = 2000;bool rescale = true;bool with_nms = true;bool use_sigmoid_cls = true;get_bboxes_single(vec_cls_scores,vec_bbox_preds,vec_rois,scores_shape_0,keep_box,keep_score,keep_label,img_width,img_height,nms_pre,cls_out_channels,bbox_preds_channels,rois_channels,rescale,with_nms,use_sigmoid_cls,scale_factor,score_thr,iou_thr,max_pre_img);float font_scale = 0.25;cv::Scalar bbox_color = cv::Scalar(226, 43, 138);cv::Scalar text_color = cv::Scalar(255, 255, 255);std::vector<std::vector<cv::Point>> contours;std::cout << keep_box.size() << std::endl;for (int i = 0; i < keep_box.size(); i++) {std::vector<cv::Point> contours_item;float p1_0 = 0, p1_1 = 0, p2_0 = 0, p2_1 = 0, p3_0 = 0, p3_1 = 0, p4_0 = 0, p4_1 = 0, xc = 0, yc = 0;convet_point(keep_box[i], p1_0, p1_1, p2_0, p2_1, p3_0, p3_1, p4_0, p4_1);contours_item.push_back(cv::Point(p1_0, p1_1));contours_item.push_back(cv::Point(p2_0, p2_1));contours_item.push_back(cv::Point(p3_0, p3_1));contours_item.push_back(cv::Point(p4_0, p4_1));contours.push_back(contours_item);xc = keep_box[i][0];yc = keep_box[i][1];cv::drawContours(img, contours, -1, bbox_color, 2);char text[256];sprintf(text, "%s %.2f%%", classes[keep_label[i]], keep_score[i]);int baseLine = 0;cv::Size label_size = cv::getTextSize(text, cv::FONT_HERSHEY_SIMPLEX, font_scale, 1, &baseLine);int text_width = label_size.width;int text_height = label_size.height;cv::rectangle(img, cv::Point(int(xc), int(yc) - text_height - 2), cv::Point(int(xc) + text_width, int(yc) + 3),cv::Scalar(0, 128, 0), -1);cv::putText(img, text, cv::Point(int(xc), int(yc)),cv::FONT_HERSHEY_SIMPLEX, font_scale, text_color);contours.clear();std::vector<std::vector<cv::Point>>().swap(contours);}cv::imshow("image", img);cv::imwrite("image.jpg", img);cv::waitKey(0);return 0;
}测试结果
对比结果
测试自己的训练结果
import mmcv
import numpy as np
import torch
from mmcv.ops import RoIPool
from mmcv.parallel import collate, scatter
from mmcv.runner import load_checkpointfrom mmdet.core import get_classes
from mmdet.datasets import replace_ImageToTensor
from mmdet.datasets.pipelines import Compose
from mmdet.models import build_detector
from argparse import ArgumentParserfrom mmdet.apis import inference_detector, init_detector
import torch
# import trochvision
import torch.utils.data
import argparse
import onnxruntime
import os
import cv2
import numpy as np
from onnxruntime.datasets import get_exampleimport mmrotate # noqa: F401device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")def getConfig(imgs, model):if isinstance(imgs, (list, tuple)):is_batch = Trueelse:imgs = [imgs]is_batch = Falsecfg = model.cfgdevice = next(model.parameters()).device # model deviceif isinstance(imgs[0], np.ndarray):cfg = cfg.copy()# set loading pipeline typecfg.data.test.pipeline[0].type = 'LoadImageFromWebcam'cfg.data.test.pipeline = replace_ImageToTensor(cfg.data.test.pipeline)test_pipeline = Compose(cfg.data.test.pipeline)datas = []for img in imgs:# prepare dataif isinstance(img, np.ndarray):# directly add imgdata = dict(img=img)else:# add information into dictdata = dict(img_info=dict(filename=img), img_prefix=None)# build the data pipelinedata = test_pipeline(data)datas.append(data)data = collate(datas, samples_per_gpu=len(imgs))# just get the actual data from DataContainerdata['img_metas'] = [img_metas.data[0] for img_metas in data['img_metas']]img_metas = data['img_metas'][0]return img_metasdef torch2onnx(args, model):img_metas = getConfig(args.img, model)model.forward = model.forward_dummymodel.to(device)input_names = ["input"] # 模型输入的name# output_names = ["output","prob"] # 模型输出的namefrom torch.autograd import Variableinput = cv2.imread(args.img)(new_h, new_w) = (1024, 1024) # 配置文件h, w = input.shape[:2]img_mmcv_scale_factor = max(new_h, new_w) / max(h, w)(new_h, new_w) = (img_mmcv_scale_factor * h, img_mmcv_scale_factor * w)h_scale = new_h / hw_scale = new_w / wscale_factor = np.array([w_scale, h_scale, w_scale, h_scale, 1],dtype=np.float32)inputr = cv2.resize(input, (int(new_w), int(new_h))) # hwc bgrinput = cv2.cvtColor(inputr, cv2.COLOR_BGR2RGB) # hwc rgb# [0.485, 0.456, 0.406], [0.229, 0.224, 0.225])input = np.transpose(input, (2, 0, 1)).astype(np.float32) # chw rgb# input=input/255.0print("befor the input[0,0,0]:{}".format(input[0, 0, 0]))print("the size of input[0,...] is {}".format(input[0, ...].shape))print("the size of input[1,...] is {}".format(input[1, ...].shape))print("the size of input[2,...] is {}".format(input[2, ...].shape))input[0, ...] = (input[0, ...] - 123.675) / 58.395 # lainput[1, ...] = (input[1, ...] - 116.28) / 57.12input[2, ...] = (input[2, ...] - 103.53) / 57.375# 'mean': array([123.675, 116.28, 103.53], dtype=float32), 'std': array([58.395, 57.12, 57.375], dtype=float32)print("after input[0,0,0]:{}".format(input[0, 0, 0]))now_image1 = Variable(torch.from_numpy(input))dummy_input = now_image1.unsqueeze(0).to(device)# vutils.save_image(dummy_input, r'G:\mmrotate\mmrotate\models\detectors\dummy_input.jpg', normalize=True)print("====", dummy_input, type(dummy_input))torch.onnx.export(model, dummy_input, args.onnx_model_path, input_names=input_names,export_params=True,keep_initializers_as_inputs=True,do_constant_folding=True,verbose=False,opset_version=11)example_model = get_example(args.onnx_model_path)session = onnxruntime.InferenceSession(example_model)input_name = session.get_inputs()[0].nameresult = session.run([], {input_name: dummy_input.data.cpu().numpy()})print("outs=", result)a0 = torch.tensor(result[0]).to(device)a1 = torch.tensor(result[1]).to(device)a2 = torch.tensor(result[2]).to(device)a3 = torch.tensor(result[3]).to(device)a4 = torch.tensor(result[4]).to(device)a = [a0, a1, a2, a3, a4]b0 = torch.tensor(result[5]).to(device)b1 = torch.tensor(result[6]).to(device)b2 = torch.tensor(result[7]).to(device)b3 = torch.tensor(result[8]).to(device)b4 = torch.tensor(result[9]).to(device)b = [b0, b1, b2, b3, b4]outs = (a, b)c0 = torch.tensor(result[10]).to(device)b0_scale_factor = c0.new_tensor(scale_factor)b0_scale_factor = c0 / b0_scale_factorc0 = b0_scale_factorc1 = torch.tensor(result[11]).to(device)c2 = torch.tensor(result[12]).to(device)c3 = torch.tensor(result[13]).to(device)c4 = torch.tensor(result[14]).to(device)c = [c0, c1, c2, c3, c4]c = [c[idx] / item.new_tensor(scale_factor) for idx, item in enumerate(c)]rois = [c]result_ = model.simple_test_onnx(img_metas, outs, rois, False)model.show_result(args.img, result_[0], score_thr=args.score_thr, out_file=args.onnxoutput)def main():"""Test a single image."""parser = ArgumentParser()parser.add_argument('--img', default=r"/home/ubuntu/CLionProjects/untitled8/3.png",help='Image file')parser.add_argument('--config', default=r"/home/ubuntu/mmrotate/configs/kfiou/r3det_kfiou_ln_r50_fpn_1x_dota_oc.py",help='Config file')parser.add_argument('--checkpoint', default=r"/home/ubuntu/Downloads/test_card/card_24.pth",help='Checkpoint file')parser.add_argument('--onnx_model_path',default=r"/home/ubuntu/Downloads/test_card/card_24.onnx",help='onnx_model_path')parser.add_argument('--device', default='cuda:0', help='Device used for inference')parser.add_argument('--output', default=r'ptsxj731533730.jpg', help='Output image')parser.add_argument('--onnxoutput', default=r'onnxsxj731533730.jpg', help='Output image')parser.add_argument('--score-thr', type=float, default=0.3, help='bbox score threshold')args = parser.parse_args()# build the model from a config file and a checkpoint filemodel = init_detector(args.config, args.checkpoint, device=args.device)# test a single imageresult = inference_detector(model, args.img)print("result=", result)# show the resultsmodel.show_result(args.img, result, score_thr=args.score_thr, out_file=args.output)torch2onnx(args, model)# D:\Python39\Lib\site-packages\mmcv\opsif __name__ == '__main__':main()原图
pt图
onnx图
c++ 新训练的模型和之前的模型的输出名，一一对应就行，数据可以不用比对
百度云中的模型
ubuntu@ubuntu:~/Downloads/test_card$ python3 -m onnxsim card_24.onnx card_24_sim.onnx
Your model contains "Tile" ops or/and "ConstantOfShape" ops. Folding these ops
can make the simplified model much larger. If it is not expected, please specify
"--no-large-tensor" (which will lose some optimization chances)
Simplifying...
Finish! Here is the difference:
┏━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┓
┃ ┃ Original Model ┃ Simplified Model ┃
┡━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━┩
│ Add │ 55 │ 43 │
│ ArgMax │ 5 │ 5 │
│ Cast │ 28 │ 0 │
│ Clip │ 10 │ 10 │
│ Concat │ 62 │ 5 │
│ Constant │ 344 │ 0 │
│ ConstantOfShape │ 29 │ 0 │
│ Conv │ 167 │ 161 │
│ Div │ 15 │ 0 │
│ Equal │ 10 │ 0 │
│ Exp │ 10 │ 10 │
│ Expand │ 50 │ 5 │
│ Gather │ 71 │ 35 │
│ GatherElements │ 10 │ 10 │
│ If │ 5 │ 0 │
│ MaxPool │ 1 │ 1 │
│ Mul │ 34 │ 25 │
│ Range │ 4 │ 0 │
│ ReduceMax │ 5 │ 5 │
│ Relu │ 129 │ 129 │
│ Reshape │ 32 │ 15 │
│ Resize │ 2 │ 2 │
│ Shape │ 79 │ 0 │
│ Slice │ 47 │ 25 │
│ Squeeze │ 5 │ 10 │
│ Sub │ 5 │ 0 │
│ Tile │ 20 │ 0 │
│ Transpose │ 10 │ 10 │
│ Unsqueeze │ 125 │ 50 │
│ Where │ 5 │ 0 │
│ Model Size │ 160.1MiB │ 166.5MiB │
└─────────────────┴────────────────┴──────────────────┘ubuntu@ubuntu:~/MNN/build$ ./MNNConvert -f ONNX --modelFile /home/ubuntu/Downloads/test_card/card_24_sim.onnx --MNNModel /home/ubuntu/Downloads/test_card/card_24_sim.mnn --bizCode MNN
Start to Convert Other Model Format To MNN Model...
[09:50:51] /home/ubuntu/MNN/tools/converter/source/onnx/onnxConverter.cpp:40: ONNX Model ir version: 6
Start to Optimize the MNN Net...
inputTensors : [ input, ]
outputTensors: [ 1351, 1524, 1697, 1870, 2043, 2095, 2096, 2113, 2114, 2131, 2132, 2149, 2150, 2167, 2168, ]
Converted Success!测试代码main.cpp
#include <iostream>
#include <ostream>
#include <random>
#include <vector>
#include <opencv2/opencv.hpp>#include<MNN/Interpreter.hpp>
#include<MNN/ImageProcess.hpp>
#include "polyiou.h"#define IMAGE_TARGET 1024struct Ploy {float p1_0;float p1_1;float p2_0;float p2_1;float p3_0;float p3_1;float p4_0;float p4_1;Ploy() : p1_0(0), p1_1(0), p2_0(0), p2_1(0), p3_0(0), p3_1(0), p4_0(0), p4_1(0) {}Ploy(float p1_0, float p1_1, float p2_0, float p2_1, float p3_0, float p3_1, float p4_0, float p4_1) : p1_0(p1_0),p1_1(p1_1),p2_0(p2_0),p2_1(p2_1),p3_0(p3_0),p3_1(p3_1),p4_0(p4_0),p4_1(p4_1) {}
};struct BBox {int idx;float conf;Ploy ploy;BBox() : idx(0), conf(0), ploy(0, 0, 0, 0, 0, 0, 0, 0) {}BBox(int idx, float conf, Ploy ploy) : idx(idx), conf(conf), ploy(ploy) {}
};double mysoftmax(double num) {return exp(num);
}double mysigmoid(double num) {return 1 / (1 + exp(-num));
}template<typename T>
void softmax(const typename ::std::vector<T> &v, typename ::std::vector<T> &s) {double sum = 0.0;transform(v.begin(), v.end(), s.begin(), mysoftmax);sum = accumulate(s.begin(), s.end(), sum);for (size_t i = 0; i < s.size(); ++i)s.at(i) /= sum;
}template<typename T>
void sigmoid(const typename ::std::vector<T> &v, typename ::std::vector<T> &s) {transform(v.begin(), v.end(), s.begin(), mysigmoid);}void cls_score_softmax(std::vector<float> &vec_cls_scores, std::vector<float> &item_cls_scores) {item_cls_scores = vec_cls_scores;softmax(vec_cls_scores, item_cls_scores);}void cls_score_sigmoid(std::vector<float> &vec_cls_scores, std::vector<float> &item_cls_scores) {item_cls_scores = vec_cls_scores;sigmoid(vec_cls_scores, item_cls_scores);}void dim_permute(std::vector<float> vec_cls_scores, std::vector<float> &item_cls_score, int w_h_dim) {for (int i = 0; i < w_h_dim; i++) {for (int j = i; j < vec_cls_scores.size(); j += w_h_dim) {item_cls_score.emplace_back(vec_cls_scores[j]);}}
}void process_top_k(std::vector<float> scores, int top_K, std::vector<float> &scores_K, std::vector<int> &index_K) {std::vector<int> idx(scores.size());std::iota(idx.begin(), idx.end(), 0);std::sort(idx.begin(), idx.end(),[&scores](int index_1, int index_2) { return scores[index_1] > scores[index_2]; });
// 获取K值int k_num = std::min<int>(scores.size(), top_K);int idx_j = 0;for (int j = 0; j < k_num; ++j) {idx_j = idx[j];index_K.emplace_back(idx_j);scores_K.emplace_back(scores[idx_j]);}}float norm_angle(float angle, std::string angle_range) {if (angle_range == "oc") {return angle;} else if (angle_range == "le135") {return fmod((angle + M_PI / 4), M_PI) - M_PI / 4;} else if (angle_range == "le90") {return fmod((angle + (M_PI) / 2), M_PI) - M_PI / 2;}std::cout << "Not yet implemented." << std::endl;return 0;
}void bbox_coder_decode(std::vector<float> top_k_anchors,std::vector<float> top_k_bbox_pred,std::vector<float> &bbox_coder,int bbox_preds_channels,int img_width,int img_height,float wh_ratio_clip = 16.0f / 1000.0f,bool add_ctr_clamp = false,int ctr_clamp = 32,bool proj_xy = false,std::string angle_range = "oc",bool edge_swap = false) {// G:\mmrotate\mmrotate\core\bbox\coder\delta_xywh_rbbox_coder.pyfloat dx_width;float dy_height;float dw;float dh;float gx;float gy;float ga;for (int i = 0; i < top_k_bbox_pred.size(); i += bbox_preds_channels) {float dx = top_k_bbox_pred[i];float dy = top_k_bbox_pred[i + 1];float dw = top_k_bbox_pred[i + 2];float dh = top_k_bbox_pred[i + 3];float da = top_k_bbox_pred[i + 4];float px = top_k_anchors[i];float py = top_k_anchors[i + 1];float pw = top_k_anchors[i + 2];float ph = top_k_anchors[i + 3];float pa = top_k_anchors[i + 4];float dx_width = pw * dx;float dy_height = ph * dy;float max_ratio = abs(log(wh_ratio_clip));if (add_ctr_clamp) {dx_width = (dx_width >= -ctr_clamp) ? dx_width : -ctr_clamp;dx_width = (dx_width <= ctr_clamp) ? dx_width : ctr_clamp;dy_height = (dy_height >= -ctr_clamp) ? dy_height : -ctr_clamp;dy_height = (dy_height <= ctr_clamp) ? dy_height : ctr_clamp;dw = (dw <= max_ratio) ? dw : max_ratio;dh = (dh <= max_ratio) ? dh : max_ratio;} else {dw = (dw <= max_ratio) ? dw : max_ratio;dw = (dw >= -max_ratio) ? dw : -max_ratio;dh = (dh <= max_ratio) ? dh : max_ratio;dh = (dh >= -max_ratio) ? dh : -max_ratio;}float gw = pw * exp(dw);float gh = ph * exp(dh);if (proj_xy) {gx = dx * pw * cos(pa) - dy * ph * sin(pa) + px;gy = dx * pw * sin(pa) + dy * ph * cos(pa) + py;} else {gx = px + dx_width;gy = py + dy_height;ga = norm_angle(pa + da, angle_range);}// if max_shape is not None:{gx = (gx >= 0) ? gx : 0;gx = (gx <= img_width - 1) ? gx : img_width - 1;gy = (gy >= 0) ? gy : 0;gy = (gy <= img_height - 1) ? gy : img_height - 1;}if (edge_swap) {float w_regular = gw > gh ? gw : gh;float h_regular = gw > gh ? gh : gw;float theta_regular = gw > gh ? ga : ga + M_PI / 2;theta_regular = norm_angle(theta_regular, angle_range);bbox_coder.emplace_back(gx);bbox_coder.emplace_back(gy);bbox_coder.emplace_back(gw);bbox_coder.emplace_back(gh);bbox_coder.emplace_back(ga);} else {bbox_coder.emplace_back(gx);bbox_coder.emplace_back(gy);bbox_coder.emplace_back(gw);bbox_coder.emplace_back(gh);bbox_coder.emplace_back(ga);}}}// 进行iou计算
float iou(Ploy &r1, Ploy &r2) {std::vector<double> p;p.emplace_back(r1.p1_0);p.emplace_back(r1.p1_1);p.emplace_back(r1.p2_0);p.emplace_back(r1.p2_1);p.emplace_back(r1.p3_0);p.emplace_back(r1.p3_1);p.emplace_back(r1.p4_0);p.emplace_back(r1.p4_1);std::vector<double> q;q.emplace_back(r2.p1_0);q.emplace_back(r2.p1_1);q.emplace_back(r2.p2_0);q.emplace_back(r2.p2_1);q.emplace_back(r2.p3_0);q.emplace_back(r2.p3_1);q.emplace_back(r2.p4_0);q.emplace_back(r2.p4_1);double iou = iou_poly(p, q);p.clear();std::vector<double>().swap(p);q.clear();std::vector<double>().swap(q);return iou;}// 进行nms计算
void single_class_non_max_suppression(std::vector<Ploy> ploys, std::vector<float> confs, std::vector<Ploy> &ans,std::vector<int> &keep_idx, float conf_thresh, float iou_thresh) {if (ploys.size() == 0) {return;}std::vector<BBox> bboxes;BBox bbox;for (int i = 0; i < (int) ploys.size(); ++i) {bboxes.emplace_back(BBox(i, confs[i], ploys[i]));}// 对bbox的conf进行降序排序sort(bboxes.begin(), bboxes.end(), [&](const BBox &a, const BBox &b) {return a.conf > b.conf;});while (!bboxes.empty()) {bbox = bboxes[0];if (bbox.conf < conf_thresh) {break;}keep_idx.emplace_back(bbox.idx);bboxes.erase(bboxes.begin());// 让conf最高的bbox与其他剩余的bbox进行iou计算int size = bboxes.size();for (int i = 0; i < size; ++i) {float iou_ans = iou(bbox.ploy, bboxes[i].ploy);if (iou_ans > iou_thresh) {bboxes.erase(bboxes.begin() + i);size = bboxes.size();i = i - 1;}}}for (const int number : keep_idx) {ans.emplace_back(ploys[number]);}}void convet_point(std::vector<float> mutil_point,float &p1_0,float &p1_1,float &p2_0,float &p2_1,float &p3_0,float &p3_1,float &p4_0,float &p4_1
) {float xc = mutil_point[0];float yc = mutil_point[1];float w = mutil_point[2];float h = mutil_point[3];float ag = mutil_point[4];float wx = w / 2 * cos(ag);float wy = w / 2 * sin(ag);float hx = -h / 2 * sin(ag);float hy = h / 2 * cos(ag);p1_0 = xc - wx - hx;p1_1 = yc - wy - hy;p2_0 = xc + wx - hx;p2_1 = yc + wy - hy;p3_0 = xc + wx + hx;p3_1 = yc + wy + hy;p4_0 = xc - wx + hx;p4_1 = yc - wy + hy;
}void ext_module_nms_rotated(std::vector<std::vector<float>> dim_mlvl_bbox,std::vector<float> dim_mlvl_score,std::vector<int> order_K,std::vector<int> &keep_idx,float nms_iou_thr,bool multi_label = false) {// nms.pystd::cout << dim_mlvl_bbox.size() << " " << " " << dim_mlvl_score.size() << std::endl;std::vector<Ploy> pts0_item;std::vector<Ploy> ans;for (int i = 0; i < dim_mlvl_bbox.size(); i++) {float p1_0 = 0, p1_1 = 0, p2_0 = 0, p2_1 = 0, p3_0 = 0, p3_1 = 0, p4_0 = 0, p4_1 = 0;convet_point(dim_mlvl_bbox[i], p1_0, p1_1, p2_0, p2_1, p3_0, p3_1, p4_0, p4_1);pts0_item.emplace_back(Ploy(p1_0, p1_1, p2_0, p2_1, p3_0, p3_1, p4_0, p4_1));}single_class_non_max_suppression(pts0_item, dim_mlvl_score, ans, keep_idx, 0.3, nms_iou_thr);pts0_item.clear();std::vector<Ploy>().swap(pts0_item);ans.clear();std::vector<Ploy>().swap(ans);}void nms_rotated(std::vector<std::vector<float>> dim_mlvl_bbox,std::vector<float> dim_mlvl_score,std::vector<int> &keep_inds,float nms_iou_thr,bool clockwise = true) {if (dim_mlvl_bbox[0].size() == 0) {return;}std::vector<int> order_K;std::vector<float> scores_K;process_top_k(dim_mlvl_score, dim_mlvl_score.size(), scores_K, order_K);ext_module_nms_rotated(dim_mlvl_bbox, dim_mlvl_score, order_K, keep_inds, nms_iou_thr);
}void multiclass_nms_rotated(std::vector<std::vector<float>> multi_bbox,std::vector<std::vector<float>> mlvl_score,std::vector<int> mlvl_data_shape,std::vector<std::vector<float>> &keep_box,std::vector<float> &keep_score,std::vector<int> &keep_label,float score_thr = 0.05,float nms_iou_thr = 0.1,int max_num = 2000,bool score_factors = false,bool return_inds = false) {std::vector<std::vector<float>> dim_mlvl_bbox;std::vector<float> temp_dim_mlvl_bbox;std::vector<std::vector<float>> dets_;std::vector<float> dim_mlvl_score;std::vector<int> dim_mlvl_label;std::vector<int> dim_mlvl_inds;float max_coordinate = -1;int num_classes = mlvl_score[0].size() - 1;if (multi_bbox[0].size() > 5) { ;} else { ;}for (int i = 0; i < mlvl_score.size(); i++) {int length = mlvl_score[i].size() - 1;for (int j = 0; j < length; j++) {if (mlvl_score[i][j] > score_thr) {dim_mlvl_score.emplace_back(mlvl_score[i][j]);dim_mlvl_inds.emplace_back(i * length + j);dim_mlvl_label.emplace_back(j);auto item = multi_bbox[(i * length + j) / num_classes];auto maxPosition = max_element(item.begin(), item.end());max_coordinate = std::max(max_coordinate, item[maxPosition - item.begin()]);dim_mlvl_bbox.emplace_back(item);}}}if (dim_mlvl_bbox.size() == 0) {for (int i = 0; i < dim_mlvl_bbox.size(); i++) {for (int j = 0; j < dim_mlvl_bbox[i].size(); j++) {temp_dim_mlvl_bbox.emplace_back(dim_mlvl_bbox[i][j]);}temp_dim_mlvl_bbox.emplace_back(dim_mlvl_score[i]);dets_.emplace_back(temp_dim_mlvl_bbox);temp_dim_mlvl_bbox.clear();std::vector<float>().swap(temp_dim_mlvl_bbox);}if (return_inds) {return; //dets, labels, inds} else {return;//dets, labels, inds}}std::vector<int> keep;nms_rotated(dim_mlvl_bbox, dim_mlvl_score, keep, nms_iou_thr);std::cout << "kip_dim= " << keep.size() << std::endl;if (max_num > 0) {if (keep.size() > max_num) {keep.resize(max_num);}}for (int i = 0; i < keep.size(); i++) {keep_box.emplace_back(dim_mlvl_bbox[keep[i]]);keep_score.emplace_back(dim_mlvl_score[keep[i]]);keep_label.emplace_back(dim_mlvl_label[keep[i]]);}std::cout << keep_box.size() << " " << keep_score.size() << " " << keep_label.size() << std::endl;dim_mlvl_bbox.clear();std::vector<std::vector<float>>().swap(dim_mlvl_bbox);temp_dim_mlvl_bbox.clear();std::vector<float>().swap(temp_dim_mlvl_bbox);dets_.clear();std::vector<std::vector<float>>().swap(dets_);dim_mlvl_score.clear();std::vector<float>().swap(dim_mlvl_score);dim_mlvl_label.clear();std::vector<int>().swap(dim_mlvl_label);dim_mlvl_inds.clear();std::vector<int>().swap(dim_mlvl_inds);}void get_bboxes_single(std::vector<std::vector<float>> vec_cls_scores,std::vector<std::vector<float>> vec_bbox_preds,std::vector<std::vector<float>> vec_rois,std::vector<int> scores_shape_0,std::vector<std::vector<float>> &keep_box,std::vector<float> &keep_score,std::vector<int> &keep_label,int img_width,int img_height,int nms_pre,int cls_out_channels,int bbox_preds_channels,int rois_channels,bool rescale,bool with_nms,bool use_sigmoid_cls,float scale_factor[],float score_thr,float iou_thr,int max_pre_img) {std::vector<std::vector<float>> mlvl_bboxes;std::vector<std::vector<float>> mlvl_scores;std::vector<float> item_cls_score;std::vector<float> item_bbox_pred;std::vector<float> item_s_cls_score;std::vector<int> topk_inds;std::vector<float> scores_K;std::vector<float> max_scores;std::vector<float> top_k_anchors;std::vector<float> top_k_bbox_pred;std::vector<float> top_k_scores;std::vector<float> bbox_coder;std::vector<std::vector<float>> mlvl_bbox;std::vector<std::vector<float>> mlvl_score;std::vector<int> mlvl_data_shape;std::vector<float> item_temp;for (int i = 0; i < vec_cls_scores.size(); i++) {dim_permute(vec_cls_scores[i], item_cls_score, vec_cls_scores[i].size() / cls_out_channels); //1,2,0dim_permute(vec_bbox_preds[i], item_bbox_pred, vec_bbox_preds[i].size() / bbox_preds_channels); //1,2,0//item_cls_score 16384 15 仍然拉伸成1维数据// item_bbox_pred 5 仍然拉伸成1维数据if (use_sigmoid_cls) {cls_score_sigmoid(item_cls_score, item_s_cls_score);} else {cls_score_softmax(item_cls_score, item_s_cls_score);}if (nms_pre > 0 && scores_shape_0[i] > nms_pre) {if (use_sigmoid_cls) {for (int j = 0; j < item_s_cls_score.size() / cls_out_channels; j++) {auto maxPosition = max_element(item_s_cls_score.begin() + j * cls_out_channels,item_s_cls_score.begin() + (j + 1) * cls_out_channels);max_scores.emplace_back(item_s_cls_score[maxPosition - item_s_cls_score.begin()]);}process_top_k(max_scores, nms_pre, scores_K, topk_inds);for (int m = 0; m < topk_inds.size(); m++) {int index = topk_inds[m];for (int n = 0; n < rois_channels; n++) {top_k_anchors.emplace_back(vec_rois[i][index * rois_channels + n]);}for (int n = 0; n < bbox_preds_channels; n++) {top_k_bbox_pred.emplace_back(item_bbox_pred[index * bbox_preds_channels + n]);}for (int n = 0; n < cls_out_channels; n++) {top_k_scores.emplace_back(item_s_cls_score[index * cls_out_channels + n]);}}topk_inds.clear();std::vector<int>().swap(topk_inds);scores_K.clear();std::vector<float>().swap(scores_K);max_scores.clear();std::vector<float>().swap(max_scores);}mlvl_data_shape.emplace_back(nms_pre);} else {top_k_anchors = vec_rois[i];top_k_bbox_pred = item_bbox_pred;top_k_scores = item_s_cls_score;mlvl_data_shape.emplace_back(scores_shape_0[i]);}bbox_coder_decode(top_k_anchors, top_k_bbox_pred, bbox_coder, bbox_preds_channels, img_width, img_height);mlvl_bbox.emplace_back(bbox_coder);mlvl_score.emplace_back(top_k_scores);top_k_anchors.clear();std::vector<float>().swap(top_k_anchors);bbox_coder.clear();std::vector<float>().swap(bbox_coder);top_k_scores.clear();std::vector<float>().swap(top_k_scores);top_k_bbox_pred.clear();std::vector<float>().swap(top_k_bbox_pred);item_cls_score.clear();std::vector<float>().swap(item_cls_score);item_bbox_pred.clear();std::vector<float>().swap(item_bbox_pred);item_s_cls_score.clear();std::vector<float>().swap(item_s_cls_score);}mlvl_bboxes = mlvl_bbox;mlvl_scores = mlvl_score;if (rescale) {mlvl_bboxes.clear();std::vector<std::vector<float>>().swap(mlvl_bboxes);// angle should not be rescaledfor (int i = 0; i < mlvl_bbox.size(); i++) {for (int j = 0; j < mlvl_bbox[i].size(); j += bbox_preds_channels) {for (int k = 0; k < bbox_preds_channels - 1; k++) {item_temp.emplace_back(mlvl_bbox[i][j + k] / scale_factor[k]);}item_temp.emplace_back(mlvl_bbox[i][j + bbox_preds_channels - 1]);}mlvl_bboxes.emplace_back(item_temp);item_temp.clear();std::vector<float>().swap(item_temp);}}if (use_sigmoid_cls) {mlvl_scores.clear();std::vector<std::vector<float>>().swap(mlvl_scores);for (int i = 0; i < mlvl_score.size(); i++) {for (int j = 0; j < mlvl_score[i].size(); j += cls_out_channels) {for (int k = 0; k < cls_out_channels; k++) {item_temp.emplace_back(mlvl_score[i][j + k]);};item_temp.emplace_back(0);}mlvl_scores.emplace_back(item_temp);item_temp.clear();std::vector<float>().swap(item_temp);}cls_out_channels = cls_out_channels + 1;}mlvl_bbox.clear();std::vector<std::vector<float>>().swap(mlvl_bbox);//cat datafor (int i = 0; i < mlvl_bboxes.size(); i++) {for (int j = 0; j < mlvl_bboxes[i].size(); j += bbox_preds_channels) {for (int k = 0; k < bbox_preds_channels; k++) {item_temp.emplace_back(mlvl_bboxes[i][j + k]);}mlvl_bbox.emplace_back(item_temp);item_temp.clear();std::vector<float>().swap(item_temp);}}mlvl_score.clear();std::vector<std::vector<float>>().swap(mlvl_score);for (int i = 0; i < mlvl_scores.size(); i++) {for (int j = 0; j < mlvl_scores[i].size(); j += cls_out_channels) { // 扩展了0维空间for (int k = 0; k < cls_out_channels; k++) {item_temp.emplace_back(mlvl_scores[i][j + k]);}mlvl_score.emplace_back(item_temp);item_temp.clear();std::vector<float>().swap(item_temp);}}if (with_nms) {multiclass_nms_rotated(mlvl_bbox,mlvl_score,mlvl_data_shape,keep_box,keep_score,keep_label,score_thr,iou_thr,max_pre_img);} else {// 引用已经回传数据}item_cls_score.clear(); //有点习惯了 释放，其实局部变量没有必要std::vector<float>().swap(item_cls_score);item_bbox_pred.clear();std::vector<float>().swap(item_bbox_pred);item_s_cls_score.clear();std::vector<float>().swap(item_s_cls_score);topk_inds.clear();std::vector<int>().swap(topk_inds);scores_K.clear();std::vector<float>().swap(scores_K);max_scores.clear();std::vector<float>().swap(max_scores);top_k_anchors.clear();std::vector<float>().swap(top_k_anchors);top_k_bbox_pred.clear();std::vector<float>().swap(top_k_bbox_pred);top_k_scores.clear();std::vector<float>().swap(top_k_scores);bbox_coder.clear();std::vector<float>().swap(bbox_coder);mlvl_bbox.clear();std::vector<std::vector<float>>().swap(mlvl_bbox);mlvl_score.clear();std::vector<std::vector<float>>().swap(mlvl_score);mlvl_data_shape.clear();std::vector<int>().swap(mlvl_data_shape);mlvl_bboxes.clear();std::vector<std::vector<float>>().swap(mlvl_bboxes);mlvl_scores.clear();std::vector<std::vector<float>>().swap(mlvl_scores);
}int main(int argc, char **argv) {cv::Mat img = cv::imread("/home/ubuntu/CLionProjects/card_detect/IMG20220913091114.jpg");const char *classes[] = {"card"};int new_h=1024.0;int new_w=1024.0;int img_width = img.cols;int img_height = img.rows;float img_mmcv_scale_factor = std::max(new_w*1.0f, new_h*1.0f) / std::max(img_height*1.0f, img_width*1.0f);new_h=img_mmcv_scale_factor*img_height;new_w=img_mmcv_scale_factor*img_width;float h_scale = new_h / (img_height*1.0f);float w_scale = new_w / (img_width*1.0f);float scale_factor[] = {w_scale, h_scale, w_scale, h_scale};cv::Mat img_resized;cv::resize(img, img_resized, cv::Size(new_w, new_h));auto net = std::shared_ptr<MNN::Interpreter>(MNN::Interpreter::createFromFile("../model/card_24_sim.mnn"));MNN::ScheduleConfig netConfig;netConfig.type = MNN_FORWARD_CPU;netConfig.numThread = 4;auto session = net->createSession(netConfig);auto input = net->getSessionInput(session, nullptr);const float mean_vals[3] = {123.675f , 116.28f, 103.53f};const float norm_255[3] = {1.0f / 58.395, 1.0f / 57.12, 1.0f / 57.375};std::shared_ptr<MNN::CV::ImageProcess> pretreat(MNN::CV::ImageProcess::create(MNN::CV::BGR, MNN::CV::RGB, mean_vals, 3,norm_255, 3));pretreat->convert(img_resized.data, (int) new_h, (int) new_w, img_resized.step[0], input);net->runSession(session);char* cls_id0="2095";char* cls_id1="2113";char* cls_id2="2131";char* cls_id3="2149";char* cls_id4="2167";char* bbox_id0="2096";char* bbox_id1="2114";char *bbox_id2="2132";char *bbox_id3="2150";char *bbox_id4="2168";char* rois_id0="1351";char* rois_id1="1524";char *rois_id2="1697";char *rois_id3="1870";char *rois_id4="2043";/*char* cls_id0="1967";//"1962";char* cls_id1="1985";//"1980";char* cls_id2="2003";//"1998";char* cls_id3="2021";//"2016";char* cls_id4="2039";//"2034";char* bbox_id0="1968";//"1963";char* bbox_id1="1986";//"1981";char *bbox_id2="2004";//"1999";char *bbox_id3="2022";//"2017";char *bbox_id4="2040";//"2035";char* rois_id0="1238";//"1233";char* rois_id1="1411";//"1406";char *rois_id2="1584";//"1579";char *rois_id3="1757";//"1752";char *rois_id4="1930";//"1925";
*/std::vector<std::vector<float>> vec_cls_scores;std::vector<int> scores_shape_0;std::vector<float> vec_temp;int cls_out_channels = 0;std::cout << std::endl;auto cls_scores_a0 = net->getSessionOutput(session, cls_id0);MNN::Tensor cls_scores_tensor_a0(cls_scores_a0, cls_scores_a0->getDimensionType());cls_scores_a0->copyToHostTensor(&cls_scores_tensor_a0);auto cls_scores_ptr_a0 = cls_scores_tensor_a0.host<float>();std::cout << std::string(cls_id0) << cls_scores_tensor_a0.height() << " " << cls_scores_tensor_a0.width() << " "<< cls_scores_tensor_a0.channel() << " " << cls_scores_tensor_a0.elementSize() << std::endl;cls_out_channels = cls_scores_tensor_a0.channel();for (int i = 0; i < cls_scores_tensor_a0.elementSize(); i++) {vec_temp.emplace_back(cls_scores_ptr_a0[i]);}scores_shape_0.emplace_back(cls_scores_tensor_a0.height() * cls_scores_tensor_a0.width());vec_cls_scores.emplace_back(vec_temp);vec_temp.clear();std::vector<float>().swap(vec_temp);std::cout << std::endl;auto cls_scores_a1 = net->getSessionOutput(session, cls_id1);MNN::Tensor cls_scores_tensor_a1(cls_scores_a1, cls_scores_a1->getDimensionType());cls_scores_a1->copyToHostTensor(&cls_scores_tensor_a1);auto cls_scores_ptr_a1 = cls_scores_tensor_a1.host<float>();std::cout << std::string(cls_id1) << cls_scores_tensor_a1.height() << " " << cls_scores_tensor_a1.width() << " "<< cls_scores_tensor_a1.channel() << " " << cls_scores_tensor_a1.elementSize() << std::endl;for (int i = 0; i < cls_scores_tensor_a1.elementSize(); i++) {vec_temp.emplace_back(cls_scores_ptr_a1[i]);}scores_shape_0.emplace_back(cls_scores_tensor_a1.height() * cls_scores_tensor_a1.width());vec_cls_scores.emplace_back(vec_temp);vec_temp.clear();std::vector<float>().swap(vec_temp);std::cout << std::endl;auto cls_scores_net_a2 = net->getSessionOutput(session, cls_id2);MNN::Tensor cls_scores_tensor_a2(cls_scores_net_a2, cls_scores_net_a2->getDimensionType());cls_scores_net_a2->copyToHostTensor(&cls_scores_tensor_a2);auto cls_scores_ptr_a2 = cls_scores_tensor_a2.host<float>();std::cout << std::string(cls_id2) << cls_scores_tensor_a2.height() << " " << cls_scores_tensor_a2.width() << " "<< cls_scores_tensor_a2.channel() << " " << cls_scores_tensor_a2.elementSize() << std::endl;for (int i = 0; i < cls_scores_tensor_a2.elementSize(); i++) {vec_temp.emplace_back(cls_scores_ptr_a2[i]);}scores_shape_0.emplace_back(cls_scores_tensor_a2.height() * cls_scores_tensor_a2.width());vec_cls_scores.emplace_back(vec_temp);vec_temp.clear();std::vector<float>().swap(vec_temp);std::cout << std::endl;auto cls_scores_net_a3 = net->getSessionOutput(session, cls_id3);MNN::Tensor cls_scores_tensor_a3(cls_scores_net_a3, cls_scores_net_a3->getDimensionType());cls_scores_net_a3->copyToHostTensor(&cls_scores_tensor_a3);auto rois_ptr_a3 = cls_scores_tensor_a3.host<float>();std::cout << std::string(cls_id3)<< cls_scores_tensor_a3.height() << " " << cls_scores_tensor_a3.width() << " "<< cls_scores_tensor_a3.channel() << " " << cls_scores_tensor_a3.elementSize() << std::endl;for (int i = 0; i < cls_scores_tensor_a3.elementSize(); i++) {vec_temp.emplace_back(rois_ptr_a3[i]);}scores_shape_0.emplace_back(cls_scores_tensor_a3.height() * cls_scores_tensor_a3.width());vec_cls_scores.emplace_back(vec_temp);vec_temp.clear();std::vector<float>().swap(vec_temp);std::cout << std::endl;auto cls_scores_net_a4 = net->getSessionOutput(session, cls_id4);MNN::Tensor cls_scores_tensor_a4(cls_scores_net_a4, cls_scores_net_a4->getDimensionType());cls_scores_net_a4->copyToHostTensor(&cls_scores_tensor_a4);auto cls_scores_ptr_a4 = cls_scores_tensor_a4.host<float>();std::cout << std::string(cls_id4) << cls_scores_tensor_a4.height() << " " << cls_scores_tensor_a4.width() << " "<< cls_scores_tensor_a4.channel() << " " << cls_scores_tensor_a4.elementSize() << std::endl;for (int i = 0; i < cls_scores_tensor_a4.elementSize(); i++) {vec_temp.emplace_back(cls_scores_ptr_a4[i]);}scores_shape_0.emplace_back(cls_scores_tensor_a4.height() * cls_scores_tensor_a4.width());vec_cls_scores.emplace_back(vec_temp);vec_temp.clear();std::vector<float>().swap(vec_temp);std::vector<std::vector<float>> vec_bbox_preds;int bbox_preds_channels = 0;std::cout << std::endl;auto bbox_preds_net_b0 = net->getSessionOutput(session, bbox_id0);MNN::Tensor bbox_preds_tensor_b0(bbox_preds_net_b0, bbox_preds_net_b0->getDimensionType());bbox_preds_net_b0->copyToHostTensor(&bbox_preds_tensor_b0);auto rois_ptr_b0 = bbox_preds_tensor_b0.host<float>();std::cout << std::string(bbox_id0) << bbox_preds_tensor_b0.height() << " " << bbox_preds_tensor_b0.width() << " "<< bbox_preds_tensor_b0.channel() << " " << bbox_preds_tensor_b0.elementSize() << std::endl;bbox_preds_channels = bbox_preds_tensor_b0.channel();for (int i = 0; i < bbox_preds_tensor_b0.elementSize(); i++) {vec_temp.emplace_back(rois_ptr_b0[i]);}vec_bbox_preds.emplace_back(vec_temp);vec_temp.clear();std::vector<float>().swap(vec_temp);std::cout << std::endl;auto vec_bbox_net_b1 = net->getSessionOutput(session, bbox_id1);MNN::Tensor vec_bbox_tensor_b1(vec_bbox_net_b1, vec_bbox_net_b1->getDimensionType());vec_bbox_net_b1->copyToHostTensor(&vec_bbox_tensor_b1);auto vec_bbox_ptr_b1 = vec_bbox_tensor_b1.host<float>();std::cout << std::string(bbox_id1) << vec_bbox_tensor_b1.height() << " " << vec_bbox_tensor_b1.width() << " "<< vec_bbox_tensor_b1.channel() << " " << vec_bbox_tensor_b1.elementSize() << std::endl;for (int i = 0; i < vec_bbox_tensor_b1.elementSize(); i++) {vec_temp.emplace_back(vec_bbox_ptr_b1[i]);}vec_bbox_preds.emplace_back(vec_temp);vec_temp.clear();std::vector<float>().swap(vec_temp);std::cout << std::endl;auto vec_bbox_net_b2 = net->getSessionOutput(session, bbox_id2);MNN::Tensor vec_bbox_tensor_b2(vec_bbox_net_b2, vec_bbox_net_b2->getDimensionType());vec_bbox_net_b2->copyToHostTensor(&vec_bbox_tensor_b2);auto rois_ptr_b2 = vec_bbox_tensor_b2.host<float>();std::cout << std::string(bbox_id2) << vec_bbox_tensor_b2.height() << " " << vec_bbox_tensor_b2.width() << " "<< vec_bbox_tensor_b2.channel() << " " << vec_bbox_tensor_b2.elementSize() << std::endl;for (int i = 0; i < vec_bbox_tensor_b2.elementSize(); i++) {vec_temp.emplace_back(rois_ptr_b2[i]);}vec_bbox_preds.emplace_back(vec_temp);vec_temp.clear();std::vector<float>().swap(vec_temp);std::cout << std::endl;auto bbox_preds_net_b3 = net->getSessionOutput(session, bbox_id3);MNN::Tensor bbox_preds_tensor_b3(bbox_preds_net_b3, bbox_preds_net_b3->getDimensionType());bbox_preds_net_b3->copyToHostTensor(&bbox_preds_tensor_b3);auto bbox_preds_ptr_b3 = bbox_preds_tensor_b3.host<float>();std::cout << std::string(bbox_id3) << bbox_preds_tensor_b3.height() << " " << bbox_preds_tensor_b3.width() << " "<< bbox_preds_tensor_b3.channel() << " " << bbox_preds_tensor_b3.elementSize() << std::endl;for (int i = 0; i < bbox_preds_tensor_b3.elementSize(); i++) {vec_temp.emplace_back(bbox_preds_ptr_b3[i]);}vec_bbox_preds.emplace_back(vec_temp);vec_temp.clear();std::vector<float>().swap(vec_temp);std::cout << std::endl;auto bbox_preds_net_b4 = net->getSessionOutput(session, bbox_id4);MNN::Tensor bbox_preds_tensor_b4(bbox_preds_net_b4, bbox_preds_net_b4->getDimensionType());bbox_preds_net_b4->copyToHostTensor(&bbox_preds_tensor_b4);auto bbox_preds_ptr_b4 = bbox_preds_tensor_b4.host<float>();std::cout << std::string(bbox_id4) << bbox_preds_tensor_b4.height() << " " << bbox_preds_tensor_b4.width() << " "<< bbox_preds_tensor_b4.channel() << " " << bbox_preds_tensor_b4.elementSize() << std::endl;for (int i = 0; i < bbox_preds_tensor_b4.elementSize(); i++) {vec_temp.emplace_back(bbox_preds_ptr_b4[i]);}vec_bbox_preds.emplace_back(vec_temp);vec_temp.clear();std::vector<float>().swap(vec_temp);std::vector<std::vector<float>> vec_rois;int rois_channels = 0;std::cout << std::endl;auto rois_net_c0 = net->getSessionOutput(session, rois_id0);MNN::Tensor rois_tensor_c0(rois_net_c0, rois_net_c0->getDimensionType());rois_net_c0->copyToHostTensor(&rois_tensor_c0);auto rois_ptr_c0 = rois_tensor_c0.host<float>();std::cout << std::string(rois_id0) << rois_tensor_c0.height() << " " << rois_tensor_c0.width() << " "<< rois_tensor_c0.channel() << " " << rois_tensor_c0.elementSize() << std::endl;rois_channels = rois_tensor_c0.channel();for (int i = 0; i < rois_tensor_c0.elementSize(); i++) {vec_temp.emplace_back(rois_ptr_c0[i]);}vec_rois.emplace_back(vec_temp);vec_temp.clear();std::vector<float>().swap(vec_temp);std::cout << std::endl;auto rois_net_c1 = net->getSessionOutput(session, rois_id1);MNN::Tensor rois_tensor_c1(rois_net_c1, rois_net_c1->getDimensionType());rois_net_c1->copyToHostTensor(&rois_tensor_c1);auto rois_ptr_c1 = rois_tensor_c1.host<float>();std::cout << std::string(rois_id1) << rois_tensor_c1.height() << " " << rois_tensor_c1.width() << " "<< rois_tensor_c1.channel() << " " << rois_tensor_c1.elementSize() << std::endl;for (int i = 0; i < rois_tensor_c1.elementSize(); i++) {vec_temp.emplace_back(rois_ptr_c1[i]);}vec_rois.emplace_back(vec_temp);vec_temp.clear();std::vector<float>().swap(vec_temp);std::cout << std::endl;auto rois_net_c2 = net->getSessionOutput(session, rois_id2);MNN::Tensor rois_tensor_c2(rois_net_c2, rois_net_c2->getDimensionType());rois_net_c2->copyToHostTensor(&rois_tensor_c2);auto rois_ptr_c2 = rois_tensor_c2.host<float>();std::cout << std::string(rois_id2) << rois_tensor_c2.height() << " " << rois_tensor_c2.width() << " "<< rois_tensor_c2.channel() << " " << rois_tensor_c2.elementSize() << std::endl;for (int i = 0; i < rois_tensor_c2.elementSize(); i++) {vec_temp.emplace_back(rois_ptr_c2[i]);}vec_rois.emplace_back(vec_temp);vec_temp.clear();std::vector<float>().swap(vec_temp);std::cout << std::endl;auto rois_net_c3 = net->getSessionOutput(session, rois_id3);MNN::Tensor rois_tensor_c3(rois_net_c3, rois_net_c3->getDimensionType());rois_net_c3->copyToHostTensor(&rois_tensor_c3);auto rois_ptr_c3 = rois_tensor_c3.host<float>();std::cout << std::string(rois_id3) << rois_tensor_c3.height() << " " << rois_tensor_c3.width() << " "<< rois_tensor_c3.channel() << " " << rois_tensor_c3.elementSize() << std::endl;for (int i = 0; i < rois_tensor_c3.elementSize(); i++) {vec_temp.emplace_back(rois_ptr_c3[i]);}vec_rois.emplace_back(vec_temp);vec_temp.clear();std::vector<float>().swap(vec_temp);std::cout << std::endl;auto rois_net_c4 = net->getSessionOutput(session, rois_id4);MNN::Tensor rois_tensor_c4(rois_net_c4, rois_net_c4->getDimensionType());rois_net_c4->copyToHostTensor(&rois_tensor_c4);auto rois_ptr_c4 = rois_tensor_c4.host<float>();std::cout << std::string(rois_id4)<< rois_tensor_c4.height() << " " << rois_tensor_c4.width() << " "<< rois_tensor_c4.channel() << " " << rois_tensor_c4.elementSize() << std::endl;for (int i = 0; i < rois_tensor_c4.elementSize(); i++) {vec_temp.emplace_back(rois_ptr_c4[i]);}vec_rois.emplace_back(vec_temp);vec_temp.clear();std::vector<float>().swap(vec_temp);// 写的尽量不依赖MNN ，还要进行NCNN的转化std::vector<int> keep;std::vector<std::vector<float>> keep_box;std::vector<float> keep_score;std::vector<int> keep_label;std::vector<Ploy> keep_box_dets;int nms_pre = 2000;int min_bbox_size = 0;float score_thr = 0.05;float iou_thr = 0.1;int max_pre_img = 2000;bool rescale = true;bool with_nms = true;bool use_sigmoid_cls = true;get_bboxes_single(vec_cls_scores,vec_bbox_preds,vec_rois,scores_shape_0,keep_box,keep_score,keep_label,img_width,img_height,nms_pre,cls_out_channels,bbox_preds_channels,rois_channels,rescale,with_nms,use_sigmoid_cls,scale_factor,score_thr,iou_thr,max_pre_img);float font_scale = 0.25;cv::Scalar bbox_color = cv::Scalar(0, 0, 255);cv::Scalar text_color = cv::Scalar(0, 255, 0);std::vector<std::vector<cv::Point>> contours;std::cout << keep_box.size() << std::endl;for (int i = 0; i < keep_box.size(); i++) {std::vector<cv::Point> contours_item;float p1_0 = 0, p1_1 = 0, p2_0 = 0, p2_1 = 0, p3_0 = 0, p3_1 = 0, p4_0 = 0, p4_1 = 0, xc = 0, yc = 0;convet_point(keep_box[i], p1_0, p1_1, p2_0, p2_1, p3_0, p3_1, p4_0, p4_1);contours_item.emplace_back(cv::Point(p1_0, p1_1));contours_item.emplace_back(cv::Point(p2_0, p2_1));contours_item.emplace_back(cv::Point(p3_0, p3_1));contours_item.emplace_back(cv::Point(p4_0, p4_1));contours.emplace_back(contours_item);xc = keep_box[i][0];yc = keep_box[i][1];cv::drawContours(img, contours, -1, bbox_color, 2);char text[256];sprintf(text, "%s %.2f%%", classes[keep_label[i]], keep_score[i]);int baseLine = 0;cv::Size label_size = cv::getTextSize(text, cv::FONT_HERSHEY_SIMPLEX, font_scale, 1, &baseLine);int text_width = label_size.width;int text_height = label_size.height;cv::rectangle(img, cv::Point(int(xc), int(yc) - text_height - 2), cv::Point(int(xc) + text_width, int(yc) + 3),cv::Scalar(0, 128, 0), -1);cv::putText(img, text, cv::Point(int(xc), int(yc)),cv::FONT_HERSHEY_SIMPLEX, font_scale, text_color);contours.clear();std::vector<std::vector<cv::Point>>().swap(contours);}cv::imshow("image", img);cv::imwrite("image.jpg", img);cv::waitKey(0);return 0;
}c++ 的测试结果
代码也放在云盘中linux+clion ，优化自己搞，商用版本不对外开放
如果要训练hsrc数据集，因为我使用rolabelimg标注的，直接该了hrsc.py的读取方式，去适应我的数据集训练，记得将/home/ubuntu/mmrotate/configs/_base_/datasets/hrsc.py 修改３个地方 classwise=True,
# Copyright (c) OpenMMLab. All rights reserved.
import os.path as osp
import xml.etree.ElementTree as ET
from collections import OrderedDictimport mmcv
import numpy as np
from mmcv import print_log
from mmdet.datasets import CustomDataset
from PIL import Imagefrom mmrotate.core import eval_rbbox_map, obb2poly_np, poly2obb_np
from .builder import ROTATED_DATASETS@ROTATED_DATASETS.register_module()
class HRSCDataset(CustomDataset):"""HRSC dataset for detection.Args:ann_file (str): Annotation file path.pipeline (list[dict]): Processing pipeline.img_subdir (str): Subdir where images are stored. Default: JPEGImages.ann_subdir (str): Subdir where annotations are. Default: Annotations.classwise (bool): Whether to use all classes or only ship.version (str, optional): Angle representations. Defaults to 'oc'."""CLASSES = NoneHRSC_CLASS = ('a','b' )HRSC_CLASSES = ('a','b' )HRSC_CLASSES_ID = ('a','b')PALETTE = [(0, 255, 0),]CLASSWISE_PALETTE = [(220, 20, 60), (119, 11, 32),(0, 0, 142), (0, 0, 230), (106, 0, 228), (0, 60, 100),(0, 80, 100), (0, 0, 70), (0, 0, 192), (250, 170, 30),(100, 170, 30), (220, 220, 0), (175, 116, 175),(250, 0, 30), (165, 42, 42), (255, 77, 255),(0, 226, 252), (182, 182, 255), (0, 82, 0),(120, 166, 157), (110, 76, 0), (174, 57, 255),(199, 100, 0), (72, 0, 118), (255, 179, 240),(0, 125, 92), (209, 0, 151), (188, 208, 182),(0, 220, 176), (255, 99, 164), (92, 0, 73)]def __init__(self,ann_file,pipeline,img_subdir='JPEGImages',ann_subdir='Annotations',classwise=False,version='oc',**kwargs):self.img_subdir = img_subdirself.ann_subdir = ann_subdirself.classwise = classwiseself.version = versionif self.classwise:HRSCDataset.PALETTE = HRSCDataset.CLASSWISE_PALETTEHRSCDataset.CLASSES = self.HRSC_CLASSESself.catid2label = {( cls_id): ifor i, cls_id in enumerate(self.HRSC_CLASSES_ID)}else:HRSCDataset.CLASSES = self.HRSC_CLASS# self.cat2label = {cat: i for i, cat in enumerate(self.CLASSES)}super(HRSCDataset, self).__init__(ann_file, pipeline, **kwargs)def load_annotations(self, ann_file):"""Load annotation from XML style ann_file.Args:ann_file (str): Path of Imageset file.Returns:list[dict]: Annotation info from XML file."""data_infos = []img_ids = mmcv.list_from_file(ann_file)for img_id in img_ids:data_info = {}filename = osp.join(self.img_subdir, f'{img_id}.jpg')data_info['filename'] = f'{img_id}.jpg'xml_path = osp.join(self.img_prefix, self.ann_subdir,f'{img_id}.xml')tree = ET.parse(xml_path)root = tree.getroot()width = int(root.find("size").find('width').text)height = int(root.find("size").find('height').text)if width is None or height is None:img_path = osp.join(self.img_prefix, filename)img = Image.open(img_path)width, height = img.sizedata_info['width'] = widthdata_info['height'] = heightdata_info['ann'] = {}gt_bboxes = []gt_labels = []gt_polygons = []gt_headers = []gt_bboxes_ignore = []gt_labels_ignore = []gt_polygons_ignore = []gt_headers_ignore = []for obj in root.findall('object'):robndbox=obj.find("robndbox")if self.classwise:class_id = obj.find('name').textlabel = self.catid2label.get(class_id)if label is None:continueelse:label = 0a=eval(robndbox.find('cx').text)# Add an extra score to use obb2poly_npbbox = np.array([[float(eval(robndbox.find('cx').text)),float(eval(robndbox.find('cy').text)),float(eval(robndbox.find('w').text)),float(eval(robndbox.find('h').text)),float(eval(robndbox.find('angle').text)), 0]],dtype=np.float32)polygon = obb2poly_np(bbox, 'le90')[0, :-1].astype(np.float32)if self.version != 'le90':bbox = np.array(poly2obb_np(polygon, self.version), dtype=np.float32)else:bbox = bbox[0, :-1]head = np.array([int(eval(robndbox.find('cx').text)),int(eval(robndbox.find('cy').text))],dtype=np.int64)gt_bboxes.append(bbox)gt_labels.append(label)gt_polygons.append(polygon)gt_headers.append(head)if gt_bboxes:data_info['ann']['bboxes'] = np.array(gt_bboxes, dtype=np.float32)data_info['ann']['labels'] = np.array(gt_labels, dtype=np.int64)data_info['ann']['polygons'] = np.array(gt_polygons, dtype=np.float32)data_info['ann']['headers'] = np.array(gt_headers, dtype=np.int64)else:data_info['ann']['bboxes'] = np.zeros((0, 5), dtype=np.float32)data_info['ann']['labels'] = np.array([], dtype=np.int64)data_info['ann']['polygons'] = np.zeros((0, 8),dtype=np.float32)data_info['ann']['headers'] = np.zeros((0, 2),dtype=np.float32)if gt_polygons_ignore:data_info['ann']['bboxes_ignore'] = np.array(gt_bboxes_ignore, dtype=np.float32)data_info['ann']['labels_ignore'] = np.array(gt_labels_ignore, dtype=np.int64)data_info['ann']['polygons_ignore'] = np.array(gt_polygons_ignore, dtype=np.float32)data_info['ann']['headers_ignore'] = np.array(gt_headers_ignore, dtype=np.float32)else:data_info['ann']['bboxes_ignore'] = np.zeros((0, 5),dtype=np.float32)data_info['ann']['labels_ignore'] = np.array([],dtype=np.int64)data_info['ann']['polygons_ignore'] = np.zeros((0, 8), dtype=np.float32)data_info['ann']['headers_ignore'] = np.zeros((0, 2),dtype=np.float32)data_infos.append(data_info)return data_infosdef _filter_imgs(self):"""Filter images without ground truths."""valid_inds = []for i, data_info in enumerate(self.data_infos):if (not self.filter_empty_gtor data_info['ann']['labels'].size > 0):valid_inds.append(i)return valid_indsdef evaluate(self,results,metric='mAP',logger=None,proposal_nums=(100, 300, 1000),iou_thr=[0.5, 0.55, 0.6, 0.65, 0.7, 0.75, 0.8, 0.85, 0.9, 0.95],scale_ranges=None,use_07_metric=True,nproc=4):"""Evaluate the dataset.Args:results (list): Testing results of the dataset.metric (str | list[str]): Metrics to be evaluated.logger (logging.Logger | None | str): Logger used for printingrelated information during evaluation. Default: None.proposal_nums (Sequence[int]): Proposal number used for evaluatingrecalls, such as recall@100, recall@1000.Default: (100, 300, 1000).iou_thr (float | list[float]): IoU threshold. It must be a floatwhen evaluating mAP, and can be a list when evaluating recall.Default: 0.5.scale_ranges (list[tuple] | None): Scale ranges for evaluating mAP.Default: None.use_07_metric (bool): Whether to use the voc07 metric.nproc (int): Processes used for computing TP and FP.Default: 4."""if not isinstance(metric, str):assert len(metric) == 1metric = metric[0]allowed_metrics = ['mAP', 'recall']if metric not in allowed_metrics:raise KeyError(f'metric {metric} is not supported')annotations = [self.get_ann_info(i) for i in range(len(self))]eval_results = OrderedDict()iou_thrs = [iou_thr] if isinstance(iou_thr, float) else iou_thrif metric == 'mAP':assert isinstance(iou_thrs, list)mean_aps = []for iou_thr in iou_thrs:print_log(f'\n{"-" * 15}iou_thr: {iou_thr}{"-" * 15}')mean_ap, _ = eval_rbbox_map(results,annotations,scale_ranges=scale_ranges,iou_thr=iou_thr,use_07_metric=use_07_metric,dataset=self.CLASSES,logger=logger,nproc=nproc)mean_aps.append(mean_ap)eval_results[f'AP{int(iou_thr * 100):02d}'] = round(mean_ap, 3)eval_results['mAP'] = sum(mean_aps) / len(mean_aps)eval_results.move_to_end('mAP', last=False)elif metric == 'recall':raise NotImplementedErrorreturn eval_resultsncnn待补充
42、使用mmrotate中k3det进行旋转目标检测，并进行mnn部署和ncnn部署相关推荐
- mmrotate旋转目标检测框架的学习与使用
目录 前言 一.环境配置 1. 下载checkpoint文件 2. 运行demo 二.制作自己的数据集 1. 标注数据 2. 标签格式转换 3. 可视化数据集 4. 数据集裁剪 三. 修改配置文件 1 ...
- mmrotate旋转目标检测框架从环境配置到训练自己的数据集
mmrotate旋转目标检测框架从环境配置到训练自己的数据集 最近有旋转目标检测的需求了,在寻找旋转目标检测算法的过程中,发现已经有人提前测评了各类旋转目标检测框架了,结果是mmrorate效果最好. ...
- 基于mmrotate的旋转目标检测入门详解
一.旋转目标检测方法对比 1 当前前沿方法的对比 首先我们打开papers with code 网站 https://paperswithcode.com/ 我们在搜索栏输入 oriented obj ...
- 旋转目标检测：The KFIoU Loss for Rotated Object Detection（Under review in ICLR 2022）
关键词:KFIoU, 倾斜IoU(SkewIoU) 参考博客: https://zhuanlan.zhihu.com/p/447286823 论文原文下载: https://openreview.ne ...
- 干货 | 一文总结旋转目标检测全面综述：论文方法与代码
点击上方"小白学视觉",选择加"星标"或"置顶" 重磅干货,第一时间送达 本文转自|计算机视觉联盟 1.RRPN(两阶段文字检测 华科白翔组 ...
- AAAI 2021 | 用于旋转目标检测的动态锚框学习策略
点上方蓝字计算机视觉联盟获取更多干货 在右上方 ··· 设为星标 ★,与你不见不散 仅作学术分享,不代表本公众号立场,侵权联系删除 转载于:AI科技评论 AI博士笔记系列推荐 周志华<机器学习& ...
- 旋转目标检测 校准的深度特征用于目标检测SSA
目录 旋转目标检测rotation-yolov5 旋转目标检测综述 SSA旋转目标检测 旋转目标检测rotation-yolov5 旋转目标检测rotation-yolov5笔记_AI视觉网奇的博客- ...
- SCRDet：Towards More Robust Detection for Small, Cluttered and Rotated Objects（摇杆旋转目标检测方法）
文章目录 SCRDet:Towards More Robust Detection for Small, Cluttered and Rotated Objects(遥感旋转目标检测方法) 一.论文摘 ...
- 睿智的目标检测——YoloV7-Tiny-OBB旋转目标检测算法部署
YoloV7-OBB旋转目标检测算法部署 学习前言 源码下载 安装TensorRT 1.TensorRT简介 2.下载TensorRT 3.TensorRT安装 安装torch2trt 1.torch ...
最新文章
- python怎么导入文件-Python模块导入详解
- 1.3 字符串拼接（连接）
- 【Java】 5.7 多态
- python shape函数_Perlin噪声和Python的ctypes
- GPU 编程入门到精通（五）之 GPU 程序优化进阶
- spark的三种运行模式以及yarn-client和yarn-cluster在提交命令上的区别
- Apache Shiro第1部分–基础
- oracle 11g job创建,Oracle 11g 手动创建库完整实施过程
- thinkphp连mysql增删改查_ThinkPHP5.1框架数据库链接和增删改查操作示例
- （一）linux下hadoop安装配置
- border-radius几种写法的原理剖析
- 如何批量保存html,如何批量保存网页信息
- word简历排版技巧
- 微步在线【威胁情报】
- 数据挖掘导论 N个考试常用的问题
- 初为人师[/size]
- C语言基础（十三）系统错误信息
- selenium报错信息-- Python 中 'unicodeescape' codec can't decode bytes in position XXX: trun错误解决方案...
- 一次使用 Go 语言编写脚本的经历
- java计算机毕业设计新能源汽车租赁管理系统源程序+mysql+系统+lw文档+远程调试