DIOR数据集转COCO格式 paddlepaddle

百度paddlepaddle框架自带了转换COCO格式代码，设置参数即可。源码地址为https://github.com/PaddlePaddle/PaddleDetection
代码如下，文件名为x2coco.py：

import argparse
import glob
import json
import os
import os.path as osp
import sys
import shutil
import xml.etree.ElementTree as ET
from tqdm import tqdm
import reimport numpy as np
import PIL.ImageDrawlabel_to_num = {}
categories_list = []
labels_list = []class MyEncoder(json.JSONEncoder):def default(self, obj):if isinstance(obj, np.integer):return int(obj)elif isinstance(obj, np.floating):return float(obj)elif isinstance(obj, np.ndarray):return obj.tolist()else:return super(MyEncoder, self).default(obj)def images_labelme(data, num):image = {}image['height'] = data['imageHeight']image['width'] = data['imageWidth']image['id'] = num + 1if '\\' in data['imagePath']:image['file_name'] = data['imagePath'].split('\\')[-1]else:image['file_name'] = data['imagePath'].split('/')[-1]return imagedef images_cityscape(data, num, img_file):image = {}image['height'] = data['imgHeight']image['width'] = data['imgWidth']image['id'] = num + 1image['file_name'] = img_filereturn imagedef categories(label, labels_list):category = {}category['supercategory'] = 'component'category['id'] = len(labels_list) + 1category['name'] = labelreturn categorydef annotations_rectangle(points, label, image_num, object_num, label_to_num):annotation = {}seg_points = np.asarray(points).copy()seg_points[1, :] = np.asarray(points)[2, :]seg_points[2, :] = np.asarray(points)[1, :]annotation['segmentation'] = [list(seg_points.flatten())]annotation['iscrowd'] = 0annotation['image_id'] = image_num + 1annotation['bbox'] = list(map(float, [points[0][0], points[0][1], points[1][0] - points[0][0], points[1][1] - points[0][1]]))annotation['area'] = annotation['bbox'][2] * annotation['bbox'][3]annotation['category_id'] = label_to_num[label]annotation['id'] = object_num + 1return annotationdef annotations_polygon(height, width, points, label, image_num, object_num,label_to_num):annotation = {}annotation['segmentation'] = [list(np.asarray(points).flatten())]annotation['iscrowd'] = 0annotation['image_id'] = image_num + 1annotation['bbox'] = list(map(float, get_bbox(height, width, points)))annotation['area'] = annotation['bbox'][2] * annotation['bbox'][3]annotation['category_id'] = label_to_num[label]annotation['id'] = object_num + 1return annotationdef get_bbox(height, width, points):polygons = pointsmask = np.zeros([height, width], dtype=np.uint8)mask = PIL.Image.fromarray(mask)xy = list(map(tuple, polygons))PIL.ImageDraw.Draw(mask).polygon(xy=xy, outline=1, fill=1)mask = np.array(mask, dtype=bool)index = np.argwhere(mask == 1)rows = index[:, 0]clos = index[:, 1]left_top_r = np.min(rows)left_top_c = np.min(clos)right_bottom_r = np.max(rows)right_bottom_c = np.max(clos)return [left_top_c, left_top_r, right_bottom_c - left_top_c,right_bottom_r - left_top_r]def deal_json(ds_type, img_path, json_path):data_coco = {}images_list = []annotations_list = []image_num = -1object_num = -1for img_file in os.listdir(img_path):img_label = os.path.splitext(img_file)[0]if img_file.split('.')[-1] not in ['bmp', 'jpg', 'jpeg', 'png', 'JPEG', 'JPG', 'PNG']:continuelabel_file = osp.join(json_path, img_label + '.json')print('Generating dataset from:', label_file)image_num = image_num + 1with open(label_file) as f:data = json.load(f)if ds_type == 'labelme':images_list.append(images_labelme(data, image_num))elif ds_type == 'cityscape':images_list.append(images_cityscape(data, image_num, img_file))if ds_type == 'labelme':for shapes in data['shapes']:object_num = object_num + 1label = shapes['label']if label not in labels_list:categories_list.append(categories(label, labels_list))labels_list.append(label)label_to_num[label] = len(labels_list)p_type = shapes['shape_type']if p_type == 'polygon':points = shapes['points']annotations_list.append(annotations_polygon(data['imageHeight'], data['imageWidth'], points, label, image_num,object_num, label_to_num))if p_type == 'rectangle':(x1, y1), (x2, y2) = shapes['points']x1, x2 = sorted([x1, x2])y1, y2 = sorted([y1, y2])points = [[x1, y1], [x2, y2], [x1, y2], [x2, y1]]annotations_list.append(annotations_rectangle(points, label, image_num,object_num, label_to_num))elif ds_type == 'cityscape':for shapes in data['objects']:object_num = object_num + 1label = shapes['label']if label not in labels_list:categories_list.append(categories(label, labels_list))labels_list.append(label)label_to_num[label] = len(labels_list)points = shapes['polygon']annotations_list.append(annotations_polygon(data['imgHeight'], data['imgWidth'], points, label, image_num, object_num,label_to_num))data_coco['images'] = images_listdata_coco['categories'] = categories_listdata_coco['annotations'] = annotations_listreturn data_cocodef voc_get_label_anno(ann_dir_path, ann_ids_path, labels_path):with open(labels_path, 'r') as f:labels_str = f.read().split()labels_ids = list(range(1, len(labels_str) + 1))with open(ann_ids_path, 'r') as f:ann_ids = f.read().split()ann_paths = []for aid in ann_ids:if aid.endswith('xml'):ann_path = os.path.join(ann_dir_path, aid)else:ann_path = os.path.join(ann_dir_path, aid + '.xml')ann_paths.append(ann_path)return dict(zip(labels_str, labels_ids)), ann_pathsdef voc_get_image_info(annotation_root, im_id):filename = annotation_root.findtext('filename')assert filename is not Noneimg_name = os.path.basename(filename)size = annotation_root.find('size')width = float(size.findtext('width'))height = float(size.findtext('height'))image_info = {'file_name': filename,'height': height,'width': width,'id': im_id}return image_infodef voc_get_coco_annotation(obj, label2id):label = obj.findtext('name')assert label in label2id, "label is not in label2id."category_id = label2id[label]bndbox = obj.find('bndbox')xmin = float(bndbox.findtext('xmin'))ymin = float(bndbox.findtext('ymin'))xmax = float(bndbox.findtext('xmax'))ymax = float(bndbox.findtext('ymax'))assert xmax > xmin and ymax > ymin, "Box size error."o_width = xmax - xmino_height = ymax - yminanno = {'area': o_width * o_height,'iscrowd': 0,'bbox': [xmin, ymin, o_width, o_height],'category_id': category_id,'ignore': 0,}return annodef voc_xmls_to_cocojson(annotation_paths, label2id, output_dir, output_file):output_json_dict = {"images": [],"type": "instances","annotations": [],"categories": []}bnd_id = 1  # bounding box start idim_id = 0print('Start converting !')for a_path in tqdm(annotation_paths):# Read annotation xmlann_tree = ET.parse(a_path)ann_root = ann_tree.getroot()img_info = voc_get_image_info(ann_root, im_id)output_json_dict['images'].append(img_info)for obj in ann_root.findall('object'):ann = voc_get_coco_annotation(obj=obj, label2id=label2id)ann.update({'image_id': im_id, 'id': bnd_id})output_json_dict['annotations'].append(ann)bnd_id = bnd_id + 1im_id += 1for label, label_id in label2id.items():category_info = {'supercategory': 'none', 'id': label_id, 'name': label}output_json_dict['categories'].append(category_info)output_file = os.path.join(output_dir, output_file)with open(output_file, 'w') as f:output_json = json.dumps(output_json_dict)f.write(output_json)def main():parser = argparse.ArgumentParser(formatter_class=argparse.ArgumentDefaultsHelpFormatter)parser.add_argument('--dataset_type',help='the type of dataset, can be `voc`, `labelme` or `cityscape`')parser.add_argument('--json_input_dir', help='input annotated directory')parser.add_argument('--image_input_dir', help='image directory')parser.add_argument('--output_dir', help='output dataset directory', default='./')parser.add_argument('--train_proportion',help='the proportion of train dataset',type=float,default=1.0)parser.add_argument('--val_proportion',help='the proportion of validation dataset',type=float,default=0.0)parser.add_argument('--test_proportion',help='the proportion of test dataset',type=float,default=0.0)parser.add_argument('--voc_anno_dir',help='In Voc format dataset, path to annotation files directory.',type=str,default=None)parser.add_argument('--voc_anno_list',help='In Voc format dataset, path to annotation files ids list.',type=str,default=None)parser.add_argument('--voc_label_list',help='In Voc format dataset, path to label list. The content of each line is a category.',type=str,default=None)parser.add_argument('--voc_out_name',type=str,default='voc.json',help='In Voc format dataset, path to output json file')args = parser.parse_args()try:assert args.dataset_type in ['voc', 'labelme', 'cityscape']except AssertionError as e:print('Now only support the voc, cityscape dataset and labelme dataset!!')os._exit(0)if args.dataset_type == 'voc':assert args.voc_anno_dir and args.voc_anno_list and args.voc_label_listlabel2id, ann_paths = voc_get_label_anno(args.voc_anno_dir, args.voc_anno_list, args.voc_label_list)voc_xmls_to_cocojson(annotation_paths=ann_paths,label2id=label2id,output_dir=args.output_dir,output_file=args.voc_out_name)else:try:assert os.path.exists(args.json_input_dir)except AssertionError as e:print('The json folder does not exist!')os._exit(0)try:assert os.path.exists(args.image_input_dir)except AssertionError as e:print('The image folder does not exist!')os._exit(0)try:assert abs(args.train_proportion + args.val_proportion \+ args.test_proportion - 1.0) < 1e-5except AssertionError as e:print('The sum of pqoportion of training, validation and test datase must be 1!')os._exit(0)# Allocate the dataset.total_num = len(glob.glob(osp.join(args.json_input_dir, '*.json')))if args.train_proportion != 0:train_num = int(total_num * args.train_proportion)out_dir = args.output_dir + '/train'if not os.path.exists(out_dir):os.makedirs(out_dir)else:train_num = 0if args.val_proportion == 0.0:val_num = 0test_num = total_num - train_numout_dir = args.output_dir + '/test'if args.test_proportion != 0.0 and not os.path.exists(out_dir):os.makedirs(out_dir)else:val_num = int(total_num * args.val_proportion)test_num = total_num - train_num - val_numval_out_dir = args.output_dir + '/val'if not os.path.exists(val_out_dir):os.makedirs(val_out_dir)test_out_dir = args.output_dir + '/test'if args.test_proportion != 0.0 and not os.path.exists(test_out_dir):os.makedirs(test_out_dir)count = 1for img_name in os.listdir(args.image_input_dir):if count <= train_num:if osp.exists(args.output_dir + '/train/'):shutil.copyfile(osp.join(args.image_input_dir, img_name),osp.join(args.output_dir + '/train/', img_name))else:if count <= train_num + val_num:if osp.exists(args.output_dir + '/val/'):shutil.copyfile(osp.join(args.image_input_dir, img_name),osp.join(args.output_dir + '/val/', img_name))else:if osp.exists(args.output_dir + '/test/'):shutil.copyfile(osp.join(args.image_input_dir, img_name),osp.join(args.output_dir + '/test/', img_name))count = count + 1# Deal with the json files.if not os.path.exists(args.output_dir + '/annotations'):os.makedirs(args.output_dir + '/annotations')if args.train_proportion != 0:train_data_coco = deal_json(args.dataset_type,args.output_dir + '/train',args.json_input_dir)train_json_path = osp.join(args.output_dir + '/annotations','instance_train.json')json.dump(train_data_coco,open(train_json_path, 'w'),indent=4,cls=MyEncoder)if args.val_proportion != 0:val_data_coco = deal_json(args.dataset_type,args.output_dir + '/val',args.json_input_dir)val_json_path = osp.join(args.output_dir + '/annotations','instance_val.json')json.dump(val_data_coco,open(val_json_path, 'w'),indent=4,cls=MyEncoder)if args.test_proportion != 0:test_data_coco = deal_json(args.dataset_type,args.output_dir + '/test',args.json_input_dir)test_json_path = osp.join(args.output_dir + '/annotations','instance_test.json')json.dump(test_data_coco,open(test_json_path, 'w'),indent=4,cls=MyEncoder)if __name__ == '__main__':main()

首先需要将DIOR目录格式调整为标准的VOC2007格式，如下：
新建label_list_dior.txt文件，内容为DIOR数据集标签类别信息：

airplane
airport
baseballfield
basketballcourt
bridge
chimney
dam
Expressway-Service-area
Expressway-toll-station
golffield
groundtrackfield
harbor
overpass
ship
stadium
storagetank
tenniscourt
trainstation
vehicle
windmill

在PaddleDetection-release-2.1目录下打开终端，输入指令：

python tools/x2coco.py --dataset_type voc --voc_anno_dir dataset/voc/VOCdevkit/VOC2007/Annotations/ --voc_anno_list  dataset/voc/VOCdevkit/VOC2007/ImageSets/Main/train.txt --voc_label_list  dataset/voc/label_list_dior.txt --voc_out_name dior_train.json

其中 --dataset_type 为数据类型设置为voc
–voc_anno_dir 为voc格式数据标签文件存储位置，即Annotations文件夹位置
–voc_anno_list 为训练集、验证集、测试集划分信息，即train.txt，val.txt，test.txt文件位置
–voc_label_list 为voc数据标签类别信息，即label_list_dior.txt文件
–voc_out_name 为输出的coco_json文件名
运行完成即可得到dior_train.json文件：

DIOR数据集转COCO格式 paddlepaddle相关推荐

使用PaddleDetection自带脚本将自制labelme数据集转为coco格式
问题 PaddleDetection(ppdet)自带一个能将labelme标注的数据集转为coco格式的脚本,还能设置比例分配train.val和test数据集的比例.当前使用的ppdet版本为ve ...
将CityScapes数据集转换为COCO格式的实例分割数据集
##￥##2019年结束前最后两小时,写出本年度最后一篇博客,不是博主没地方去跨年,实在是Coding乐趣无穷ahhhh ##￥## COCO数据集是研究实例分割的通用数据集,但是针对道路环境的数据较 ...
python：VOC数据集转COCO格式
#图片与各自的xml文件放于同一文件夹 import tqdm import os import shutil import json import sys import xml.etree.Elem ...
【数据集格式】coco格式转txt格式
json_to_txt txt格式环境代码转后路径结果 txt格式训练tensorflow版的yolov3需要的数据格式是txt的,具体要求如下 xxx/xxx.jpg 18.19,6.32 ...
CVPPA彩色图片转COCO格式
本博客主要参考了:将彩色RGB分割标注图像数据集转换为COCO格式的json文件_Liu, Xu的博客-CSDN博客_json转rgb 1.将彩色图片转换为黑白mask import cv2 impo ...
coco格式的数据集进行训练
前几篇博客一直在使用网上别人写好的代码,按照航拍数据打标的.json,.mask,.yaml,原图片,四个文件来制作数据集(我认为是自定义格式的数据集),最近跟一个热心的网友交流(超级感谢他)决定改为 ...
【mmdetection小目标检测教程】三、使用sahi库切分高分辨率图片，一键生成coco格式数据集
[mmdetection小目标检测教程]三.使用sahi库切分高分辨率图片,一键生成coco格式数据集 1.确认是否需要切分图像 2.子图切分 (1)安装sahi库 (2)基于sahi切图本文我们将 ...
coco数据集（yoloV5格式）中生成子类数据集
从coco数据集(yoloV5格式)中生成子类数据集. import os from tqdm import tqdmnames = ['person', 'bicycle', 'car', 'mot ...
DAGM2007数据集扩增（COCO格式以及VOC格式的灰度图数据集都可用）
文章目录前言 COCO格式的数据集扩增代码 AugmentMethodClass.py DatasetAugment.py VOC格式的数据集扩增代码 AugmentMethodClass.py ...
制作自己的COCO格式数据集，附代码！
最近做了一个细胞检测的练习项目.之前的思路是参考其他大神的代码,后来发现其他人的代码有很多自定义的内容,包括读取的数据格式等等,小白表示看不懂所以改变思路,用最简单的方法--选择mmdetection ...

DIOR数据集转COCO格式 paddlepaddle

DIOR数据集转COCO格式 paddlepaddle相关推荐

最新文章

热门文章