voc数据集转coco数据集

由于voc是将有的xml文件都放在了Annotations文件夹下，但是coco数据集，是将train和val的数据信息分别放在一个文件里，并且都是json文件。

为了方便数据整理，文件夹自己设置为（coco数据集本来不是这种个数，但是为了自己方便转换，所以这样子建的文件夹格式）：

|---coco2021

|---ann_train2012

|---ann_val2012

|---Main_new(存放准备步骤得到的txt文件，其实就是复制了voc里的ImageSets里Main文件，然后重命名了)

|---VOC2012（里面存放的自己voc数据集，这里的voc数据集格式是标准的）

|---Annotations

|---ImageSets

|---JPEGImages

|---TempAnnotations

|---classes.txt

所以

第一步，是要将voc数据集里Annotations下的文件，先按照train和val分别放在两个文件夹里。
第二步，将两个文件夹里的xml文件再分别转换成coco数据集的json格式。

准备步骤：第一步的前提，需要有存放voc数据集的train和val的xml文件名的txt文件。

程序如下：

（说明：程序来源于Bubbliiiing的yolov5里的voc_annotation.py，b站有视频）

import os
import random
import xml.etree.ElementTree as ETimport numpy as npfrom utils.utils import get_classes#--------------------------------------------------------------------------------------------------------------------------------#
#   annotation_mode用于指定该文件运行时计算的内容
#   annotation_mode为0代表整个标签处理过程，包括获得VOCdevkit/VOC2007/ImageSets里面的txt以及训练用的2007_train.txt、2007_val.txt
#   annotation_mode为1代表获得VOCdevkit/VOC2007/ImageSets里面的txt
#   annotation_mode为2代表获得训练用的2007_train.txt、2007_val.txt
#--------------------------------------------------------------------------------------------------------------------------------#
annotation_mode     = 0
#-------------------------------------------------------------------#
#   必须要修改，用于生成2007_train.txt、2007_val.txt的目标信息
#   与训练和预测所用的classes_path一致即可
#   如果生成的2007_train.txt里面没有目标信息
#   那么就是因为classes没有设定正确
#   仅在annotation_mode为0和2的时候有效
#-------------------------------------------------------------------#
classes_path        = 'model_data/classes.txt'
#--------------------------------------------------------------------------------------------------------------------------------#
#   trainval_percent用于指定(训练集+验证集)与测试集的比例，默认情况下 (训练集+验证集):测试集 = 9:1
#   train_percent用于指定(训练集+验证集)中训练集与验证集的比例，默认情况下 训练集:验证集 = 9:1
#   仅在annotation_mode为0和1的时候有效
#--------------------------------------------------------------------------------------------------------------------------------#
trainval_percent    = 0.9
train_percent       = 0.9
#-------------------------------------------------------#
#   指向VOC数据集所在的文件夹
#   默认指向根目录下的VOC数据集
#-------------------------------------------------------#
VOCdevkit_path  = 'VOCdevkit'VOCdevkit_sets  = [('2007', 'train'), ('2007', 'val')]
classes, _      = get_classes(classes_path)#-------------------------------------------------------#
#   统计目标数量
#-------------------------------------------------------#
photo_nums  = np.zeros(len(VOCdevkit_sets))
nums        = np.zeros(len(classes))
def convert_annotation(year, image_id, list_file):in_file = open(os.path.join(VOCdevkit_path, 'VOC%s/Annotations/%s.xml'%(year, image_id)), encoding='utf-8')tree=ET.parse(in_file)root = tree.getroot()for obj in root.iter('object'):difficult = 0 if obj.find('difficult')!=None:difficult = obj.find('difficult').textcls = obj.find('name').textif cls not in classes or int(difficult)==1:continuecls_id = classes.index(cls)xmlbox = obj.find('bndbox')b = (int(float(xmlbox.find('xmin').text)), int(float(xmlbox.find('ymin').text)), int(float(xmlbox.find('xmax').text)), int(float(xmlbox.find('ymax').text)))list_file.write(" " + ",".join([str(a) for a in b]) + ',' + str(cls_id))nums[classes.index(cls)] = nums[classes.index(cls)] + 1if __name__ == "__main__":random.seed(0)if " " in os.path.abspath(VOCdevkit_path):raise ValueError("数据集存放的文件夹路径与图片名称中不可以存在空格，否则会影响正常的模型训练，请注意修改。")if annotation_mode == 0 or annotation_mode == 1:print("Generate txt in ImageSets.")xmlfilepath     = os.path.join(VOCdevkit_path, 'VOC2007/Annotations')saveBasePath    = os.path.join(VOCdevkit_path, 'VOC2007/ImageSets/Main')temp_xml        = os.listdir(xmlfilepath)total_xml       = []for xml in temp_xml:if xml.endswith(".xml"):total_xml.append(xml)num     = len(total_xml)  list    = range(num)  tv      = int(num*trainval_percent)  tr      = int(tv*train_percent)  trainval= random.sample(list,tv)  train   = random.sample(trainval,tr)  print("train and val size",tv)print("train size",tr)ftrainval   = open(os.path.join(saveBasePath,'trainval.txt'), 'w')  ftest       = open(os.path.join(saveBasePath,'test.txt'), 'w')  ftrain      = open(os.path.join(saveBasePath,'train.txt'), 'w')  fval        = open(os.path.join(saveBasePath,'val.txt'), 'w')  for i in list:  name=total_xml[i][:-4]+'\n'  if i in trainval:  ftrainval.write(name)  if i in train:  ftrain.write(name)  else:  fval.write(name)  else:  ftest.write(name)  ftrainval.close()  ftrain.close()  fval.close()  ftest.close()print("Generate txt in ImageSets done.")if annotation_mode == 0 or annotation_mode == 2:print("Generate 2007_train.txt and 2007_val.txt for train.")type_index = 0for year, image_set in VOCdevkit_sets:image_ids = open(os.path.join(VOCdevkit_path, 'VOC%s/ImageSets/Main/%s.txt'%(year, image_set)), encoding='utf-8').read().strip().split()list_file = open('%s_%s.txt'%(year, image_set), 'w', encoding='utf-8')for image_id in image_ids:list_file.write('%s/VOC%s/JPEGImages/%s.jpg'%(os.path.abspath(VOCdevkit_path), year, image_id))convert_annotation(year, image_id, list_file)list_file.write('\n')photo_nums[type_index] = len(image_ids)type_index += 1list_file.close()print("Generate 2007_train.txt and 2007_val.txt for train done.")def printTable(List1, List2):for i in range(len(List1[0])):print("|", end=' ')for j in range(len(List1)):print(List1[j][i].rjust(int(List2[j])), end=' ')print("|", end=' ')print()str_nums = [str(int(x)) for x in nums]tableData = [classes, str_nums]colWidths = [0]*len(tableData)len1 = 0for i in range(len(tableData)):for j in range(len(tableData[i])):if len(tableData[i][j]) > colWidths[i]:colWidths[i] = len(tableData[i][j])printTable(tableData, colWidths)if photo_nums[0] <= 500:print("训练集数量小于500，属于较小的数据量，请注意设置较大的训练世代（Epoch）以满足足够的梯度下降次数（Step）。")if np.sum(nums) == 0:print("在数据集中并未获得任何目标，请注意修改classes_path对应自己的数据集，并且保证标签名字正确，否则训练将会没有任何效果！")print("在数据集中并未获得任何目标，请注意修改classes_path对应自己的数据集，并且保证标签名字正确，否则训练将会没有任何效果！")print("在数据集中并未获得任何目标，请注意修改classes_path对应自己的数据集，并且保证标签名字正确，否则训练将会没有任何效果！")print("（重要的事情说三遍）。")

通过15行，可以获得几个txt文件，其中包含了train和val的txt文件，存放的是对应的训练集和验证集的 xml文件的名字。

然后就可以进行步骤一了。

步骤1：先建立两个文件夹，ann_train2012和ann_val2012，用于存放生成的对应的xml文件（将annnotations文件夹里的文件，根据上面脚本得到的两个txt文件，进行分开存放）。

要改的地方，就4个，就是写中文注释的地方。

ann_train2012存放对应的xml文件：

import os
import shutil# the path is you original file directory
# the newpath is the new directory
class BatchCopy():def __init__(self):self.path = '/****/coco2012/VOC2012/Annotations'   ####voc是将将所有xml文件都放在同一目录下self.newpath = '/***/coco2012/ann_train2012'    ####要将训练集的xml文件单独放一个目录def copy_file(self):filelist = os.listdir(self.path)  # file list in this directory# print(len(filelist))test_list = loadFileList()# print(len(test_list))for f in filelist:filedir = os.path.join(self.path, f)(shotname, extension) = os.path.splitext(f)if str(shotname) in test_list:# print('success')shutil.copyfile(str(filedir), os.path.join(self.newpath, f))# load the list of train/test file list
def loadFileList():filelist = []f = open("/***/coco2012/Main_new/new_train.txt", "r")    ##此txt文件，存放的是train的xml文件的名字（因为上述脚本得到的有train test val三个txt文件，此处我将train和test的txt文件内容进行合并，得到了new_train.txt，也就是不用test）lines = f.readlines()for line in lines:line = line.strip('\r\n')  # to remove the '\n' for test.txt, '\r\n' for tainval.txtline = str(line)filelist.append(line)f.close()# print(filelist)return filelistif __name__ == '__main__':demo = BatchCopy()demo.copy_file()filelist = os.listdir('/***/coco2012/ann_train2012')       ####训练集的xml文件单独放的目录# print(len(filelist))

ann_val2012存放对应的文件（其实就是把上面的程序修改了txt文件和保存地址）：

import os
import shutil# the path is you original file directory
# the newpath is the new directory
class BatchCopy():def __init__(self):self.path = '/****/coco2012/VOC2012/Annotations'   ####voc是将将所有xml文件都放在同一目录下self.newpath = '/***/coco2012/ann_val2012'    ####要将验证集的xml文件单独放一个目录def copy_file(self):filelist = os.listdir(self.path)  # file list in this directory# print(len(filelist))test_list = loadFileList()# print(len(test_list))for f in filelist:filedir = os.path.join(self.path, f)(shotname, extension) = os.path.splitext(f)if str(shotname) in test_list:# print('success')shutil.copyfile(str(filedir), os.path.join(self.newpath, f))# load the list of train/test file list
def loadFileList():filelist = []f = open("/***/coco2012/Main_new/val.txt", "r")    ##此txt文件，存放的是val的xml文件的名字lines = f.readlines()for line in lines:line = line.strip('\r\n')  # to remove the '\n' for test.txt, '\r\n' for tainval.txtline = str(line)filelist.append(line)f.close()# print(filelist)return filelistif __name__ == '__main__':demo = BatchCopy()demo.copy_file()filelist = os.listdir('/***/coco2012/ann_val2012')       ####验证集的xml文件单独放的目录# print(len(filelist))

步骤2：将xml文件转换成json文件

import xml.etree.ElementTree as ET
import os
import jsoncoco = dict()
coco['images'] = []
coco['type'] = 'instances'
coco['annotations'] = []
coco['categories'] = []category_set = dict()
image_set = set()category_item_id = -1
image_id = 20180000000
annotation_id = 0def addCatItem(name):global category_item_idcategory_item = dict()category_item['supercategory'] = 'none'category_item_id += 1category_item['id'] = category_item_idcategory_item['name'] = namecoco['categories'].append(category_item)category_set[name] = category_item_idreturn category_item_iddef addImgItem(file_name, size):global image_idif file_name is None:raise Exception('Could not find filename tag in xml file.')if size['width'] is None:raise Exception('Could not find width tag in xml file.')if size['height'] is None:raise Exception('Could not find height tag in xml file.')image_id += 1image_item = dict()image_item['id'] = image_idimage_item['file_name'] = file_nameimage_item['width'] = size['width']image_item['height'] = size['height']coco['images'].append(image_item)image_set.add(file_name)return image_iddef addAnnoItem(object_name, image_id, category_id, bbox):global annotation_idannotation_item = dict()annotation_item['segmentation'] = []seg = []# bbox[] is x,y,w,h# left_topseg.append(bbox[0])seg.append(bbox[1])# left_bottomseg.append(bbox[0])seg.append(bbox[1] + bbox[3])# right_bottomseg.append(bbox[0] + bbox[2])seg.append(bbox[1] + bbox[3])# right_topseg.append(bbox[0] + bbox[2])seg.append(bbox[1])annotation_item['segmentation'].append(seg)annotation_item['area'] = bbox[2] * bbox[3]annotation_item['iscrowd'] = 0annotation_item['ignore'] = 0annotation_item['image_id'] = image_idannotation_item['bbox'] = bboxannotation_item['category_id'] = category_idannotation_id += 1annotation_item['id'] = annotation_idcoco['annotations'].append(annotation_item)def parseXmlFiles(xml_path):for f in os.listdir(xml_path):if not f.endswith('.xml'):continuebndbox = dict()size = dict()current_image_id = Nonecurrent_category_id = Nonefile_name = Nonesize['width'] = Nonesize['height'] = Nonesize['depth'] = Nonexml_file = os.path.join(xml_path, f)print(xml_file)tree = ET.parse(xml_file)root = tree.getroot()if root.tag != 'annotation':raise Exception('pascal voc xml root element should be annotation, rather than {}'.format(root.tag))# elem is <folder>, <filename>, <size>, <object>for elem in root:current_parent = elem.tagcurrent_sub = Noneobject_name = Noneif elem.tag == 'folder':continueif elem.tag == 'filename':file_name = elem.textif file_name in category_set:raise Exception('file_name duplicated')# add img item only after parse <size> tagelif current_image_id is None and file_name is not None and size['width'] is not None:if file_name not in image_set:current_image_id = addImgItem(file_name, size)print('add image with {} and {}'.format(file_name, size))else:raise Exception('duplicated image: {}'.format(file_name))# subelem is <width>, <height>, <depth>, <name>, <bndbox>for subelem in elem:bndbox['xmin'] = Nonebndbox['xmax'] = Nonebndbox['ymin'] = Nonebndbox['ymax'] = Nonecurrent_sub = subelem.tagif current_parent == 'object' and subelem.tag == 'name':object_name = subelem.textif object_name not in category_set:current_category_id = addCatItem(object_name)else:current_category_id = category_set[object_name]elif current_parent == 'size':if size[subelem.tag] is not None:raise Exception('xml structure broken at size tag.')size[subelem.tag] = int(subelem.text)# option is <xmin>, <ymin>, <xmax>, <ymax>, when subelem is <bndbox>for option in subelem:if current_sub == 'bndbox':if bndbox[option.tag] is not None:raise Exception('xml structure corrupted at bndbox tag.')bndbox[option.tag] = int(option.text)# only after parse the <object> tagif bndbox['xmin'] is not None:if object_name is None:raise Exception('xml structure broken at bndbox tag')if current_image_id is None:raise Exception('xml structure broken at bndbox tag')if current_category_id is None:raise Exception('xml structure broken at bndbox tag')bbox = []# xbbox.append(bndbox['xmin'])# ybbox.append(bndbox['ymin'])# wbbox.append(bndbox['xmax'] - bndbox['xmin'])# hbbox.append(bndbox['ymax'] - bndbox['ymin'])print('add annotation with {},{},{},{}'.format(object_name, current_image_id, current_category_id,bbox))addAnnoItem(object_name, current_image_id, current_category_id, bbox)if __name__ == '__main__':xml_path = '/****/coco2012/ann_val2012'  # 这是xml文件所在的地址json_file = '/****/coco2012/annotations/val.json'  # 这是你要生成的json文件parseXmlFiles(xml_path)  # 只需要改动这两个参数就行了json.dump(coco, open(json_file, 'w'))

voc数据集转coco数据集相关推荐

【数据集转换】VOC数据集转COCO数据集·代码实现+操作步骤
在自己的数据集上实验时,往往需要将VOC数据集转化为coco数据集,因为这种需求所以才记录这篇文章,代码出处未知,感谢开源. 在远程服务器上测试目标检测算法需要用到测试集,最常用的是coco2014/ ...
Dataset之COCO数据集：COCO数据集的简介、下载、使用方法之详细攻略
Dataset之COCO数据集:COCO数据集的简介.安装.使用方法之详细攻略目录 COCO数据集的简介 0.COCO数据集的80个类别-YoloV3算法采用的数据集 1.COCO数据集的意义 2. ...
2020-12-18 Dataset之COCO数据集：COCO数据集的简介、下载
Dataset之COCO数据集:COCO数据集的简介.下载 COCO数据集的简介 MS COCO的全称是Microsoft Common Objects in Context,起源于微软于2014年出 ...
Cityscapes数据集与COCO数据集
Cityscapes数据集与COCO数据集对两个公开数据集的简要概述. 文章目录 Cityscapes数据集与COCO数据集 Cityscapes数据集 COCO数据集 info字段 images字 ...
目标检测实战篇1——数据集介绍(PASCAL VOC，MS COCO)
前言前面我们讲过了目标检测的YOLO系列算法,SSD算法.从这个博文开始,我们要真实开启实战篇章.在正式介绍实战篇之前,我们需要先知道两个数据集:PASCAL VOC和COCO数据集. 一.PA ...
voc数据集格式转换为coco数据集格式+修改xml格式文件
voc数据集格式转换为coco格式+修改xml格式文件中部分内容 voc数据集格式→coco数据集格式修改xml格式文件中部分内容 voc数据集格式→coco数据集格式下面这份代码只需修改文件所在 ...
【SSD论文解读】论文中用到的数据集：Pascal VOC、MS COCO、ImageNet
[SSD论文解读] 论文中用到的数据集一.数据集介绍 1.Pascal VOC 2.MS COCO 3.ILSVRC 二.数据集处理函数 1.专门的datasets类 2.论文中的数据集处理函数三 ...
voc数据集格式转coco数据集格式
小白AI:VOC数据集史上最实用的介绍和使用(1) 刚入门目标检测时,用的都是VOC格式的数据集,简单且评价标准单一.目前cv领域用的都是COCO格式的评价标准,前期使用Labelimg自制的数据集格 ...
COCO数据集数据转换为XML格式
nanodet需要xml格式, 百度下载地址: coco2017数据集百度网盘链接_m0_37835084的博客-CSDN博客_coco数据集百度网盘官方下载地址:https://cocodatas ...

voc数据集转coco数据集

voc数据集转coco数据集相关推荐

最新文章

热门文章