YOLOv3训练自己的数据集

最近在做目标检测相关的工作，先用faster R-CNN训练了一下，感觉效果不是很好，换成了res101特征提取，mAP也只达到了30+，于是想换个模型试试，就选择了yolov3
yolov3的训练需要用到自己的标签格式，我就直接拿了上次制作的VOC格式数据集进行处理和训练。https://blog.csdn.net/qq_36852276/article/details/100154097
记录一下
https://blog.csdn.net/helloworld1213800/article/details/79749359
https://blog.csdn.net/weixin_41813620/article/details/92799338
https://blog.csdn.net/xuanlang39/article/details/88642010

常见问题和注释

https://blog.csdn.net/maweifei/article/details/81137563

笔记

将文件夹内的绝对路径写入txt文件

ls -R /home/datalab/work/datasets/test_7pilang/*.jpg > file.txt

无绝对路径，当前文件夹文件名写入txt文件

ls -R *.jpg > file.txt

训练自己数据集的流程

下载darknet项目

git clone https://github.com/pjreddie/darknet
cd darknet

修改Makefile

主要是修改3处
一是GPU、CUDNN、OPENCV
二是修改算力符合自己的显卡
三是cudnn路径改成自己的GPU=1
CUDNN=1
OPENCV=1
OPENMP=0
DEBUG=0ARCH= -gencode arch=compute_61,code=sm_61
#      -gencode arch=compute_35,code=sm_35 \
#      -gencode arch=compute_50,code=[sm_50,compute_50] \
#      -gencode arch=compute_52,code=[sm_52,compute_52]
#      -gencode arch=compute_20,code=[sm_20,sm_21] \ This one is deprecated?# This is what I use, uncomment if you know your arch and want to specify
# ARCH= -gencode arch=compute_52,code=compute_52ifeq ($(GPU), 1)
COMMON+= -DGPU -I/usr/local/cuda/include/
CFLAGS+= -DGPU
LDFLAGS+= -L/usr/local/cuda/lib64 -lcuda -lcudart -lcublas -lcurand
endif

修改完之后执行编译 make

准备自己的数据集

我使用的是labelme进行标注，然后把产生的json文件转化成xml格式，接着进行数据增强，我自己的脚本如下所示
https://blog.csdn.net/qq_36852276/article/details/102539858

有了图片和标签后，把图片和标签整理成VOC数据集的格式，为了方便放在scripts文件夹下
文件夹结构是

VOCdevkit
—VOC2007
——Annotations
——ImageSets
———Layout
———Main
———Segmentation
——JPEGImages

其中Main下又分别是四个文件 train.txt trainval.txt val.txt test.txt
它们分别是不带后缀的文件名

接着利用项目中的./scripts/voc_label.py 直接生成yolo所需的文件
要修改一部分适应自己的数据，这里一共三处，注释掉的是原来的，我只需要识别一类boat
分别是
sets
classes
os.system

准备预训练权重

从官网上就可以下载

wget https://pjreddie.com/media/files/darknet53.conv.74

为了方便放在model文件下

修改配置文件cfg/voc.data

这个要修改成适应自己数据的地方
classes改为类别数
train,valid是生成的yolo所需的绝对路径的txt文件序列
names是类别的txt文件序列

classes= 1
train  = /home/xxx/darknet_2/darknet/scripts/2007_train.txt
valid  = /home/xxx/darknet_2/darknet/scripts/2007_val.txt
names = /home/xxx/darknet_2/darknet/data/voc.names
backup = backup

修改配置文件data/voc.name

改成自己的标签名

修改配置文件cfg/yolov3-voc.cfg

这个就是模型的各项配置，各项介绍如下

[net]
# Testing            ### 测试模式
# batch=1
# subdivisions=1
# Training           ### 训练模式，每次前向的图片数目 = batch/subdivisions
batch=64
subdivisions=16
width=416            ### 网络的输入宽、高、通道数
height=416
channels=3
momentum=0.9         ### 动量
decay=0.0005         ### 权重衰减
angle=0
saturation = 1.5     ### 饱和度
exposure = 1.5       ### 曝光度
hue=.1               ### 色调
learning_rate=0.001  ### 学习率
burn_in=1000         ### 学习率控制的参数
max_batches = 5000  ### 迭代次数
policy=steps         ### 学习率策略
steps=40000,45000    ### 学习率变动步长
scales=.1,.1         ### 学习率变动因子  [convolutional]
batch_normalize=1    ### BN
filters=32           ### 卷积核数目
size=3               ### 卷积核尺寸
stride=1             ### 卷积核步长
pad=1                ### pad
activation=leaky     ### 激活函数

这个文件共需要修改6处
其实是有一个一模一样的结构出现了3次，每个都要修改2个地方，就是下面的结构，建议搜索yolo关键字就能找到这三处

[convolutional]
size=1
stride=1
pad=1
filters=18   #修改这里，3*（类别数+4+1）
activation=linear[yolo]
mask = 0,1,2
anchors = 10,13,  16,30,  33,23,  30,61,  62,45,  59,119,  116,90,  156,198,  373,326
classes=1  #修改这里，改成你自己的类别数
num=9
jitter=.3
ignore_thresh = .5
truth_thresh = 1
random=1

上述的filters、classes需要修改修改方式看上面注释
除此之外，还有几个关键的地方要考虑

#batch表示每次多少张图被读取，subdivisions表示它们在训练上被分了多少份，因此batch_size = batch/subdivisions，所以可以按照它适当调节来适应你的显存大小，这个除法结果越小，需要显存越少
batch=64
subdivisions=16

#max_batches是迭代次数，且默认保存方式是1000以前每100保存一次，1000以后每10000保存一次
max_batches = 5000

#random是多尺度训练，就是图像会被初始化成某个范围内的随机大小进行训练，因此占用显存是浮动的，要是显存紧张可以设为0，就会每次固定成width=416，height=416（设置里有）的大小进行训练

开始训练

#三个路径分别是修改好的  数据集文件、模型配置文件、预训练权重
./darknet detector train cfg/voc.data cfg/yolov3-voc.cfg ./model/darknet53.conv.74 -gpus 0

训练好的模型测试

方法1

#这个是默认使用了coco.names，有80类
./darknet detect cfg/yolov3-voc.cfg backup/yolov3-voc_final.weights data/dog.jpg

方法2

./darknet detector test cfg/voc.data cfg/yolov3-voc.cfg backup/yolov3-voc_final.weights xxx/xxx.jpg

调用python接口

darknet项目下有一个python文件夹，里面的darknet.py就是官方给出的python接口，将darknet.py移动到darknet项目路径下，然后可以对着它进行修改即可，注意它产生的结果是x,y,w,h的格式，就是中心点坐标、宽高
下面是我直接使用的例子

if __name__ == "__main__":#net = load_net("cfg/densenet201.cfg", "/home/pjreddie/trained/densenet201.weights", 0)#im = load_image("data/wolf.jpg", 0, 0)#meta = load_meta("cfg/imagenet1k.data")#r = classify(net, meta, im)#print r[:10]net = load_net(b"cfg/yolov3-voc.cfg", b"backup/yolov3-voc_final.weights", 0)meta = load_meta(b"cfg/voc.data")root_path = "/home/xxx/wurenting/dataset_total/"save_path = "/home/xxx/guangdong/guangdong_yolov3/"path_list = os.listdir(root_path)for filename in path_list:r = detect(net, meta, (root_path+filename).encode('utf-8'),thresh=.3,nms=.3)#draw triangleimg = cv2.imread(root_path + filename)weight_img = img.shape[1]height_img = img.shape[0]for r_res in r:cls = r_res[0]score = r_res[1]x = r_res[2][0]y = r_res[2][1]w = r_res[2][2]h = r_res[2][3]x_left_top = float("%.2f"%(x - w/2))y_left_top = float("%.2f"%(y - h/2))x_right_bottom = float("%.2f"%(x + w/2))y_right_bottom = float("%.2f"%(y + h/2))if x_left_top > weight_img:x_left_top = weight_imgif x_right_bottom > weight_img:x_right_bottom = weight_imgif y_left_top > height_img:y_left_top = height_imgif y_right_bottom > height_img:y_right_bottom = height_imgif x_left_top<0:x_left_top = 0if x_right_bottom<0:x_right_bottom = 0if y_left_top < 0:y_left_top = 0if y_right_bottom < 0:y_right_bottom = 0bbox = (x_left_top,y_left_top,x_right_bottom,y_right_bottom)with open('/home/xxx/guangdong/test_b_yolo.txt','a') as xxx_obj:xxx_obj.write(filename + '_' + cls.decode('utf-8') + '_' + str(bbox) + '_' + str(score) + '_\n') cv2.namedWindow("res", cv2.WINDOW_NORMAL)cv2.rectangle(img, (int(x_left_top),int(y_left_top)),(int(x_right_bottom),int(y_right_bottom)), (0, 204, 0), 2)cv2.putText(img, '%s: %.3f' % (str(cls)[2:-1], score), (int(x_left_top), int(y_left_top) + 15), cv2.FONT_HERSHEY_PLAIN,1.0, (0, 0, 255), thickness=2)
#        cv2.imwrite(save_path +filename,img)cv2.imshow("res",img)if cv2.waitKey(0)==ord('q'):sys.exit()
#        print(r)cv2.destroyAllWindows()

使用python接口对视频、图像进行处理

预留的端口只能输入路径，也就是只能进行单张离线图片测试，不是很方便，于是可以对源码进行修改，将接口输入路径修改为输入图片
看这位大佬的博客，亲测可用

https://blog.csdn.net/phinoo/article/details/83009061

修改后就为所欲为，放一个我读取RTSP网络视频流进行处理的例子

from ctypes import *
import math
import random
import cv2
import os
import sys
import threading#define global variable
count = 0
flag = 0def sample(probs):s = sum(probs)probs = [a/s for a in probs]r = random.uniform(0, 1)for i in range(len(probs)):r = r - probs[i]if r <= 0:return ireturn len(probs)-1def c_array(ctype, values):arr = (ctype*len(values))()arr[:] = valuesreturn arrclass BOX(Structure):_fields_ = [("x", c_float),("y", c_float),("w", c_float),("h", c_float)]class DETECTION(Structure):_fields_ = [("bbox", BOX),("classes", c_int),("prob", POINTER(c_float)),("mask", POINTER(c_float)),("objectness", c_float),("sort_class", c_int)]class IMAGE(Structure):_fields_ = [("w", c_int),("h", c_int),("c", c_int),("data", POINTER(c_float))]class METADATA(Structure):_fields_ = [("classes", c_int),("names", POINTER(c_char_p))]#lib = CDLL("/home/pjreddie/documents/darknet/libdarknet.so", RTLD_GLOBAL)
lib = CDLL("/home/xxx/darknet_1/darknet/libdarknet.so", RTLD_GLOBAL)
lib.network_width.argtypes = [c_void_p]
lib.network_width.restype = c_int
lib.network_height.argtypes = [c_void_p]
lib.network_height.restype = c_intndarray_image = lib.ndarray_to_image
ndarray_image.argtypes = [POINTER(c_ubyte), POINTER(c_long), POINTER(c_long)]
ndarray_image.restype = IMAGEpredict = lib.network_predict
predict.argtypes = [c_void_p, POINTER(c_float)]
predict.restype = POINTER(c_float)set_gpu = lib.cuda_set_device
set_gpu.argtypes = [c_int]make_image = lib.make_image
make_image.argtypes = [c_int, c_int, c_int]
make_image.restype = IMAGEget_network_boxes = lib.get_network_boxes
get_network_boxes.argtypes = [c_void_p, c_int, c_int, c_float, c_float, POINTER(c_int), c_int, POINTER(c_int)]
get_network_boxes.restype = POINTER(DETECTION)make_network_boxes = lib.make_network_boxes
make_network_boxes.argtypes = [c_void_p]
make_network_boxes.restype = POINTER(DETECTION)free_detections = lib.free_detections
free_detections.argtypes = [POINTER(DETECTION), c_int]free_ptrs = lib.free_ptrs
free_ptrs.argtypes = [POINTER(c_void_p), c_int]network_predict = lib.network_predict
network_predict.argtypes = [c_void_p, POINTER(c_float)]reset_rnn = lib.reset_rnn
reset_rnn.argtypes = [c_void_p]load_net = lib.load_network
load_net.argtypes = [c_char_p, c_char_p, c_int]
load_net.restype = c_void_pdo_nms_obj = lib.do_nms_obj
do_nms_obj.argtypes = [POINTER(DETECTION), c_int, c_int, c_float]do_nms_sort = lib.do_nms_sort
do_nms_sort.argtypes = [POINTER(DETECTION), c_int, c_int, c_float]free_image = lib.free_image
free_image.argtypes = [IMAGE]letterbox_image = lib.letterbox_image
letterbox_image.argtypes = [IMAGE, c_int, c_int]
letterbox_image.restype = IMAGEload_meta = lib.get_metadata
lib.get_metadata.argtypes = [c_char_p]
lib.get_metadata.restype = METADATAload_image = lib.load_image_color
load_image.argtypes = [c_char_p, c_int, c_int]
load_image.restype = IMAGErgbgr_image = lib.rgbgr_image
rgbgr_image.argtypes = [IMAGE]predict_image = lib.network_predict_image
predict_image.argtypes = [c_void_p, IMAGE]
predict_image.restype = POINTER(c_float)def nparray_to_image(img):data = img.ctypes.data_as(POINTER(c_ubyte))image = ndarray_image(data, img.ctypes.shape, img.ctypes.strides)return imagedef classify(net, meta, im):out = predict_image(net, im)res = []for i in range(meta.classes):res.append((meta.names[i], out[i]))res = sorted(res, key=lambda x: -x[1])return resdef detect(net, meta, im, thresh=.5, hier_thresh=.5, nms=.45):num = c_int(0)pnum = pointer(num)predict_image(net, im)dets = get_network_boxes(net, im.w, im.h, thresh, hier_thresh, None, 0, pnum)num = pnum[0]if (nms): do_nms_obj(dets, num, meta.classes, nms);res = []for j in range(num):for i in range(meta.classes):if dets[j].prob[i] > 0:b = dets[j].bboxres.append((meta.names[i], dets[j].prob[i], (b.x, b.y, b.w, b.h)))res = sorted(res, key=lambda x: -x[1])free_image(im)free_detections(dets, num)return resdef deal_img():global arrglobal countglobal netglobal metaglobal flag
#    global video_writerwhile True:arr_temp = arrif flag==1:if count%3==0:cv2.imwrite('/home/xxx/wurenting/dataset_10_21/' +str(int(count/3))+'.jpg',arr_temp)count += 1img = nparray_to_image(arr_temp)r = detect(net, meta, img,thresh=.6,nms=.3)#draw triangleweight_img = arr_temp.shape[1]height_img = arr_temp.shape[0]for r_res in r:cls = r_res[0]score = r_res[1]x = r_res[2][0]y = r_res[2][1]w = r_res[2][2]h = r_res[2][3]x_left_top = float("%.2f"%(x - w/2))y_left_top = float("%.2f"%(y - h/2))x_right_bottom = float("%.2f"%(x + w/2))y_right_bottom = float("%.2f"%(y + h/2))if x_left_top > weight_img:x_left_top = weight_imgif x_right_bottom > weight_img:x_right_bottom = weight_imgif y_left_top > height_img:y_left_top = height_imgif y_right_bottom > height_img:y_right_bottom = height_imgif x_left_top<0:x_left_top = 0if x_right_bottom<0:x_right_bottom = 0if y_left_top < 0:y_left_top = 0if y_right_bottom < 0:y_right_bottom = 0bbox = (x_left_top,y_left_top,x_right_bottom,y_right_bottom)#            with open('/home/xxx/guangdong/test_b_yolo.txt','a') as xxx_obj:#                xxx_obj.write(filename + '_' + cls.decode('utf-8') + '_' + str(bbox) + '_' + str(score) + '_\n') cv2.rectangle(arr_temp, (int(x_left_top),int(y_left_top)),(int(x_right_bottom),int(y_right_bottom)), (0, 204, 0), 2)cv2.putText(arr_temp, '%s: %.3f' % (str(cls)[2:-1], score), (int(x_left_top), int(y_left_top) + 15), cv2.FONT_HERSHEY_PLAIN,1.0, (0, 0, 255), thickness=2)cv2.namedWindow("res", cv2.WINDOW_NORMAL)cv2.imshow("res",arr_temp)#    video_writer.write(arr_temp)if cv2.waitKey(1)==ord('k'):flag = 1elif cv2.waitKey(1)==ord('q'):flag = 0#        video_writer.release()#        print(r)
def change_arr():global countglobal arrglobal vidwhile True:return_value , arr = vid.read()#cv2.imwrite('/home/xxx/wurenting/dataset_10_17/' +str(count)+'.jpg',arr_temp)#count+=1if __name__ == "__main__":#net = load_net("cfg/densenet201.cfg", "/home/pjreddie/trained/densenet201.weights", 0)#im = load_image("data/wolf.jpg", 0, 0)#meta = load_meta("cfg/imagenet1k.data")#r = classify(net, meta, im)#print r[:10]#加载官方参数#net = load_net(b"cfg/yolov3.cfg", b"yolov3.weights", 0)#meta = load_meta(b"cfg/coco.data")#加载船只检测参数net = load_net(b"/homexxx/darknet_2/darknet/cfg/yolov3-voc.cfg", b"/home/xxx/darknet_2/darknet/backup/yolov3-voc_final.weights", 0)meta = load_meta(b"/home/xxx/darknet_2/darknet/cfg/voc.data")vid = cv2.VideoCapture("rtsp://admin:12345@192.168.1.113:554/")#vid = cv2.VideoCapture("rtsp://127.0.0.1:8554/test")#vid = cv2.VideoCapture("rtsp://192.168.1.146:8553/PSIA/Streaming/channels/0?videoCodecType=H.264")#vid = cv2.VideoCapture("/home/xxx/wurenting/2019_10_16_16_07_IMG_1522.mp4")return_value , arr = vid.read()#video_writer = cv2.VideoWriter('/home/xxx/wurenting/video_result_10_21/result.avi', cv2.VideoWriter_fourcc('M','J','P','G'), 30, (1920,1080))thread_1 = threading.Thread(target=deal_img)   # 定义线程 1thread_2 = threading.Thread(target=change_arr) # 定义线程 2thread_1.start()  # 让线程1开始工作thread_2.start()  # 让线程2开始工作

其他的问题，例如修改保存权重的迭代值之类的问题，看最上面的常见问题、注释那篇博客

使用已知json和图片将目标切割出来代码

import os
from lxml.etree import Element, SubElement, tostring
from xml.dom.minidom import parseString
from PIL import Image
import json
import cv2json_dir = '/home/xbw/wurenting/train_buoy_1117/red/json/'
img_dir = '/home/xbw/wurenting/train_buoy_1117/red/img/'
json_list = os.listdir(json_dir)
json_list.sort(key=lambda x:int(x[:-5]))
for image_name in json_list:print(image_name)img_source = cv2.imread(img_dir+image_name[:-5]+'.jpg')with open(json_dir+image_name) as obj:nums = json.load(obj)labels = []bboxes = []for i in nums['shapes']:labels.append(i['label'])bboxes.append([min(i['points'][0][0],i['points'][1][0]),min(i['points'][0][1],i['points'][1][1]),max(i['points'][0][0],i['points'][1][0]),max(i['points'][0][1],i['points'][1][1])])bbox = bboxes[0]crop = img_source[int(bbox[1]):int(bbox[3]),int(bbox[0]):int(bbox[2])]cv2.imwrite("/home/xbw/wurenting/train_buoy_1117/red/crop/"+image_name[:-5]+'.jpg',crop)cv2.imshow("res",crop)cv2.waitKey(1)