【github】https://github.com/DocF/multispectral-object-detection

一.环境

1.1 环境

基本依赖和yolov5基本相同，当然也可以配置在虚拟环境中

git clone https://github.com/DocF/multispectral-object-detection
cd  multispectral-object-detection
pip install -r requirements.txt

1.2 报错解决

1.2.1 找不到sppf

AttributeError: Can't get attribute 'SPPF' on <module 'models.common' from '/hy-tmp/multispectral-object-detection/models/common.py'>

【参考文章】找不到SPPF错误
在models/common.py下找到ssp，将下面这段添加到ssp之前

class SPPF(nn.Module):def __init__(self, c1, c2, k=5):super().__init__()c_ = c1 // 2self.cv1 = Conv(c1, c_, 1, 1)self.cv2 = Conv(c_ * 4, c2, 1, 1)self.m = nn.MaxPool2d(kernel_size=k, stride=1, padding=k // 2)def forward(self, x):x = self.cv1(x)with warnings.catch_warnings():warnings.simplefilter('ignore')y1 = self.m(x)y2 = self.m(y1)return self.cv2(torch.cat([x, y1, y2, self.m(y2)], 1))

1.2.2

RuntimeError: result type Float can't be cast to the desired output type __int64

【参考】报错解决方法
将下面这段替换utils/loss.py中build_targets函数，注意保留返回值

        for i in range(self.nl):anchors, shape = self.anchors[i], p[i].shapegain[2:6] = torch.tensor(shape)[[3, 2, 3, 2]]  # xyxy gain# Match targets to anchorst = targets * gain  # shape(3,n,7)if nt:# Matchesr = t[..., 4:6] / anchors[:, None]  # wh ratioj = torch.max(r, 1 / r).max(2)[0] < self.hyp['anchor_t']  # compare# j = wh_iou(anchors, t[:, 4:6]) > model.hyp['iou_t']  # iou(3,n)=wh_iou(anchors(3,2), gwh(n,2))t = t[j]  # filter# Offsetsgxy = t[:, 2:4]  # grid xygxi = gain[[2, 3]] - gxy  # inversej, k = ((gxy % 1 < g) & (gxy > 1)).Tl, m = ((gxi % 1 < g) & (gxi > 1)).Tj = torch.stack((torch.ones_like(j), j, k, l, m))t = t.repeat((5, 1, 1))[j]offsets = (torch.zeros_like(gxy)[None] + off[:, None])[j]else:t = targets[0]offsets = 0# Definebc, gxy, gwh, a = t.chunk(4, 1)  # (image, class), grid xy, grid wh, anchorsa, (b, c) = a.long().view(-1), bc.long().T  # anchors, image, classgij = (gxy - offsets).long()gi, gj = gij.T  # grid indices# Appendindices.append((b, a, gj.clamp_(0, shape[2] - 1), gi.clamp_(0, shape[3] - 1)))  # image, anchor, gridtbox.append(torch.cat((gxy - gij, gwh), 1))  # boxanch.append(anchors[a])  # anchorstcls.append(c)  # class

二. 数据集处理

2.1 数据集下载

【github】https://github.com/DocF/multispectral-object-detection包含了对应的链接

链接：https://pan.baidu.com/s/1zO_1Olognq2atY6m4StZUA?pwd=4i77 提取码：4i77
–来自百度网盘超级会员V1的分享

权重还有数据集全部都打包在这里面了

2.2 数据集放置格式

其实没有严格的规定，我的话是这样：在datasets文件夹下

2.3 数据集预处理成txt

以FLIR（就是那个align）为例

2.3.1 训练集验证集

split_train_val.py

import os
import random
import argparseparser = argparse.ArgumentParser()
parser.add_argument('--xml_path', type=str, help='input xml label path')
parser.add_argument('--txt_path', type=str, help='output txt label path')
opt = parser.parse_args()trainval_percent = 1.0
train_percent = 0.9
xmlfilepath = opt.xml_path
txtsavepath = opt.txt_path
total_xml = os.listdir(xmlfilepath)
if not os.path.exists(txtsavepath):os.makedirs(txtsavepath)num=len(total_xml)
list=range(num)ftrainval = open(txtsavepath + '/trainval.txt', 'w')
ftest = open(txtsavepath + '/test.txt', 'w')
ftrain = open(txtsavepath + '/train.txt', 'w')
fval = open(txtsavepath + '/val.txt', 'w')for i in list:name=total_xml[i][:-4]+'\n'ftrainval.write(name)if i%7 == 0:fval.write(name)else:ftrain.write(name)ftrainval.close()
ftrain.close()
fval.close()
ftest.close()

输入命令：

python split_train_val.py --xml_path xml文件路径 --txt_path 输出txt文件路径

（1）xml文件路径：我是先将xml为文件全部放到一个文件夹里面
以我的为例就是：

cp D:\computervision\cross\detection\align\Annotations\*.xml D:\computervision\cross\detection\align\annotation

(2)输出txt文件路径：直接输出到前面提到的datasets下
得到下面这四个

2.3.2 格式转换

voc_label.py文件，应该改一下路径就可以用了，就不多说了

import xml.etree.ElementTree as ET
import pickle
import os
from os import listdir, getcwd
from os.path import joinsets=['train', 'val', 'test']
classes = ['person','car','bicycle']abs_path = os.getcwd()
def convert(size, box):dw = 1./(size[0])dh = 1./(size[1])x = (box[0] + box[1])/2.0 - 1y = (box[2] + box[3])/2.0 - 1w = box[1] - box[0]h = box[3] - box[2]x = x*dww = w*dwy = y*dhh = h*dhreturn (x,y,w,h)
def convert_annotation(image_id ,RGBid ):in_file = open(r'D:\computervision\cross\detection\align\annotation\%s.xml'%( image_id))irout_file = open('D:\computervision\cross\detection\multispectral-object-detection-main\datasets\IR\labels\%s.txt'%(image_id), 'w')rgbout_file= open('D:\computervision\cross\detection\multispectral-object-detection-main\datasets\RGB\labels\%s.txt'%(RGBid), 'w')tree=ET.parse(in_file)root = tree.getroot()size = root.find('size')w = int(size.find('width').text)h = int(size.find('height').text)for obj in root.iter('object'):#difficult = obj.find('difficult').textcls = obj.find('name').textif cls not in classes :continuecls_id = classes.index(cls)xmlbox = obj.find('bndbox')b = (float(xmlbox.find('xmin').text), float(xmlbox.find('xmax').text), float(xmlbox.find('ymin').text), float(xmlbox.find('ymax').text))bb = convert((w,h), b)irout_file.write(str(cls_id) + " " + " ".join([str(a) for a in bb]) + '\n')rgbout_file.write(str(cls_id) + " " + " ".join([str(a) for a in bb]) + '\n')for image_set in sets:# if not os.path.exists('D:\computervision\cross\detection\multispectral-object-detection-main\datasets'):#     os.makedirs('D:\computervision\cross\detection\multispectral-object-detection-main\datasets')#创建两个txt文件#（1）先创建rgb文件#image_ids = open('D:\computervision\cross\detection\multispectral-object-detection-main\datasets\%s.txt'%(image_set)).read().strip().split()ir_file = open('D:\computervision\cross\detection\multispectral-object-detection-main\datasets\IR\%s.txt'%(image_set), 'w')rgb_file= open('D:\computervision\cross\detection\multispectral-object-detection-main\datasets\RGB\%s.txt'%(image_set), 'w')for image_id in image_ids:ir_file.write('D:\computervision\cross\detection\multispectral-object-detection-main\datasets\IR\images\%s.jpeg\n'%(image_id))id=image_id.split("_")[1]RGBid='FLIR_'+id+"_RGB"rgb_file.write('D:\computervision\cross\detection\multispectral-object-detection-main\datasets\RGB\images\%s.jpg\n' % (RGBid))convert_annotation(image_id,RGBid)ir_file.close()rgb_file.close()

三 .训练

修改data/multispectral/FLIR_aligned.yaml文件夹

直接

python train.py

多模态（红外，可见光）目标检测相关推荐

ALCNet——红外小目标检测网络论文阅读
论文链接:Attentional Local Contrast Networks for Infrared Small Target Detection | IEEE Journals & M ...
目标立体检测红外图像_一种有效的红外小目标检测方法
[3]CHEN Bing-wen, WANG Wen-wei, QIN Qian-qing. Infrared dim target detection based on fuzzy-ART neur ...
我们是如何改进YOLOv3进行红外小目标检测的？
点击上方"3D视觉工坊",选择"星标" 干货第一时间送达 [导语]本文将介绍BBuf.小武和笔者一起在过年期间完成的一个目标检测项目,将描述我们模型改进的思路. ...
红外小目标检测之DANNet
Dense Nested Attention Network for Infrared Small Target Detection 文章亮点: 1.提出一种密集嵌套交互模块和通道-空间注意力模块,实 ...
一文尽览 | 基于点云、多模态的3D目标检测算法综述！（Point/Voxel/Point-Voxel）
点击下方卡片,关注"自动驾驶之心"公众号 ADAS巨卷干货,即可获取点击进入→自动驾驶之心技术交流群后台回复[ECCV2022]获取ECCV2022所有自动驾驶方向论文! 目前 ...
哈尔滨工业大学提出 RISTDnet:强鲁棒性的红外小目标检测网络
哈工大提出***RISTDnet***:强鲁棒性的红外小目标检测网络 RISTDnet: Robust Infrared Small Target Detection Network 作者单位:哈尔滨 ...
红外弱小目标检测之“Asymmetric Contextual Modulation for Infrared Small Target Detection“阅读笔记
Asymmetric Contextual Modulation for Infrared Small Target Detection 本文开源代码及单帧红外小目标检测数据集: code:https ...
2021-03-22 基于视觉显著性的红外弱小目标检测-王警予 |笔记
基于视觉显著性的红外弱小目标检测-王警予 | 阅读笔记第四章基于稀疏低秩分解的红外弱小目标检测 key 视觉注意力机制稀疏低秩分解针对人类视觉显著性是基于人类视觉注意的特点,将红外图像中显著性 ...
红外小目标检测的非对称上下文调制
摘要因为缺乏固有的目标特征,并且缺乏一个公共的数据集,单帧红外小目标检测依然是一个挑战.在本文中,我们首先提供了一个带有高质量标注的开放数据集,以推进这一领域的研究.我们还提出了一种专门为检测红外小 ...
弱小目标检测跟踪算法研究（5）基于顶帽变换（Top_hat）算法的红外弱小目标检测之背景抑制
基于顶帽变换(Top_hat)算法的红外弱小目标检测之背景抑制 1. 前言 2. 顶帽变换(Top_hat)算法 3. Matlab仿真 4. 小结 1. 前言红外图像中的弱小目标,目标属性包涵&q ...

多模态（红外，可见光）目标检测