【camera】YOLOV7实现实例分割+目标检测任务（训练、测试、量化、部署）

【camera】YOLOV7实现实例分割+目标检测任务

代码下载地址

训练、测试、量化、部署代码

训练

For training, quite simple, same as detectron2:

python train_net.py --config-file configs/coco/darknet53.yaml --num-gpus 8

If you want train YOLOX, you can using config file configs/coco/yolox_s.yaml. All support arch are:

YOLOX: anchor free yolo;
YOLOv7: traditional yolo with some explorations, mainly focus on loss experiments;
YOLOv7P: traditional yolo merged with decent arch from YOLOX;
YOLOMask: arch do detection and segmentation at the same time (tbd);
YOLOInsSeg: instance segmentation based on YOLO detection (tbd);

Demo

Run a quick demo would be like:

python3 demo.py --config-file configs/wearmask/darknet53.yaml --input ./datasets/wearmask/images/val2017 --opts MODEL.WEIGHTS output/model_0009999.pth

Run SparseInst:

python demo.py --config-file configs/coco/sparseinst/sparse_inst_r50vd_giam_aug.yaml --video-input ~/Movies/Videos/86277963_nb2-1-80.flv -c 0.4 --opts MODEL.WEIGHTS weights/sparse_inst_r50vd_giam_aug_8bc5b3.pth

an update based on detectron2 newly introduced LazyConfig system, run with a LazyConfig model using:

python3 demo_lazyconfig.py --config-file configs/new_baselines/panoptic_fpn_regnetx_0.4g.py --opts train.init_checkpoint=output/model_0004999.pth

预训练模型

model	backbone	input	aug	AP^val	AP	FPS	weights
SparseInst	R-50	640	✘	32.8	-	44.3	model
SparseInst	R-50-vd	640	✘	34.1	-	42.6	model
SparseInst (G-IAM)	R-50	608	✘	33.4	-	44.6	model
SparseInst (G-IAM)	R-50	608	✓	34.2	34.7	44.6	model
SparseInst (G-IAM)	R-50-DCN	608	✓	36.4	36.8	41.6	model
SparseInst (G-IAM)	R-50-vd	608	✓	35.6	36.1	42.8	model
SparseInst (G-IAM)	R-50-vd-DCN	608	✓	37.4	37.9	40.0	model
SparseInst (G-IAM)	R-50-vd-DCN	640	✓	37.7	38.1	39.3	model

部署

detr:

python export_onnx.py --config-file detr/config/file

this works has been done, inference script included inside tools.

AnchorDETR:

anchorDETR also supported training and exporting to ONNX.

效果

Here is a dedicated performance compare with other packages.

tbd.

支持的任务列表

YOLOv4 contained with CSP-Darknet53;
YOLOv7 arch with resnets backbone;
YOLOv7 arch with resnet-vd backbone (likely as PP-YOLO), deformable conv, Mish etc;
GridMask augmentation from PP-YOLO included;
Mosiac transform supported with a custom datasetmapper;
YOLOv7 arch Swin-Transformer support (higher accuracy but lower speed);
YOLOv7 arch Efficientnet + BiFPN;
YOLOv5 style positive samples selection, new coordinates coding style;
RandomColorDistortion, RandomExpand, RandomCrop, RandomFlip;
CIoU loss (DIoU, GIoU) and label smoothing (from YOLOv5 & YOLOv4);
YOLOF also included;
YOLOv7 Res2net + FPN supported;
Pyramid Vision Transformer v2 (PVTv2) supported;
WBF (Weighted Box Fusion), this works better than NMS, link;
YOLOX like head design and anchor design, also training support;
YOLOX s,m,l backbone and PAFPN added, we have a new combination of YOLOX backbone and pafpn;
YOLOv7 with Res2Net-v1d backbone, we found res2net-v1d have a better accuracy then darknet53;
Added PPYOLOv2 PAN neck with SPP and dropblock;
YOLOX arch added, now you can train YOLOX model (anchor free yolo) as well;
DETR: transformer based detection model and onnx export supported, as well as TensorRT acceleration;
AnchorDETR: Faster converge version of detr, now supported!

what’s more, there are some features awesome inside repo:

Almost all models can export to onnx;
Supports TensorRT deployment for DETR and other transformer models;
It will integrate with wanwu, a torch-free deploy framework run fastest on your target platform.

Help wanted! If you have spare time or if you have GPU card, then help YOLOv7 become more stronger! Here is the guidance of contribute:

Claim task: I have some ideas but do not have enough time to do it, if you want implement it, claim the task, I will give u fully advise on how to do, and you can learn a lot from it;
Test mAP: When you finished new idea implementation, create a thread to report experiment mAP, if it work, then merge into our main master branch;
Pull request: YOLOv7 is open and always tracking on SOTA and light models, if a model is useful, we will merge it and deploy it, distribute to all users want to try.

Here are some tasks need to be claimed:

VAN: Visual Attention Network, paper, VAN-Segmentation, it was better than Swin and PVT and DeiT:
- D2 VAN backbone integration;
- Test with YOLOv7 arch;
ViDet: code, this provides a realtime detector based on transformer, Swin-Nano mAP: 40, while 20 FPS, it can be integrated into YOLOv7;
- Integrate into D2 backbone, remove MSAtten deps;
- Test with YOLOv7 or DETR arch;
DINO: 63.3mAP highest in 2022 on coco. https://github.com/IDEACVR/DINO
- waiting for DINO opensource code.
ConvNext: https://github.com/facebookresearch/ConvNeXt, combined convolution and transformer.