1.研究背景

随着科技的进步和社会需求的增长，近年来摄像头逐渐高清化、高帧率化，摄像头作为信息获取设备的载体也不再局限于固定场景。路口、路侧、室内、高位、低位等不同场景下产生了各种对于检测识别的需求，比如路侧车牌识别一体机需要对车位进行无人值守，完成自动收费。传统的车牌识别方案无法面对困难场景的挑战，先验知识也随着场景类别增加而逐渐难以涵盖。我国作为车辆保有量最大的国家，在汽车数量增长的同时，其种类也在逐步增长。在这些车牌中，有传统的蓝牌、新能源汽车的绿牌、运载车的黄牌、特种车辆的双层车牌、粤港澳通行的双牌等等，目前兼容各类车牌且具有较高精确度的双层车牌识别系统具有很高的研究价值[4.5]。

2.图片识别效果

3.视频演示

Python基于YOLOv7和CRNN的双层车牌分割＆识别系统（源码＆教程）_哔哩哔哩_bilibili

4.准备YOLOv7格式车牌数据集（数据集文末链接提供下载）

如果不懂yolo格式数据集是什么样子的，建议先学习一下该博客。大部分CVer都会推荐用labelImg进行数据的标注，我也不例外，推荐大家用labelImg进行数据标注。不过这里我不再详细介绍如何使用labelImg，网上有很多的教程。同时，标注数据需要用到图形交互界面，远程服务器就不太方便了，因此建议在本地电脑上标注好后再上传到服务器上。

这里假设我们已经得到标注好的yolo格式数据集，那么这个数据集将会按照如下的格式进行存放。

不过在这里面，train_list.txt和val_list.txt是后来我们要自己生成的，而不是labelImg生成的；其他的则是labelImg生成的。

接下来，就是生成 train_list.txt和val_list.txt。train_list.txt存放了所有训练图片的路径，val_list.txt则是存放了所有验证图片的路径，如下图所示，一行代表一个图片的路径。这两个文件的生成写个循环就可以了，不算难。

5.修改配置文件

总共有两个文件需要配置，一个是/yolov7/cfg/training/yolov7.yaml，这个文件是有关模型的配置文件；一个是/yolov7/data/coco.yaml，这个是数据集的配置文件。

第一步，复制yolov7.yaml文件到相同的路径下，然后重命名，我们重命名为yolov7-Helmet.yaml。

第二步，打开yolov7-Helmet.yaml文件，进行如下图所示的修改，这里修改的地方只有一处，就是把nc修改为我们数据集的目标总数即可。然后保存。

第三步，复制coco.yaml文件到相同的路径下，然后重命名，我们命名为Helmet.yaml。

第四步，打开Helmet.yaml文件，进行如下所示的修改，需要修改的地方为5处。

第一处：把代码自动下载COCO数据集的命令注释掉，以防代码自动下载数据集占用内存；第二处：修改train的位置为train_list.txt的路径；第三处：修改val的位置为val_list.txt的路径；第四处：修改nc为数据集目标总数；第五处：修改names为数据集所有目标的名称。然后保存。

6.YOLOv7训练代码

import argparse
import logging
import math
import os
import random
import time
from copy import deepcopy
from pathlib import Path
from threading import Threadimport numpy as np
import torch.distributed as dist
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
import torch.optim.lr_scheduler as lr_scheduler
import torch.utils.data
import yaml
from torch.cuda import amp
from torch.nn.parallel import DistributedDataParallel as DDP
from torch.utils.tensorboard import SummaryWriter
from tqdm import tqdmimport test  # import test.py to get mAP after each epoch
from models.experimental import attempt_load
from models.yolo import Model
from utils.autoanchor import check_anchors
from utils.datasets import create_dataloader
from utils.general import labels_to_class_weights, increment_path, labels_to_image_weights, init_seeds, \fitness, strip_optimizer, get_latest_run, check_dataset, check_file, check_git_status, check_img_size, \check_requirements, print_mutation, set_logging, one_cycle, colorstr
from utils.google_utils import attempt_download
from utils.loss import ComputeLoss, ComputeLossOTA
from utils.plots import plot_images, plot_labels, plot_results, plot_evolution
from utils.torch_utils import ModelEMA, select_device, intersect_dicts, torch_distributed_zero_first, is_parallel
from utils.wandb_logging.wandb_utils import WandbLogger, check_wandb_resumelogger = logging.getLogger(__name__)def train(hyp, opt, device, tb_writer=None):logger.info(colorstr('hyperparameters: ') + ', '.join(f'{k}={v}' for k, v in hyp.items()))save_dir, epochs, batch_size, total_batch_size, weights, rank, freeze = \Path(opt.save_dir), opt.epochs, opt.batch_size, opt.total_batch_size, opt.weights, opt.global_rank, opt.freeze# Directorieswdir = save_dir / 'weights'wdir.mkdir(parents=True, exist_ok=True)  # make dirlast = wdir / 'last.pt'best = wdir / 'best.pt'results_file = save_dir / 'results.txt'# Save run settingswith open(save_dir / 'hyp.yaml', 'w') as f:yaml.dump(hyp, f, sort_keys=False)with open(save_dir / 'opt.yaml', 'w') as f:yaml.dump(vars(opt), f, sort_keys=False)# Configureplots = not opt.evolve  # create plotscuda = device.type != 'cpu'init_seeds(2 + rank)with open(opt.data) as f:data_dict = yaml.load(f, Loader=yaml.SafeLoader)  # data dictis_coco = opt.data.endswith('coco.yaml')# Logging- Doing this before checking the dataset. Might update data_dictloggers = {'wandb': None}  # loggers dictif rank in [-1, 0]:opt.hyp = hyp  # add hyperparametersrun_id = torch.load(weights, map_location=device).get('wandb_id') if weights.endswith('.pt') and os.path.isfile(weights) else Nonewandb_logger = WandbLogger(opt, Path(opt.save_dir).stem, run_id, data_dict)loggers['wandb'] = wandb_logger.wandbdata_dict = wandb_logger.data_dictif wandb_logger.wandb:weights, epochs, hyp = opt.weights, opt.epochs, opt.hyp  # WandbLogger might update weights, epochs if resumingnc = 1 if opt.single_cls else int(data_dict['nc'])  # number of classesnames = ['item'] if opt.single_cls and len(data_dict['names']) != 1 else data_dict['names']  # class namesassert len(names) == nc, '%g names found for nc=%g dataset in %s' % (len(names), nc, opt.data)  # check# Modelpretrained = weights.endswith('.pt')if pretrained:with torch_distributed_zero_first(rank):attempt_download(weights)  # download if not found locallyckpt = torch.load(weights, map_location=device)  # load checkpointmodel = Model(opt.cfg or ckpt['model'].yaml, ch=3, nc=nc, anchors=hyp.get('anchors')).to(device)  # createexclude = ['anchor'] if (opt.cfg or hyp.get('anchors')) and not opt.resume else []  # exclude keysstate_dict = ckpt['model'].float().state_dict()  # to FP32state_dict = intersect_dicts(state_dict, model.state_dict(), exclude=exclude)  # intersectmodel.load_state_dict(state_dict, strict=False)  # loadlogger.info('Transferred %g/%g items from %s' % (len(state_dict), len(model.state_dict()), weights))  # reportelse:model = Model(opt.cfg, ch=3, nc=nc, anchors=hyp.get('anchors')).to(device)  # createwith torch_distributed_zero_first(rank):check_dataset(data_dict)  # checktrain_path = data_dict['train']test_path = data_dict['val']# Freezefreeze = [f'model.{x}.' for x in (freeze if len(freeze) > 1 else range(freeze[0]))]  # parameter names to freeze (full or partial)for k, v in model.named_parameters():v.requires_grad = True  # train all layersif any(x in k for x in freeze):print('freezing %s' % k)v.requires_grad = False# Optimizernbs = 64  # nominal batch sizeaccumulate = max(round(nbs / total_batch_size), 1)  # accumulate loss before optimizinghyp['weight_decay'] *= total_batch_size * accumulate / nbs  # scale weight_decaylogger.info(f"Scaled weight_decay = {hyp['weight_decay']}")pg0, pg1, pg2 = [], [], []  # optimizer parameter groupsfor k, v in model.named_modules():if hasattr(v, 'bias') and isinstance(v.bias, nn.Parameter):pg2.append(v.bias)  # biasesif isinstance(v, nn.BatchNorm2d):pg0.append(v.weight)  # no decayelif hasattr(v, 'weight') and isinstance(v.weight, nn.Parameter):pg1.append(v.weight)  # apply decayif hasattr(v, 'im'):if hasattr(v.im, 'implicit'):           pg0.append(v.im.implicit)else:for iv in v.im:pg0.append(iv.implicit)if hasattr(v, 'imc'):if hasattr(v.imc, 'implicit'):           pg0.append(v.imc.implicit)else:for iv in v.imc:pg0.append(iv.implicit)if hasattr(v, 'imb'):if hasattr(v.imb, 'implicit'):           pg0.append(v.imb.implicit)else:for iv in v.imb:pg0.append(iv.implicit)if hasattr(v, 'imo'):if hasattr(v.imo, 'implicit'):           pg0.append(v.imo.implicit)else:for iv in v.imo:pg0.append(iv.implicit)if hasattr(v, 'ia'):if hasattr(v.ia, 'implicit'):           pg0.append(v.ia.implicit)else:for iv in v.ia:pg0.append(iv.implicit)if hasattr(v, 'attn'):if hasattr(v.attn, 'logit_scale'):   pg0.append(v.attn.logit_scale)if hasattr(v.attn, 'q_bias'):   pg0.append(v.attn.q_bias)if hasattr(v.attn, 'v_bias'):  pg0.append(v.attn.v_bias)if hasattr(v.attn, 'relative_position_bias_table'):  pg0.append(v.attn.relative_position_bias_table)if hasattr(v, 'rbr_dense'):if hasattr(v.rbr_dense, 'weight_rbr_origin'):  pg0.append(v.rbr_dense.weight_rbr_origin)if hasattr(v.rbr_dense, 'weight_rbr_avg_conv'): pg0.append(v.rbr_dense.weight_rbr_avg_conv)if hasattr(v.rbr_dense, 'weight_rbr_pfir_conv'):  pg0.append(v.rbr_dense.weight_rbr_pfir_conv)if hasattr(v.rbr_dense, 'weight_rbr_1x1_kxk_idconv1'): pg0.append(v.rbr_dense.weight_rbr_1x1_kxk_idconv1)if hasattr(v.rbr_dense, 'weight_rbr_1x1_kxk_conv2'):   pg0.append(v.rbr_dense.weight_rbr_1x1_kxk_conv2)if hasattr(v.rbr_dense, 'weight_rbr_gconv_dw'):   pg0.append(v.rbr_dense.weight_rbr_gconv_dw)if hasattr(v.rbr_dense, 'weight_rbr_gconv_pw'):   pg0.append(v.rbr_dense.weight_rbr_gconv_pw)if hasattr(v.rbr_dense, 'vector'):   pg0.append(v.rbr_dense.vector)if opt.adam:optimizer = optim.Adam(pg0, lr=hyp['lr0'], betas=(hyp['momentum'], 0.999))  # adjust beta1 to momentumelse:optimizer = optim.SGD(pg0, lr=hyp['lr0'], momentum=hyp['momentum'], nesterov=True)optimizer.add_param_group({'params': pg1, 'weight_decay': hyp['weight_decay']})  # add pg1 with weight_decayoptimizer.add_param_group({'params': pg2})  # add pg2 (biases)logger.info('Optimizer groups: %g .bias, %g conv.weight, %g other' % (len(pg2), len(pg1), len(pg0)))del pg0, pg1, pg2# Scheduler https://arxiv.org/pdf/1812.01187.pdf# https://pytorch.org/docs/stable/_modules/torch/optim/lr_scheduler.html#OneCycleLRif opt.linear_lr:lf = lambda x: (1 - x / (epochs - 1)) * (1.0 - hyp['lrf']) + hyp['lrf']  # linearelse:lf = one_cycle(1, hyp['lrf'], epochs)  # cosine 1->hyp['lrf']scheduler = lr_scheduler.LambdaLR(optimizer, lr_lambda=lf)# plot_lr_scheduler(optimizer, scheduler, epochs)# EMAema = ModelEMA(model) if rank in [-1, 0] else None# Resumestart_epoch, best_fitness = 0, 0.0if pretrained:# Optimizerif ckpt['optimizer'] is not None:optimizer.load_state_dict(ckpt['optimizer'])best_fitness = ckpt['best_fitness']# EMAif ema and ckpt.get('ema'):ema.ema.load_state_dict(ckpt['ema'].float().state_dict())ema.updates = ckpt['updates']# Resultsif ckpt.get('training_results') is not None:results_file.write_text(ckpt['training_results'])  # write results.txt# Epochsstart_epoch = ckpt['epoch'] + 1if opt.resume:assert start_epoch > 0, '%s training to %g epochs is finished, nothing to resume.' % (weights, epochs)if epochs < start_epoch:logger.info('%s has been trained for %g epochs. Fine-tuning for %g additional epochs.' %(weights, ckpt['epoch'], epochs))epochs += ckpt['epoch']  # finetune additional epochsdel ckpt, state_dict# Image sizesgs = max(int(model.stride.max()), 32)  # grid size (max stride)nl = model.model[-1].nl  # number of detection layers (used for scaling hyp['obj'])imgsz, imgsz_test = [check_img_size(x, gs) for x in opt.img_size]  # verify imgsz are gs-multiples# DP modeif cuda and rank == -1 and torch.cuda.device_count() > 1:model = torch.nn.DataParallel(model)# SyncBatchNormif opt.sync_bn and cuda and rank != -1:model = torch.nn.SyncBatchNorm.convert_sync_batchnorm(model).to(device)logger.info('Using SyncBatchNorm()')# Trainloaderdataloader, dataset = create_dataloader(train_path, imgsz, batch_size, gs, opt,hyp=hyp, augment=True, cache=opt.cache_images, rect=opt.rect, rank=rank,world_size=opt.world_size, workers=opt.workers,image_weights=opt.image_weights, quad=opt.quad, prefix=colorstr('train: '))mlc = np.concatenate(dataset.labels, 0)[:, 0].max()  # max label classnb = len(dataloader)  # number of batchesassert mlc < nc, 'Label class %g exceeds nc=%g in %s. Possible class labels are 0-%g' % (mlc, nc, opt.data, nc - 1)# Process 0if rank in [-1, 0]:testloader = create_dataloader(test_path, imgsz_test, batch_size * 2, gs, opt,  # testloaderhyp=hyp, cache=opt.cache_images and not opt.notest, rect=True, rank=-1,world_size=opt.world_size, workers=opt.workers,pad=0.5, prefix=colorstr('val: '))[0]if not opt.resume:labels = np.concatenate(dataset.labels, 0)c = torch.tensor(labels[:, 0])  # classes# cf = torch.bincount(c.long(), minlength=nc) + 1.  # frequency# model._initialize_biases(cf.to(device))if plots:#plot_labels(labels, names, save_dir, loggers)if tb_writer:tb_writer.add_histogram('classes', c, 0)# Anchorsif not opt.noautoanchor:check_anchors(dataset, model=model, thr=hyp['anchor_t'], imgsz=imgsz)model.half().float()  # pre-reduce anchor precision# DDP modeif cuda and rank != -1:model = DDP(model, device_ids=[opt.local_rank], output_device=opt.local_rank,# nn.MultiheadAttention incompatibility with DDP https://github.com/pytorch/pytorch/issues/26698find_unused_parameters=any(isinstance(layer, nn.MultiheadAttention) for layer in model.modules()))# Model parametershyp['box'] *= 3. / nl  # scale to layershyp['cls'] *= nc / 80. * 3. / nl  # scale to classes and layershyp['obj'] *= (imgsz / 640) ** 2 * 3. / nl  # scale to image size and layershyp['label_smoothing'] = opt.label_smoothingmodel.nc = nc  # attach number of classes to modelmodel.hyp = hyp  # attach hyperparameters to modelmodel.gr = 1.0  # iou loss ratio (obj_loss = 1.0 or iou)model.class_weights = labels_to_class_weights(dataset.labels, nc).to(device) * nc  # attach class weightsmodel.names = names# Start trainingt0 = time.time()nw = max(round(hyp['warmup_epochs'] * nb), 1000)  # number of warmup iterations, max(3 epochs, 1k iterations)# nw = min(nw, (epochs - start_epoch) / 2 * nb)  # limit warmup to < 1/2 of trainingmaps = np.zeros(nc)  # mAP per classresults = (0, 0, 0, 0, 0, 0, 0)  # P, R, mAP@.5, mAP@.5-.95, val_loss(box, obj, cls)scheduler.last_epoch = start_epoch - 1  # do not movescaler = amp.GradScaler(enabled=cuda)compute_loss_ota = ComputeLossOTA(model)  # init loss classcompute_loss = ComputeLoss(model)  # init loss classlogger.info(f'Image sizes {imgsz} train, {imgsz_test} test\n'f'Using {dataloader.num_workers} dataloader workers\n'f'Logging results to {save_dir}\n'f'Starting training for {epochs} epochs...')torch.save(model, wdir / 'init.pt')for epoch in range(start_epoch, epochs):  # epoch ------------------------------------------------------------------model.train()# Update image weights (optional)if opt.image_weights:# Generate indicesif rank in [-1, 0]:cw = model.class_weights.cpu().numpy() * (1 - maps) ** 2 / nc  # class weightsiw = labels_to_image_weights(dataset.labels, nc=nc, class_weights=cw)  # image weightsdataset.indices = random.choices(range(dataset.n), weights=iw, k=dataset.n)  # rand weighted idx# Broadcast if DDPif rank != -1:indices = (torch.tensor(dataset.indices) if rank == 0 else torch.zeros(dataset.n)).int()dist.broadcast(indices, 0)if rank != 0:dataset.indices = indices.cpu().numpy()# Update mosaic border# b = int(random.uniform(0.25 * imgsz, 0.75 * imgsz + gs) // gs * gs)# dataset.mosaic_border = [b - imgsz, -b]  # height, width bordersmloss = torch.zeros(4, device=device)  # mean lossesif rank != -1:dataloader.sampler.set_epoch(epoch)pbar = enumerate(dataloader)logger.info(('\n' + '%10s' * 8) % ('Epoch', 'gpu_mem', 'box', 'obj', 'cls', 'total', 'labels', 'img_size'))if rank in [-1, 0]:pbar = tqdm(pbar, total=nb)  # progress baroptimizer.zero_grad()for i, (imgs, targets, paths, _) in pbar:  # batch -------------------------------------------------------------ni = i + nb * epoch  # number integrated batches (since train start)imgs = imgs.to(device, non_blocking=True).float() / 255.0  # uint8 to float32, 0-255 to 0.0-1.0# Warmupif ni <= nw:xi = [0, nw]  # x interp# model.gr = np.interp(ni, xi, [0.0, 1.0])  # iou loss ratio (obj_loss = 1.0 or iou)accumulate = max(1, np.interp(ni, xi, [1, nbs / total_batch_size]).round())for j, x in enumerate(optimizer.param_groups):# bias lr falls from 0.1 to lr0, all other lrs rise from 0.0 to lr0x['lr'] = np.interp(ni, xi, [hyp['warmup_bias_lr'] if j == 2 else 0.0, x['initial_lr'] * lf(epoch)])if 'momentum' in x:x['momentum'] = np.interp(ni, xi, [hyp['warmup_momentum'], hyp['momentum']])# Multi-scaleif opt.multi_scale:sz = random.randrange(imgsz * 0.5, imgsz * 1.5 + gs) // gs * gs  # sizesf = sz / max(imgs.shape[2:])  # scale factorif sf != 1:ns = [math.ceil(x * sf / gs) * gs for x in imgs.shape[2:]]  # new shape (stretched to gs-multiple)imgs = F.interpolate(imgs, size=ns, mode='bilinear', align_corners=False)# Forwardwith amp.autocast(enabled=cuda):pred = model(imgs)  # forwardif 'loss_ota' not in hyp or hyp['loss_ota'] == 1:loss, loss_items = compute_loss_ota(pred, targets.to(device), imgs)  # loss scaled by batch_sizeelse:loss, loss_items = compute_loss(pred, targets.to(device))  # loss scaled by batch_sizeif rank != -1:loss *= opt.world_size  # gradient averaged between devices in DDP modeif opt.quad:loss *= 4.# Backwardscaler.scale(loss).backward()# Optimizeif ni % accumulate == 0:scaler.step(optimizer)  # optimizer.stepscaler.update()optimizer.zero_grad()if ema:ema.update(model)# Printif rank in [-1, 0]:mloss = (mloss * i + loss_items) / (i + 1)  # update mean lossesmem = '%.3gG' % (torch.cuda.memory_reserved() / 1E9 if torch.cuda.is_available() else 0)  # (GB)s = ('%10s' * 2 + '%10.4g' * 6) % ('%g/%g' % (epoch, epochs - 1), mem, *mloss, targets.shape[0], imgs.shape[-1])pbar.set_description(s)# Plotif plots and ni < 10:f = save_dir / f'train_batch{ni}.jpg'  # filenameThread(target=plot_images, args=(imgs, targets, paths, f), daemon=True).start()# if tb_writer:#     tb_writer.add_image(f, result, dataformats='HWC', global_step=epoch)#     tb_writer.add_graph(torch.jit.trace(model, imgs, strict=False), [])  # add model graphelif plots and ni == 10 and wandb_logger.wandb:wandb_logger.log({"Mosaics": [wandb_logger.wandb.Image(str(x), caption=x.name) for x insave_dir.glob('train*.jpg') if x.exists()]})# end batch ------------------------------------------------------------------------------------------------# end epoch ----------------------------------------------------------------------------------------------------# Schedulerlr = [x['lr'] for x in optimizer.param_groups]  # for tensorboardscheduler.step()# DDP process 0 or single-GPUif rank in [-1, 0]:# mAPema.update_attr(model, include=['yaml', 'nc', 'hyp', 'gr', 'names', 'stride', 'class_weights'])final_epoch = epoch + 1 == epochsif not opt.notest or final_epoch:  # Calculate mAPwandb_logger.current_epoch = epoch + 1results, maps, times = test.test(data_dict,batch_size=batch_size * 2,imgsz=imgsz_test,model=ema.ema,single_cls=opt.single_cls,dataloader=testloader,save_dir=save_dir,verbose=nc < 50 and final_epoch,plots=plots and final_epoch,wandb_logger=wandb_logger,compute_loss=compute_loss,is_coco=is_coco)# Writewith open(results_file, 'a') as f:f.write(s + '%10.4g' * 7 % results + '\n')  # append metrics, val_lossif len(opt.name) and opt.bucket:os.system('gsutil cp %s gs://%s/results/results%s.txt' % (results_file, opt.bucket, opt.name))# Logtags = ['train/box_loss', 'train/obj_loss', 'train/cls_loss',  # train loss'metrics/precision', 'metrics/recall', 'metrics/mAP_0.5', 'metrics/mAP_0.5:0.95','val/box_loss', 'val/obj_loss', 'val/cls_loss',  # val loss'x/lr0', 'x/lr1', 'x/lr2']  # paramsfor x, tag in zip(list(mloss[:-1]) + list(results) + lr, tags):if tb_writer:tb_writer.add_scalar(tag, x, epoch)  # tensorboardif wandb_logger.wandb:wandb_logger.log({tag: x})  # W&B# Update best mAPfi = fitness(np.array(results).reshape(1, -1))  # weighted combination of [P, R, mAP@.5, mAP@.5-.95]if fi > best_fitness:best_fitness = fiwandb_logger.end_epoch(best_result=best_fitness == fi)# Save modelif (not opt.nosave) or (final_epoch and not opt.evolve):  # if saveckpt = {'epoch': epoch,'best_fitness': best_fitness,'training_results': results_file.read_text(),'model': deepcopy(model.module if is_parallel(model) else model).half(),'ema': deepcopy(ema.ema).half(),'updates': ema.updates,'optimizer': optimizer.state_dict(),'wandb_id': wandb_logger.wandb_run.id if wandb_logger.wandb else None}# Save last, best and deletetorch.save(ckpt, last)if best_fitness == fi:torch.save(ckpt, best)if (best_fitness == fi) and (epoch >= 200):torch.save(ckpt, wdir / 'best_{:03d}.pt'.format(epoch))if epoch == 0:torch.save(ckpt, wdir / 'epoch_{:03d}.pt'.format(epoch))elif ((epoch+1) % 25) == 0:torch.save(ckpt, wdir / 'epoch_{:03d}.pt'.format(epoch))elif epoch >= (epochs-5):torch.save(ckpt, wdir / 'epoch_{:03d}.pt'.format(epoch))if wandb_logger.wandb:if ((epoch + 1) % opt.save_period == 0 and not final_epoch) and opt.save_period != -1:wandb_logger.log_model(last.parent, opt, epoch, fi, best_model=best_fitness == fi)del ckpt# end epoch ----------------------------------------------------------------------------------------------------# end trainingif rank in [-1, 0]:# Plotsif plots:plot_results(save_dir=save_dir)  # save as results.pngif wandb_logger.wandb:files = ['results.png', 'confusion_matrix.png', *[f'{x}_curve.png' for x in ('F1', 'PR', 'P', 'R')]]wandb_logger.log({"Results": [wandb_logger.wandb.Image(str(save_dir / f), caption=f) for f in filesif (save_dir / f).exists()]})# Test best.ptlogger.info('%g epochs completed in %.3f hours.\n' % (epoch - start_epoch + 1, (time.time() - t0) / 3600))if opt.data.endswith('coco.yaml') and nc == 80:  # if COCOfor m in (last, best) if best.exists() else (last):  # speed, mAP testsresults, _, _ = test.test(opt.data,batch_size=batch_size * 2,imgsz=imgsz_test,conf_thres=0.001,iou_thres=0.7,model=attempt_load(m, device).half(),single_cls=opt.single_cls,dataloader=testloader,save_dir=save_dir,save_json=True,plots=False,is_coco=is_coco)# Strip optimizersfinal = best if best.exists() else last  # final modelfor f in last, best:if f.exists():strip_optimizer(f)  # strip optimizersif opt.bucket:os.system(f'gsutil cp {final} gs://{opt.bucket}/weights')  # uploadif wandb_logger.wandb and not opt.evolve:  # Log the stripped modelwandb_logger.wandb.log_artifact(str(final), type='model',name='run_' + wandb_logger.wandb_run.id + '_model',aliases=['last', 'best', 'stripped'])wandb_logger.finish_run()else:dist.destroy_process_group()torch.cuda.empty_cache()return resultsif __name__ == '__main__':parser = argparse.ArgumentParser()parser.add_argument('--weights', type=str, default='yolov7.pt', help='initial weights path')parser.add_argument('--cfg', type=str, default='cfg/training/yolov7.yaml', help='model.yaml path')parser.add_argument('--data', type=str, default='data/coco.yaml', help='data.yaml path')parser.add_argument('--hyp', type=str, default='data/hyp.scratch.p5.yaml', help='hyperparameters path')parser.add_argument('--epochs', type=int, default=300)parser.add_argument('--batch-size', type=int, default=4, help='total batch size for all GPUs')parser.add_argument('--img-size', nargs='+', type=int, default=[640, 640], help='[train, test] image sizes')parser.add_argument('--rect', action='store_true', help='rectangular training')parser.add_argument('--resume', nargs='?', const=True, default=False, help='resume most recent training')parser.add_argument('--nosave', action='store_true', help='only save final checkpoint')parser.add_argument('--notest', action='store_true', help='only test final epoch')parser.add_argument('--noautoanchor', action='store_true', help='disable autoanchor check')parser.add_argument('--evolve', action='store_true', help='evolve hyperparameters')parser.add_argument('--bucket', type=str, default='', help='gsutil bucket')parser.add_argument('--cache-images', action='store_true', help='cache images for faster training')parser.add_argument('--image-weights', action='store_true', help='use weighted image selection for training')parser.add_argument('--device', default='0', help='cuda device, i.e. 0 or 0,1,2,3 or cpu')parser.add_argument('--multi-scale', action='store_true', help='vary img-size +/- 50%%')parser.add_argument('--single-cls', action='store_true', help='train multi-class data as single-class')parser.add_argument('--adam', action='store_true', help='use torch.optim.Adam() optimizer')parser.add_argument('--sync-bn', action='store_true', help='use SyncBatchNorm, only available in DDP mode')parser.add_argument('--local_rank', type=int, default=-1, help='DDP parameter, do not modify')parser.add_argument('--workers', type=int, default=0, help='maximum number of dataloader workers')parser.add_argument('--project', default='runs/train', help='save to project/name')parser.add_argument('--entity', default=None, help='W&B entity')parser.add_argument('--name', default='exp', help='save to project/name')parser.add_argument('--exist-ok', action='store_true', help='existing project/name ok, do not increment')parser.add_argument('--quad', action='store_true', help='quad dataloader')parser.add_argument('--linear-lr', action='store_true', help='linear LR')parser.add_argument('--label-smoothing', type=float, default=0.0, help='Label smoothing epsilon')parser.add_argument('--upload_dataset', action='store_true', help='Upload dataset as W&B artifact table')parser.add_argument('--bbox_interval', type=int, default=-1, help='Set bounding-box image logging interval for W&B')parser.add_argument('--save_period', type=int, default=-1, help='Log model after every "save_period" epoch')parser.add_argument('--artifact_alias', type=str, default="latest", help='version of dataset artifact to be used')parser.add_argument('--freeze', nargs='+', type=int, default=[0], help='Freeze layers: backbone of yolov7=50, first3=0 1 2')opt = parser.parse_args()# Set DDP variablesopt.world_size = int(os.environ['WORLD_SIZE']) if 'WORLD_SIZE' in os.environ else 1opt.global_rank = int(os.environ['RANK']) if 'RANK' in os.environ else -1set_logging(opt.global_rank)#if opt.global_rank in [-1, 0]:#    check_git_status()#    check_requirements()# Resumewandb_run = check_wandb_resume(opt)if opt.resume and not wandb_run:  # resume an interrupted runckpt = opt.resume if isinstance(opt.resume, str) else get_latest_run()  # specified or most recent pathassert os.path.isfile(ckpt), 'ERROR: --resume checkpoint does not exist'apriori = opt.global_rank, opt.local_rankwith open(Path(ckpt).parent.parent / 'opt.yaml') as f:opt = argparse.Namespace(**yaml.load(f, Loader=yaml.SafeLoader))  # replaceopt.cfg, opt.weights, opt.resume, opt.batch_size, opt.global_rank, opt.local_rank = '', ckpt, True, opt.total_batch_size, *apriori  # reinstatelogger.info('Resuming training from %s' % ckpt)else:# opt.hyp = opt.hyp or ('hyp.finetune.yaml' if opt.weights else 'hyp.scratch.yaml')opt.data, opt.cfg, opt.hyp = check_file(opt.data), check_file(opt.cfg), check_file(opt.hyp)  # check filesassert len(opt.cfg) or len(opt.weights), 'either --cfg or --weights must be specified'opt.img_size.extend([opt.img_size[-1]] * (2 - len(opt.img_size)))  # extend to 2 sizes (train, test)opt.name = 'evolve' if opt.evolve else opt.nameopt.save_dir = increment_path(Path(opt.project) / opt.name, exist_ok=opt.exist_ok | opt.evolve)  # increment run# DDP modeopt.total_batch_size = opt.batch_sizedevice = select_device(opt.device, batch_size=opt.batch_size)if opt.local_rank != -1:assert torch.cuda.device_count() > opt.local_ranktorch.cuda.set_device(opt.local_rank)device = torch.device('cuda', opt.local_rank)dist.init_process_group(backend='nccl', init_method='env://')  # distributed backendassert opt.batch_size % opt.world_size == 0, '--batch-size must be multiple of CUDA device count'opt.batch_size = opt.total_batch_size // opt.world_size# Hyperparameterswith open(opt.hyp) as f:hyp = yaml.load(f, Loader=yaml.SafeLoader)  # load hyps# Trainlogger.info(opt)if not opt.evolve:tb_writer = None  # init loggersif opt.global_rank in [-1, 0]:prefix = colorstr('tensorboard: ')logger.info(f"{prefix}Start with 'tensorboard --logdir {opt.project}', view at http://localhost:6006/")tb_writer = SummaryWriter(opt.save_dir)  # Tensorboardtrain(hyp, opt, device, tb_writer)# Evolve hyperparameters (optional)else:# Hyperparameter evolution metadata (mutation scale 0-1, lower_limit, upper_limit)meta = {'lr0': (1, 1e-5, 1e-1),  # initial learning rate (SGD=1E-2, Adam=1E-3)'lrf': (1, 0.01, 1.0),  # final OneCycleLR learning rate (lr0 * lrf)'momentum': (0.3, 0.6, 0.98),  # SGD momentum/Adam beta1'weight_decay': (1, 0.0, 0.001),  # optimizer weight decay'warmup_epochs': (1, 0.0, 5.0),  # warmup epochs (fractions ok)'warmup_momentum': (1, 0.0, 0.95),  # warmup initial momentum'warmup_bias_lr': (1, 0.0, 0.2),  # warmup initial bias lr'box': (1, 0.02, 0.2),  # box loss gain'cls': (1, 0.2, 4.0),  # cls loss gain'cls_pw': (1, 0.5, 2.0),  # cls BCELoss positive_weight'obj': (1, 0.2, 4.0),  # obj loss gain (scale with pixels)'obj_pw': (1, 0.5, 2.0),  # obj BCELoss positive_weight'iou_t': (0, 0.1, 0.7),  # IoU training threshold'anchor_t': (1, 2.0, 8.0),  # anchor-multiple threshold'anchors': (2, 2.0, 10.0),  # anchors per output grid (0 to ignore)'fl_gamma': (0, 0.0, 2.0),  # focal loss gamma (efficientDet default gamma=1.5)'hsv_h': (1, 0.0, 0.1),  # image HSV-Hue augmentation (fraction)'hsv_s': (1, 0.0, 0.9),  # image HSV-Saturation augmentation (fraction)'hsv_v': (1, 0.0, 0.9),  # image HSV-Value augmentation (fraction)'degrees': (1, 0.0, 45.0),  # image rotation (+/- deg)'translate': (1, 0.0, 0.9),  # image translation (+/- fraction)'scale': (1, 0.0, 0.9),  # image scale (+/- gain)'shear': (1, 0.0, 10.0),  # image shear (+/- deg)'perspective': (0, 0.0, 0.001),  # image perspective (+/- fraction), range 0-0.001'flipud': (1, 0.0, 1.0),  # image flip up-down (probability)'fliplr': (0, 0.0, 1.0),  # image flip left-right (probability)'mosaic': (1, 0.0, 1.0),  # image mixup (probability)'mixup': (1, 0.0, 1.0),   # image mixup (probability)'copy_paste': (1, 0.0, 1.0),  # segment copy-paste (probability)'paste_in': (1, 0.0, 1.0)}    # segment copy-paste (probability)with open(opt.hyp, errors='ignore') as f:hyp = yaml.safe_load(f)  # load hyps dictif 'anchors' not in hyp:  # anchors commented in hyp.yamlhyp['anchors'] = 3assert opt.local_rank == -1, 'DDP mode not implemented for --evolve'opt.notest, opt.nosave = True, True  # only test/save final epoch# ei = [isinstance(x, (int, float)) for x in hyp.values()]  # evolvable indicesyaml_file = Path(opt.save_dir) / 'hyp_evolved.yaml'  # save best result hereif opt.bucket:os.system('gsutil cp gs://%s/evolve.txt .' % opt.bucket)  # download evolve.txt if existsfor _ in range(300):  # generations to evolveif Path('evolve.txt').exists():  # if evolve.txt exists: select best hyps and mutate# Select parent(s)parent = 'single'  # parent selection method: 'single' or 'weighted'x = np.loadtxt('evolve.txt', ndmin=2)n = min(5, len(x))  # number of previous results to considerx = x[np.argsort(-fitness(x))][:n]  # top n mutationsw = fitness(x) - fitness(x).min()  # weightsif parent == 'single' or len(x) == 1:# x = x[random.randint(0, n - 1)]  # random selectionx = x[random.choices(range(n), weights=w)[0]]  # weighted selectionelif parent == 'weighted':x = (x * w.reshape(n, 1)).sum(0) / w.sum()  # weighted combination# Mutatemp, s = 0.8, 0.2  # mutation probability, sigmanpr = np.randomnpr.seed(int(time.time()))g = np.array([x[0] for x in meta.values()])  # gains 0-1ng = len(meta)v = np.ones(ng)while all(v == 1):  # mutate until a change occurs (prevent duplicates)v = (g * (npr.random(ng) < mp) * npr.randn(ng) * npr.random() * s + 1).clip(0.3, 3.0)for i, k in enumerate(hyp.keys()):  # plt.hist(v.ravel(), 300)hyp[k] = float(x[i + 7] * v[i])  # mutate# Constrain to limitsfor k, v in meta.items():hyp[k] = max(hyp[k], v[1])  # lower limithyp[k] = min(hyp[k], v[2])  # upper limithyp[k] = round(hyp[k], 5)  # significant digits# Train mutationresults = train(hyp.copy(), opt, device)# Write mutation resultsprint_mutation(hyp.copy(), results, yaml_file, opt.bucket)# Plot resultsplot_evolution(yaml_file)print(f'Hyperparameter evolution complete. Best results saved as: {yaml_file}\n'f'Command to train a new model with these hyperparameters: $ python train.py --hyp {yaml_file}')

7.CRNN简介

现今基于深度学习的端到端OCR技术有两大主流技术：CRNN OCR和attention OCR。其实这两大方法主要区别在于最后的输出层（翻译层），即怎么将网络学习到的序列特征信息转化为最终的识别结果。这两大主流技术在其特征学习阶段都采用了CNN+RNN的网络结构，CRNN OCR在对齐时采取的方式是CTC算法，而attention OCR采取的方式则是attention机制。本部分主要介绍应用更为广泛的CRNN算法。

8.CRNN网络结构

CRNN全称为Convolutional Recurrent Neural Network，主要用于端到端地对不定长的文本序列进行识别，不用先对单个文字进行切割，而是将文本识别转化为时序依赖的序列学习问题，就是基于图像的序列识别。

因为RNN有梯度消失的问题，不能获取更多上下文信息，所以CRNN中使用的是LSTM，LSTM的特殊设计允许它捕获长距离依赖。

LSTM是单向的，它只使用过去的信息。然而，在基于图像的序列中，两个方向的上下文是相互有用且互补的。将两个LSTM，一个向前和一个向后组合到一个双向LSTM中。此外，可以堆叠多层双向LSTM，深层结构允许比浅层抽象更高层次的抽象。

参考该博客采用的是两层各256单元的双向LSTM网络：

通过上面一步，我们得到了40个特征向量，每个特征向量长度为512，在LSTM中一个时间步就传入一个特征向量进行分类，这里一共有40个时间步。

我们知道一个特征向量就相当于原图中的一个小矩形区域，RNN的目标就是预测这个矩形区域为哪个字符，即根据输入的特征向量，进行预测，得到所有字符的softmax概率分布，这是一个长度为字符类别数的向量，作为CTC层的输入。

因为每个时间步都会有一个输入特征向量xT，输出一个所有字符的概率分布yT，所以输出为40个长度为字符类别数的向量构成的后验概率矩阵。如下图所示：

9.系统整合

下图源码&环境部署视频教程&数据集&自定义UI界面

参考博客《Python基于YOLOv7和CRNN的双层车牌分割＆识别系统（源码＆教程）》

10.参考文献

[1]黄山.车牌识别技术的研究和实现[D].2005,
[2]李小平,任江兴,杨德刚.车牌识别系统中若干问题的探讨[J].北京理工大学学报,2001,
21(1): 116-119
[3]葛海江,方江江,张翔.车牌识别系统中车牌定位及倾斜矫正算法研究[J].杭州电子科技大
学学报, 2007,2: 49-52
[4]S.M.Silva,C.R.Jung. License plate detection and recognition in unconstrained scenarios[C].
Proceedings of the European conference on computer vision (ECCV), 2018,580-596
[5]韩立明,王波涛.车牌识别中关键技术的研究与实现[J.计算机工程与设计,2010,31(17):
2919-3923
[6]D.Shan， M.Ibrahim, M.Shehata, et al.Automatic license plate recognition (alpr): A statc-of-
the-art review[J].IEEE Transactions on Circuits and Systems for Video Technology,2013,23(2):311-325
[7]T.Manson. Automatic license plate recognition systems[J]. Bureau of Justice Statistics, 2006
[8]S. Du,M.Ibrahim,M. Shehata, et al.Automatic license plate recognition (alpr): A statc-of-the-artreview[J].IEEE Transactions on circuits and systems for video technology, 2012,23(2): 311-325
[9]G.R.Goncalves, D. Menotti,W.R. Schwartz. License plate recognition based on temporal re-
dundancy[C]. 2016 IEEE 19th International Conference on Intelligent Transportation Systems(ITSC),2016,2577-2582
[10] S. Montazzolli, C.Jung.Real-time brazilian license plate detection and recognition using deep
convolutional neural networks[C]. 2017 30th SIBGRAPl conference on graphics, patterns andimages (SIBGRAPI),2017,55-62
[11] J.Spaihel, J. Sochor,R.Juranek, et al.Holistic recognition of low quality license plates by cnn
using track annotated data[C].2017 14th IEEE International Conference on Advanced Video andSignal Based Surveillance (AVSS), 2017,1-6
[12] Y.LeCun, Y.Bengio,G.Hinton. Deep learning[J]. nature, 2015,521(7553): 436-444
[13] H. Li, P. Wang, M. You, et al. Reading car license plates using deep neural networks[J]. Imageand Vision Computing, 2018,72: 14-23

Python基于YOLOv7和CRNN的车牌分割＆识别系统（源码＆教程）相关推荐

Python基于OpenCV的人脸表情识别系统[源码＆部署教程]
1.项目背景人脸表情识别是模式识别中一个非常重要却十分复杂的课题.首先对计算机人脸表情识别技术的研究背景及发展历程作了简单回顾.然后对近期人脸表情识别的方法进行了分类综述.通过对各种识别方法的分析与 ...
python本科毕业设计基于神经网络的虚假评论识别系统源码，含模型及数据
主要函数: 1.corpusprocess原始语料处理函数 2.train_word2vec生成word2vec向量 3.generate_id2wec获得索引的w2id,和嵌入权重embedding ...
基于Python tensorflow2.3实现的水果识别系统源码+模型+数据集，卷积神经网络的入门案例
水果识别-基于tensorflow2.3实现水果识别是卷积神经网络的入门案例,这里我将模型的训练.测试.保存以及使用整合在了一起,至于原理部分,大家可以参考知乎或者B站上的回答,在这里我就不赘述了 ...
Python基于YOLOv5的交通标志识别系统[源码]
1.图片演示: 2.视频演示: [项目分享]Python基于YOLOv5的交通标志识别系统[源码&技术文档&部署视频&数据集]_哔哩哔哩_bilibili 3.标注好的数据集: ...
基于Java毕业设计校园线上点餐系统源码+系统+mysql+lw文档+部署软件
基于Java毕业设计校园线上点餐系统源码+系统+mysql+lw文档+部署软件基于Java毕业设计校园线上点餐系统源码+系统+mysql+lw文档+部署软件本源码技术栈: 项目架构:B/S架构开 ...
Python＆OpenCV自动人脸打马赛克＆调色系统[源码＆UI操作界面＆部署教程]
1.视频演示: [项目分享]Python&OpenCV自动人脸打马赛克&调色系统[源码&UI操作界面&部署教程] 2.图片演示: 3.图像颜色检索&替换: 4. ...
基于tensorflow2.0+opencv的花卉识别系统源码（含数据集）
花卉识别-基于tensorflow2.3实现完整代码下载地址:基于tensorflow2.0+opencv的花卉识别系统源码( 文件目录 # 数据下载地址 https://storage.googl ...
基于Java毕业设计新疆旅游专列订票系统源码+系统+mysql+lw文档+部署软件
基于Java毕业设计新疆旅游专列订票系统源码+系统+mysql+lw文档+部署软件基于Java毕业设计新疆旅游专列订票系统源码+系统+mysql+lw文档+部署软件本源码技术栈: 项目架构:B/S ...
亲测无限坐席在线客服系统源码，基于ThinkPHP的一款在线客服系统源码
源码简介东西没问题,和别人换的本来说是多语言带机器人翻译之类的,给了个这... 直接一键安装的,启动两个端口就行了,安装倒是简单编号:ym270 品牌:无语言:PHP 大小:34.5MB 类型: ...

Python基于YOLOv7和CRNN的车牌分割＆识别系统（源码＆教程）