OpenVINO 2021r4.1 - 瞎搞YOLOV5 模型转换，INT8量化及C++推理实现
2024-05-16 11:39:51
今年年初的时候曾经玩了一阵openvino yolov5量化，后来找到了这个github的大神教程完美解决GitHub - Chen-MingChang/pytorch_YOLO_OpenVINO_demoContribute to Chen-MingChang/pytorch_YOLO_OpenVINO_demo development by creating an account on GitHub.
https://github.com/Chen-MingChang/pytorch_YOLO_OpenVINO_demo但是当时有几个问题不是很完美
- 这个大神在openvino 2021.2上手写代码扩充了accuracy checker的yolov5的支持，需要在ov2021.2上打patch, 如果openvino升级的话，因为AC代码的变动, merge的时候大概率会有问题
- 记得当时转换完测了一下，CPU上推理是正常的，但是GPU推理有问题。在那个bus的推理图片上，莫名的多出了几个识别的红框框，明显是错了。感觉clDNN在计算上可能有数据溢出的现象，但是时间有点久了，忘了是在FP32还是在INT8时候出错了。
最近OpenVINO 2021r4.1发布了，再回过头来看看2021r4.1版本的yolov5支持
1. 搭建环境
安装windows版的openvino 2021.4.1 LTS, 按照官网的教程安装ac和pot的依赖项。
2. 下载和准备yolov5的模型
这里完全按照大神 https://github.com/Chen-MingChang/pytorch_YOLO_OpenVINO_demo里面的步骤走。
- 下载github上分享的yolov5l v4版本的pytorch模型
- 运行export.py将pytorch模型转成onnx模型
$ python3 models/export.py --weights yolov5l.pt --img-size 640- 然后用netron观察转换出的onnx网络节点图，记录最后3个输出点前的最后一个Conv层的名字，等下转IR时候需要指定--output输出节点，这个模型分别是Conv_403， Conv_491， Conv_435
3. 转换yolov5的OpenVINO FP32模型
- 进入openvino环境，将onnx模型转成FP32的IR模型
C:\temp\yolov5_ac_ov2021_4>python "c:\Program Files (x86)\intel\openvino_2021\deployment_tools\model_optimizer\mo.py" --input_model yolov5l_v4.onnx -s 255 --reverse_input_channels --output Conv_403,Conv_419,Conv_435
Model Optimizer arguments:
Common parameters:- Path to the Input Model: C:\temp\yolov5_ac_ov2021_4\yolov5l_v4.onnx- Path for generated IR: C:\temp\yolov5_ac_ov2021_4\.- IR output name: yolov5l_v4- Log level: ERROR- Batch: Not specified, inherited from the model- Input layers: Not specified, inherited from the model- Output layers: Conv_403,Conv_419,Conv_435- Input shapes: Not specified, inherited from the model- Mean values: Not specified- Scale values: Not specified- Scale factor: 255.0- Precision of IR: FP32- Enable fusing: True- Enable grouped convolutions fusing: True- Move mean values to preprocess section: None- Reverse input channels: True
ONNX specific parameters:- Inference Engine found in: c:\Program Files (x86)\Intel\openvino_2021\python\python3.7\openvino
Inference Engine version: 2021.4.1-3926-14e67d86634-releases/2021/4
Model Optimizer version: 2021.4.1-3926-14e67d86634-releases/2021/4
[ SUCCESS ] Generated IR version 10 model.
[ SUCCESS ] XML file: C:\temp\yolov5_ac_ov2021_4\yolov5l_v4.xml
[ SUCCESS ] BIN file: C:\temp\yolov5_ac_ov2021_4\yolov5l_v4.bin
[ SUCCESS ] Total execution time: 14.90 seconds.- 接下来先做一下FP32的精度检测，好等下和INT8模型精度做比对
- 下载和准备coco val2017数据集， Dataset Preparation Guide - OpenVINO™ Toolkit
- 下载的原始数据集有点大，用CPU做完整的AC太费时间，可以参考这里对数据库做裁剪 Cut Datasets 这里只截了256张图片做实验
- accuracy checker
准备yolov5_640_ac.yml配置文件
models:- name: yolo-v5launchers:- framework: dlsdkmodel: yolov5l_v4.xmlweights: yolov5l_v4.binadapter:type: yolo_v3anchors: "10,13, 16,30, 33,23, 30,61, 62,45, 59,119, 116,90, 156,198, 373,326"num: 9coords: 4classes: 80anchor_masks: [[6, 7, 8], [3, 4, 5], [0, 1, 2], ]outputs:- Conv_435- Conv_419- Conv_403datasets:- name: ms_coco_detection_80_class_without_backgrounddata_source: val2017annotation_conversion:converter: mscoco_detectionannotation_file: instances_val2017.jsonhas_background: Falsesort_annotations: Trueuse_full_label_map: Falseannotation: mscoco_det_80.pickledataset_meta: mscoco_det_80.jsonpreprocessing:- type: resizesize: 640postprocessing:- type: resize_prediction_boxes- type: filterapply_to: predictionmin_confidence: 0.001remove_filtered: True- type: nmsoverlap: 0.5- type: clip_boxesapply_to: predictionmetrics:- type: mapintegral: 11pointignore_difficult: truepresenter: print_scalar- type: coco_precisionmax_detections: 100threshold: 0.5
运行ac命令
C:\temp\yolov5_ac_ov2021_4>accuracy_check -c yolov5_640_ac.yml -s ./ -td CPU
09:01:27 accuracy_checker WARNING: c:\Program Files (x86)\Intel\openvino_2021\python\python3.7\ngraph\utils\types.py:25: DeprecationWarning: `np.bool` is a deprecated alias for the builtin `bool`. To silence this warning, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations(NgraphType.boolean, np.bool),Processing info:
model: yolo-v5
launcher: dlsdk
device: CPU
dataset: ms_coco_detection_80_class_without_background
OpenCV version: 4.5.2
Annotation for ms_coco_detection_80_class_without_background dataset will be loaded from mscoco_det_80.pickle
Loaded dataset info:Dataset name: ms_coco_detection_80_class_without_background_1Accuracy Checker version 0.8.7Dataset size 256Conversion parameters:converter: mscoco_detectionannotation_file: PATH/instances_val2017.jsonhas_background: Falsesort_annotations: Trueuse_full_label_map: False
ms_coco_detection_80_class_without_background dataset metadata will be loaded from mscoco_det_80.json
IE version: 2021.4.1-3926-14e67d86634-releases/2021/4
Loaded CPU plugin version:CPU - MKLDNNPlugin: 2.1.2021.4.1-3926-14e67d86634-releases/2021/4
Found model yolov5l_v4.xml
Found weights yolov5l_v4.bin
Input info:Layer name: imagesprecision: FP32shape [1, 3, 640, 640]Output infoLayer name: Conv_403precision: FP32shape: [1, 255, 80, 80]Layer name: Conv_419precision: FP32shape: [1, 255, 40, 40]Layer name: Conv_435precision: FP32shape: [1, 255, 20, 20]09:01:28 accuracy_checker WARNING: c:\users\intel\anaconda3\lib\site-packages\accuracy_checker-0.8.7-py3.7.egg\accuracy_checker\metrics\metric_executor.py:168: DeprecationWarning: threshold option is deprecated. Please use abs_threshold insteadwarnings.warn('threshold option is deprecated. Please use abs_threshold instead', DeprecationWarning)256 objects processed in 133.234 seconds
map: 27.70%
coco_precision: 31.18%
这个精度有问题啊，用GitHub - Chen-MingChang/pytorch_YOLO_OpenVINO_demo的代码复现的精度有60%多呢... 但是看网上很多人已经用openvino+yolov5做推理了，精度应该没有大问题，还是accuracy checker自己有问题的可能性比较大
4. 转换yolov5的OpenVINO INT8模型
- 用pot工具做int8模型量化
准备pot配置json文件, 量化方法选择了最简单的DefaultQuantization, 这种方法在量化时不做精度检测对比，只是把所有能转成int8的fp32操作全部转成int8类型
准备yolov5l_v4_int8_simple_mode.json配置文件
{"model": {"model_name": "yolov5l_v4_int8_cpu","model": "yolov5l_v4_640.xml","weights": "yolov5l_v4_640.bin"},"engine": {"type": "simplified",// you can specify path to directory with images or video file// also you can specify template for file names to filter images to load// templates are unix style"data_source": "val2017"},"compression": {"target_device": "CPU","algorithms": [{"name": "DefaultQuantization","params": {"preset": "performance","stat_subset_size": 128}}]}
}
运行pot命令量化
C:\temp\yolov5_ac_ov2021_4>pot -c yolov5l_v4_int8_simple_mode.json
10:12:41 accuracy_checker WARNING: c:\users\intel\appdata\roaming\python\python37\site-packages\defusedxml\__init__.py:30: DeprecationWarning: defusedxml.cElementTree is deprecated, import from defusedxml.ElementTree instead.from . import cElementTree10:12:41 accuracy_checker WARNING: c:\users\intel\anaconda3\lib\site-packages\pot-1.0-py3.7.egg\compression\algorithms\quantization\optimization\algorithm.py:39: UserWarning: Nevergrad package could not be imported. If you are planning to useany hyperparameter optimization algo, consider installing itusing pip. This implies advanced usage of the tool.Note that nevergrad is compatible only with Python 3.6+'Nevergrad package could not be imported. If you are planning to use'10:12:41 accuracy_checker WARNING: c:\users\intel\anaconda3\lib\site-packages\past\builtins\misc.py:4: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop workingfrom collections import MappingINFO:app.run:Output log dir: ./results\yolov5l_v4_int8_gpu_DefaultQuantization\2021-09-16_10-12-41
INFO:app.run:Creating pipeline:Algorithm: DefaultQuantizationParameters:preset : performancestat_subset_size : 128target_device : CPUmodel_type : Nonedump_intermediate_model : Falseexec_log_dir : ./results\yolov5l_v4_int8_gpu_DefaultQuantization\2021-09-16_10-12-41===========================================================================
INFO:compression.statistics.collector:Start computing statistics for algorithms : DefaultQuantization
INFO:compression.statistics.collector:Computing statistics finished
INFO:compression.pipeline.pipeline:Start algorithm: DefaultQuantization
INFO:compression.algorithms.quantization.default.algorithm:Start computing statistics for algorithm : ActivationChannelAlignment
INFO:compression.algorithms.quantization.default.algorithm:Computing statistics finished
INFO:compression.algorithms.quantization.default.algorithm:Start computing statistics for algorithms : MinMaxQuantization,FastBiasCorrection
INFO:compression.algorithms.quantization.default.algorithm:Computing statistics finished
INFO:compression.pipeline.pipeline:Finished: DefaultQuantization===========================================================================
- 用ac测一下int8模型的精度
C:\temp\yolov5_ac_ov2021_4>accuracy_check -c yolov5_int8_640_ac.yml -s ./ -td CPU
10:17:13 accuracy_checker WARNING: c:\Program Files (x86)\Intel\openvino_2021\python\python3.7\ngraph\utils\types.py:25: DeprecationWarning: `np.bool` is a deprecated alias for the builtin `bool`. To silence this warning, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations(NgraphType.boolean, np.bool),Processing info:
model: yolo-v5-int8
launcher: dlsdk
device: CPU
dataset: ms_coco_detection_80_class_without_background
OpenCV version: 4.5.2
Annotation for ms_coco_detection_80_class_without_background dataset will be loaded from mscoco_det_80.pickle
Loaded dataset info:Dataset name: ms_coco_detection_80_class_without_background_1Accuracy Checker version 0.8.7Dataset size 256Conversion parameters:converter: mscoco_detectionannotation_file: PATH/instances_val2017.jsonhas_background: Falsesort_annotations: Trueuse_full_label_map: False
ms_coco_detection_80_class_without_background dataset metadata will be loaded from mscoco_det_80.json
IE version: 2021.4.1-3926-14e67d86634-releases/2021/4
Loaded CPU plugin version:CPU - MKLDNNPlugin: 2.1.2021.4.1-3926-14e67d86634-releases/2021/4
Found model yolov5l_v4_int8_cpu.xml
Found weights yolov5l_v4_int8_cpu.bin
Input info:Layer name: imagesprecision: FP32shape [1, 3, 640, 640]Output infoLayer name: Conv_403precision: FP32shape: [1, 255, 80, 80]Layer name: Conv_419precision: FP32shape: [1, 255, 40, 40]Layer name: Conv_435precision: FP32shape: [1, 255, 20, 20]10:17:13 accuracy_checker WARNING: c:\users\intel\anaconda3\lib\site-packages\accuracy_checker-0.8.7-py3.7.egg\accuracy_checker\metrics\metric_executor.py:168: DeprecationWarning: threshold option is deprecated. Please use abs_threshold insteadwarnings.warn('threshold option is deprecated. Please use abs_threshold instead', DeprecationWarning)256 objects processed in 106.666 seconds
map: 28.67%
coco_precision: 33.07%
虽说精度不对，但是看上去int8模型的精度竟然比fp32的精度还好一点，这个有点意思
5. yolov5的OpenVINO C++推理实现
这部分相对就比较简单了，主要参考了大神的项目
c++实现yolov5的OpenVINO部署
代码的工作就是把OpenVINO推理输出的3个输出层的结果（窗口的座标，分类label和信心度）做一些处理，先筛掉信心度比较低的检测窗，再把同一物体上不同大小的检测窗再筛一遍，保留信息度最大的框
具体代码就不贴了，整个项目奉上 yolov5-ov2021: 基于c++/openvino 2021r4的yolov5推理实现
运行FP32模型，GPU推理
运行INT8模型，GPU推理
6. 模型benchmarking
用OpenVINO2021r4自带的benchmarking工具测试
CPU推理性能测试
9900K 8核16线程
FP32模型CPU推理 5.59FPS
C:\Users\test\Documents\Intel\OpenVINO\inference_engine_cpp_samples_build\intel64\Release>benchmark_app.exe -m yolov5l_v4_640.xml -nireq 1 -nstreams 1 -b 1 -d CPU
[Step 1/11] Parsing and validating input arguments
[ INFO ] Parsing input parameters
[Step 2/11] Loading Inference Engine
[ INFO ] InferenceEngine:IE version ......... 2021.4.1Build ........... 2021.4.1-3926-14e67d86634-releases/2021/4
[ INFO ] Device info:CPUMKLDNNPlugin version ......... 2021.4.1Build ........... 2021.4.1-3926-14e67d86634-releases/2021/4[Step 3/11] Setting device configuration
[Step 4/11] Reading network files
[ INFO ] Loading network files
[ INFO ] Read network took 151.36 ms
[Step 5/11] Resizing network to match image sizes and given batch
[ INFO ] Network batch size was changed to: 1
[Step 6/11] Configuring input of the model
Network inputs:images : U8 / NCHW
Network outputs:Conv_403 : FP32 / NCHWConv_419 : FP32 / NCHWConv_435 : FP32 / NCHW
[Step 7/11] Loading the model to the device
[ INFO ] Load network took 195.02 ms
[Step 8/11] Setting optimal runtime parameters
[Step 9/11] Creating infer requests and filling input blobs with images
[ INFO ] Network input 'images' precision U8, dimensions (NCHW): 1 3 640 640
[ WARNING ] No input files were given: all inputs will be filled with random values!
[ INFO ] Infer Request 0 filling
[ INFO ] Fill input 'images' with random values (image is expected)
[Step 10/11] Measuring performance (Start inference asynchronously, 1 inference requests using 1 streams for CPU, limits: 60000 ms duration)
[ INFO ] First inference took 182.24 ms[Step 11/11] Dumping statistics report
Count: 337 iterations
Duration: 60327.10 ms
Latency: 177.38 ms
Throughput: 5.59 FPSINT8模型CPU推理 13.37FPS,
C:\Users\test\Documents\Intel\OpenVINO\inference_engine_cpp_samples_build\intel64\Release>benchmark_app.exe -m yolov5l_v4_640_int8_cpu.xml -nireq 1 -nstreams 1 -b 1 -d CPU
[Step 1/11] Parsing and validating input arguments
[ INFO ] Parsing input parameters
[Step 2/11] Loading Inference Engine
[ INFO ] InferenceEngine:IE version ......... 2021.4.1Build ........... 2021.4.1-3926-14e67d86634-releases/2021/4
[ INFO ] Device info:CPUMKLDNNPlugin version ......... 2021.4.1Build ........... 2021.4.1-3926-14e67d86634-releases/2021/4[Step 3/11] Setting device configuration
[Step 4/11] Reading network files
[ INFO ] Loading network files
[ INFO ] Read network took 82.20 ms
[Step 5/11] Resizing network to match image sizes and given batch
[ INFO ] Network batch size was changed to: 1
[Step 6/11] Configuring input of the model
Network inputs:images : U8 / NCHW
Network outputs:Conv_403 : FP32 / NCHWConv_419 : FP32 / NCHWConv_435 : FP32 / NCHW
[Step 7/11] Loading the model to the device
[ INFO ] Load network took 409.73 ms
[Step 8/11] Setting optimal runtime parameters
[Step 9/11] Creating infer requests and filling input blobs with images
[ INFO ] Network input 'images' precision U8, dimensions (NCHW): 1 3 640 640
[ WARNING ] No input files were given: all inputs will be filled with random values!
[ INFO ] Infer Request 0 filling
[ INFO ] Fill input 'images' with random values (image is expected)
[Step 10/11] Measuring performance (Start inference asynchronously, 1 inference requests using 1 streams for CPU, limits: 60000 ms duration)
[ INFO ] First inference took 82.14 ms[Step 11/11] Dumping statistics report
Count: 804 iterations
Duration: 60135.29 ms
Latency: 74.23 ms
Throughput: 13.37 FPSGPU推理性能测试
TGL 集显Gen12 96EU
FP32模型GPU推理 9.86FPS
C:\Users\test\Documents\Intel\OpenVINO\inference_engine_cpp_samples_build\intel64\Release>benchmark_app.exe -m yolov5l_v4_640.xml -nireq 1 -nstreams 1 -b 1 -d GPU
[Step 1/11] Parsing and validating input arguments
[ INFO ] Parsing input parameters
[Step 2/11] Loading Inference Engine
[ INFO ] InferenceEngine:IE version ......... 2021.4.1Build ........... 2021.4.1-3926-14e67d86634-releases/2021/4
[ INFO ] Device info:GPUclDNNPlugin version ......... 2021.4.1Build ........... 2021.4.1-3926-14e67d86634-releases/2021/4[Step 3/11] Setting device configuration
[Step 4/11] Reading network files
[ INFO ] Loading network files
[ INFO ] Read network took 137.94 ms
[Step 5/11] Resizing network to match image sizes and given batch
[ INFO ] Network batch size was changed to: 1
[Step 6/11] Configuring input of the model
Network inputs:images : U8 / NCHW
Network outputs:Conv_403 : FP32 / NCHWConv_419 : FP32 / NCHWConv_435 : FP32 / NCHW
[Step 7/11] Loading the model to the device
[ INFO ] Load network took 57321.88 ms
[Step 8/11] Setting optimal runtime parameters
[Step 9/11] Creating infer requests and filling input blobs with images
[ INFO ] Network input 'images' precision U8, dimensions (NCHW): 1 3 640 640
[ WARNING ] No input files were given: all inputs will be filled with random values!
[ INFO ] Infer Request 0 filling
[ INFO ] Fill input 'images' with random values (image is expected)
[Step 10/11] Measuring performance (Start inference asynchronously, 1 inference requests using 1 streams for GPU, limits: 60000 ms duration)
[ INFO ] First inference took 89.20 ms[Step 11/11] Dumping statistics report
Count: 593 iterations
Duration: 60131.64 ms
Latency: 113.72 ms
Throughput: 9.86 FPS
INT8模型GPU推理 30.91FPS
C:\Users\test\Documents\Intel\OpenVINO\inference_engine_cpp_samples_build\intel64\Release>benchmark_app.exe -m yolov5l_v4_640_int8_cpu.xml -nireq 1 -nstreams 1 -b 1 -d GPU
[Step 1/11] Parsing and validating input arguments
[ INFO ] Parsing input parameters
[Step 2/11] Loading Inference Engine
[ INFO ] InferenceEngine:IE version ......... 2021.4.1Build ........... 2021.4.1-3926-14e67d86634-releases/2021/4
[ INFO ] Device info:GPUclDNNPlugin version ......... 2021.4.1Build ........... 2021.4.1-3926-14e67d86634-releases/2021/4[Step 3/11] Setting device configuration
[Step 4/11] Reading network files
[ INFO ] Loading network files
[ INFO ] Read network took 65.74 ms
[Step 5/11] Resizing network to match image sizes and given batch
[ INFO ] Network batch size was changed to: 1
[Step 6/11] Configuring input of the model
Network inputs:images : U8 / NCHW
Network outputs:Conv_403 : FP32 / NCHWConv_419 : FP32 / NCHWConv_435 : FP32 / NCHW
[Step 7/11] Loading the model to the device
[ INFO ] Load network took 61720.85 ms
[Step 8/11] Setting optimal runtime parameters
[Step 9/11] Creating infer requests and filling input blobs with images
[ INFO ] Network input 'images' precision U8, dimensions (NCHW): 1 3 640 640
[ WARNING ] No input files were given: all inputs will be filled with random values!
[ INFO ] Infer Request 0 filling
[ INFO ] Fill input 'images' with random values (image is expected)
[Step 10/11] Measuring performance (Start inference asynchronously, 1 inference requests using 1 streams for GPU, limits: 60000 ms duration)
[ INFO ] First inference took 34.45 ms[Step 11/11] Dumping statistics report
Count: 1856 iterations
Duration: 60050.95 ms
Latency: 31.02 ms
Throughput: 30.91 FPS
CPU INT8/FP32 = 13.37/5.59 = 2.4X
GPU INT8/FP32 = 30.91/9.86 = 3.13X
GPU/CPU FP32 = 9.86/5.59 = 1.76X
GPU/CPU INT8 = 30.91/13.37 = 2.31X
性能提升非常不错，在Intel集显上Yolov5l的V4版本终于到30FPS了 :)
收工!!! :)
最后照例分享一下踩过的小坑
- 今年在openvino 2021的4个版本(2021.1/2/3/4 )上尝试了yolov5的精度检测，发现好像accuracy checker对python的不同版本有依赖，我用的不同anaconda里带的python, 3.6 3.7 3.8安装测试，发现有些python版本装ac/pot的依赖项时候会出错，不知道是不是我这边环境的问题还是openvino问题(我这机器好多年没重装python了, 各种包反复升级卸载)
- 从 https://github.com/openvinotoolkit/open_model_zoo/pull/2051 看accuracy checker已经支持了yolov5, 但是我这边从来没成功过，不知道是哪里有问题，还要求各位大佬解惑
- mo转换的时候必须要用--output 指定那几个输出的conv层的名字, 如果不指定output的话单纯做推理是没问题的，但是做ac的时候会报一个输入数据无法reshape成输出数组大小的错误。
OpenVINO 2021r4.1 - 瞎搞YOLOV5 模型转换，INT8量化及C++推理实现相关推荐
- yolov5模型转换rknn
yolov5模型转换rknn 安装virtualenv,用来来管理python环境 导出包到另一个系统 图片标注 安装yolo onnx模型转换为rknn RK3588安装RKNN-toolkit 安 ...
- yolov5模型转换（pt=＞onnx=＞rknn）和板端验证测试
测试环境说明: (1)由于模型转换工具需要onnx版本和rknn的tool工具需要的版本相互矛盾需要创建量开发环境,当前测试转换的模型是yolov5_v5.0的模型 (2)由于在搭建开发环境时还存在部 ...
- Openvino 模型转换命令mo.py
openvino中用于将训练的模型转换成IR文件,可以使用mo.py脚本 该脚本位于: /opt/intel/openvino/deployment_tools/model_optimizer/mo. ...
- 【YoloV5 6.0|6.1 部署 TensorRT到torchserve】环境搭建|模型转换|engine模型部署（详细的packet文件编写方法）
忽然发现,关于部署TensorRT的文章少的可怜,于是乎,决定分享一下我自己关于这部分内容的一些成功实操和心得.还是希望大家可以分享出去,让更多人看到!!! QQ: 1757093754 我的操作环境 ...
- 一文搞懂模型量化算法
1,模型量化概述 1.1,模型量化优点 1.2,模型量化的方案 1.2.1,PTQ 理解 1.3,量化的分类 1.3.1,线性量化概述 2,量化算术 2.1,定点和浮点 2.2,量化浮点 2.2,量化 ...
- 较为详细的记录总结TensorRT的python接口的使用，环境配置，模型转换和静态动态模型推理
先来一段摘抄自网上的TensorRT介绍: TensorRT是英伟达针对自家平台做的加速包,TensorRT主要做了这么两件事情,来提升模型的运行速度. TensorRT支持INT8和FP16的计算. ...
- 地平线Horizon模型转换和PC端仿真测试
特别说明:参考地平线的官方文档,如有侵权告知删,谢谢. 完整转换代码参见github.代码 1 工具链 地平线工具链支持使用Docker和手工安装两种方式,"地平线强烈建议使用 Dock ...
- Atlas 200 DK 系列 --高级篇--模型转换
一.模型转换工具的介绍与操作演示–模型转换OMG介绍 Atlas200DK平台中提供了模型转换工具(OMG).可以将caffe.Tensorflow等开源框架模型转换成Atlas200DK支持的模型, ...
- 使用OpenVINOTM对YOLOv5进行INT8量化
终于来了一个yolo5的 本身一直停留在v5阶段 这个也是拿来主义 自学 真的为自学 多谢理解啦 如题 .. 使用OpenVINOTM 2022.1 Post-training Optimizati ...
最新文章
- Axure 8 window 下载,破解
- 华为擎云w510_鸿蒙系统之后，华为突然官宣新生态，正式对windows说不
- grub 引导 多linux系统,GRUB 多系统引导
- throws与throw
- Frog Jumps CodeForces - 1324C（二分）
- 重学java基础第十四课:java特性和优势
- Django：ORM基本操作-CRUD，管理器对象objects，-----＞查询2（filter，exclude，get，查询谓词）
- C++ 不定参数（转）
- 数论知识(2)-------------欧拉函数
- html页面打开字都有蓝色背景,为什么有些网页打不开？网页空白页、白底蓝字问题怎么解决？...
- 爬取B站程序猿up主，分析程序猿up出什么类型的视频会受欢迎
- 解决win7 sp1专业版无法安装更新
- podman配置加速器、harbor镜像仓库部署
- DAPP 开发直通车-如何基于NEL 轻钱包来开发DAPP
- cocos2dx掼蛋_cocos2d/x 自带字体（label）
- Java IDE漫谈(一)
- 骨灰级的魔兽伤害计算（包括物理和…
- 考研初试备考，感谢曾经努力的自己
- 关于OpenCV的个人小心得
- labelmelabelImg