In my previous articles, I have discussed the basics of the OpenVINO toolkit, OpenVINO’s Model Optimizer and Inference Engine. In this article, we will be exploring:-

在之前的文章中，我讨论了OpenVINO工具箱 ，OpenVINO的模型优化器和推理引擎的基础 。 在本文中，我们将探索：

• Types of Computer Vision models.计算机视觉模型的类型。
• Pre-trained models in OpenVINO.OpenVINO中的预训练模型。
• Downloading Pre-trained models.下载预训练的模型。
• Deploying an Edge App using a pre-trained model.使用预先训练的模型部署Edge App。

计算机视觉模型的类型 (Types of Computer Vision Models)

There are different types of computer vision models which are used for various purposes. But the three main computer vision models are:-

有多种类型的计算机视觉模型可用于各种目的。 但是，三种主要的计算机视觉模型是：

• Classification分类
• Object Detection物体检测
• Segmentation分割

The classification model identifies the “class” of a given image or an object in the image. The classification can be binary i.e. Yes or No, or thousands of classes like a person, apple, car, cat, etc.. There are several classification models like- ResNet, DenseNet, Inception, etc..

分类模型识别给定图像或图像中对象的“类别”。 分类可以是二进制的，即“是”或“否”，也可以是数千种类别，例如人，苹果，汽车，猫等。有几种分类模型，例如ResNet，DenseNet，Inception等。

Object Detection models are used to determine the objects present in the image and oftentimes draw bounding boxes around the detected objects. They also use classification to identify the class of the object inside the bounding box. You can also set a threshold for bounding boxes so that you can reject low-threshold detections. RCNN, Fast-RCNN, YOLO, etc. are some examples of Object Detection Models.

对象检测模型用于确定图像中存在的对象，并经常在检测到的对象周围绘制边界框。 他们还使用分类来识别边界框内对象的类。 您还可以为边界框设置阈值，以便拒绝低阈值检测。 RCNN，Fast-RCNN，YOLO等是对象检测模型的一些示例。

Segmentation models perform pixel-wise classification in the given image. There are two different types of Segmentation- Semantic Segmentation and Instance Segmentation. In Semantic Segmentation, all the objects which belong to the same class are considered the same, whereas in Instance Segmentation each and every object is considered different even if it belongs to the same class. For example, if there are five people in an image, a Semantic Segmentation model will treat all five of them as same, whereas in Instance Segmentation model all five of them will be treated differently. U-Net, DRN, etc..

分割模型在给定图像中执行逐像素分类。 有两种不同类型的细分-语义细分和实例细分。 在语义分割中，属于同一类的所有对象都被认为是相同的，而在实例分割中，即使每个对象属于同一类，也被认为是不同的。 例如，如果图像中有五个人，则语义分割模型将把所有五个人视为相同，而在实例分割模型中，将把所有五个人区别对待。 U-Net，DRN等。

OpenVINO中的预训练模型 (Pre-trained Models in OpenVINO)

Pre-trained models, as the name suggests, are models which are already trained with high, or even cutting edge accuracy. Training a deep learning model requires a lot of time and computation power. Although, it is exciting to create your own model and train it by fine-tuning the hyperparameters (number of hidden layers, learning rate, activation function, etc.) to achieve higher accuracy. But, this needs hours of work.

顾名思义，预训练模型是已经以高甚至尖端精度进行训练的模型。 训练深度学习模型需要大量时间和计算能力。 虽然，创建自己的模型并通过微调超参数(隐藏层数，学习率，激活函数等)以达到更高的精度来训练它是令人兴奋的。 但是，这需要数小时的工作。

By using pre-trained models, we avoid the need for large-scale data collection and long, costly training. Given knowledge of how to preprocess the inputs and handle the outputs of the network, you can plug these directly into your own app.

通过使用预先训练的模型，我们避免了大规模数据收集和长期，昂贵的训练的需要。 在掌握了如何预处理输入和处理网络输出的知识之后，您可以将其直接插入自己的应用程序中。

OpenVINO has a lot of pre-trained models in the model zoo. The model zoo has Free Model Set and Public Model Set, the Free Model Set contains pre-trained models already converted to Intermediate Representation(.xml and .bin) using the Model Optimizer. These models can be used directly with the Inference Engine. The Public Model Set contains pre-trained models, but these are not converted to the intermediate representation.

OpenVINO在模型动物园中有很多预先训练的模型 。 模型动物园具有“免费模型集”和“公共模型集”，“免费模型集”包含已使用“模型优化器”转换为中间表示(.xml和.bin)的预训练模型。 这些模型可以直接与推理引擎一起使用。 公共模型集包含预训练的模型，但是不会将其转换为中间表示。

下载预训练的模型 (Downloading Pre-trained Models)

In this article, I will be loading the “vehicle-attributes-recognition-barrier-0039” model from the open model zoo.

在本文中，我将从开放模型动物园加载“ vehicle-attributes-recognition-barrier-0039”模型。

To download a pre-trained model, follow these steps(type the commands in Command Prompt/Terminal):-

要下载预训练的模型，请按照以下步骤操作(在命令提示符/终端中键入命令)：

1. Navigate to the Model Downloader directory导航到“模型下载器”目录

For Linux:-

对于Linux：-

cd /opt/intel/openvino/deployment_tools/open_model_zoo/tools/model_downloader

For Windows:-

对于Windows：-

cd C:/Program Files (x86)/IntelSWTools/openvino

I have used the default installation directory in the above command if your installation directory is different then navigate to the appropriate directory.

如果您的安装目录不同，那么我在上面的命令中使用了默认的安装目录，然后导航到适当的目录。

2. Run the downloader.py

2.运行downloader.py

The downloader Python file requires some arguments, you can use the “-h” argument to see available arguments.

下载器的Python文件需要一些参数，您可以使用“ -h”参数查看可用参数。

python downloader.py -h

Let’s download the model,

让我们下载模型，

python downloader.py --name vehicle-attributes-recognition-barrier-0039 --precisions -FP32 --output_dir /home/pretrained_models

• — name → model name.

  —名称 →型号名称。

• — precision → model precision (FP16, FP32 or INT8).

  —精度 →模型精度(FP16，FP32或INT8)。

• — output_dir → path where to save models.

  — output_dir →保存模型的路径。

After successfully downloading the model, navigate to the path where you have downloaded the model and you will find the “.xml” and “.bin” files of the model.

成功下载模型后，导航至下载模型的路径，您将找到模型的“ .xml”和“ .bin”文件。

Kindly refer the documentation to know more details(inputs and outputs) about the model.

请参考文档以了解有关该模型的更多详细信息(输入和输出)。

部署Edge应用 (Deploying an Edge App)

Now, since we have downloaded the pre-trained model, Let’s deploy it in an Edge app.

现在，由于我们已经下载了预训练的模型，因此让我们将其部署在Edge应用程序中。

Let’s create a file “inference.py” to define and work with the inference engine. In my previous article, about the inference engine, I have used different functions, but here I will be defining a class.

让我们创建一个文件“ inference.py”来定义和使用推理引擎。 在上一篇关于推理引擎的文章中 ，我使用了不同的函数，但是在这里我将定义一个类。

from openvino.inference_engine import IENetwork, IECoreclass Network:    def __init__(self):        self.plugin = None        self.network = None        self.input_blob = None        self.exec_network = None        self.infer_request = None    def load_model(self):        self.plugin = IECore()        self.network = IENetwork(model='path_to_xml', weights='path_to_bin')

        ### Defining CPU Extension path        CPU_EXT_PATH=      "/opt/intel/openvino/deployment_tools/inference_engine/lib/intel64/ libcpu_extension_sse4.so"           ### Adding CPU Extension        plugin.add_extension(CPU_EXT_PATH,"CPU")        ### Get the supported layers of the network        supported_layers = plugin.query_network(network=network, device_name="CPU")            ### Finding unsupported layers        unsupported_layers = [l for l in network.layers.keys() if l not in supported_layers]            ### Checking for unsupported layers        if len(unsupported_layers) != 0:            print("Unsupported layers found")            print(unsupported_layers)            exit(1)        ### Loading the network        self.exec_network =             self.plugin.load_network(self.network,"CPU")        self.input_blob  = next(iter(self.network.inputs))        print("MODEL LOADED SUCCESSFULLY!!!)    def get_input_shape(self):        return self.network.inputs[self.input_blob].shape    def synchronous_inference(self,image):          self.exec_network.infer({self.input_blob: image})    def extract_output(self):        return self.exec_network.requests[0].outputs

Don’t get confused! I’ll explain every function.

不要困惑！ 我将解释每个功能。

• __init__(self):

  __在自身)：

It’s the constructor of the class Network, where I initialize the data members of the class.

它是网络类的构造函数，在这里我初始化类的数据成员。

• load_model(self):

  load_model(self)：

As the name suggests, it is used to load the model(pre-trained), in this function we:-

顾名思义，它用于加载模型(预先训练)，在此函数中，我们：-

▹ Declared an IECore object.

ed声明为IECore对象。

▹ Declare an IENetwork object.

▹声明IENetwork对象。

▹ Loaded the model xml and bin files.

▹加载了模型xml和bin文件。

▹ Checked for unsupported layers

▹检查了不支持的图层

▹ Load the IENetwork object in IECore Object.

▹将IENetwork对象加载到IECore对象中。

• get_input_shape(self):

  get_input_shape(self)：

Returns the shape of the input required by the model

返回模型所需输入的形状

• synchronous_inference(self, image):

  sync_inference(自己，图片)：

Performs Synchronous Inference on the input image

在输入图像上执行同步推理

• extract_output(self):

  extract_output()：

Returns the output from the model after the inference is completed.

推断完成后，返回模型的输出。

So, that was “inference.py”, now let’s create a file “main.py”.

因此，那是“ inference.py”，现在让我们创建一个文件“ main.py”。

import cv2import numpy as npfrom inference import Networkdef preprocessing(image,height,width):    ### Resize the Image    image = cv2.resize(image,(width,height))    ### Add color channel first    image = image.transpose((2,0,1))    ### Add Batch Size    image  = np.reshape((image,(1,3,height,width))    return image

1. While resizing the image using the resize() of OpenCV, you should give the width first and then the height.使用OpenCVresize()调整图像大小时，应首先提供宽度，然后再提供高度。

2. According to the documentation, the model reads the channels first and then the image dimensions, but OpenCV reads the image dimensions first and then the channels, so I’ve used the transpose(), to bring the color channel first.

   根据文档 ，模型首先读取通道，然后读取图像尺寸，但是OpenCV首先读取图像尺寸，然后读取通道，因此我使用了transpose()首先打开了彩色通道。

3. The model takes the input as (batch_size, color_channels, height, width), so we reshape the image to give a “batch_size” which is 1.该模型将输入作为(batch_size，color_channels，height，width)，因此我们将图像重塑为“ batch_size”，即1。

def main():    ### Read the image    image = cv2.imread('path_to_image')    ### Declare a Network Object    plugin = Network()    ### Input shape required by model    input_shape = plugin.get_input_shape()    height = input_shape[2]    width = input_shape[3]    ### Preprocess the input        p_image = preprocessing(image,height,width)    ### Perform Synchronous Inference    plugin.synchronous_inference(p_image)    ### Extract the output    results = plugin.extract_output()

According to the documentation, the output(results) from the model is a dictionary, which contains the following information:-

根据文档，模型的输出(结果)是一个字典，其中包含以下信息：

1. “color”, shape: [1, 7, 1, 1] — Softmax output across seven color classes [white, grey, yellow, red, green, blue, black]“颜色”，形状：[1、7、1、1]-跨七个颜色类别[白色，灰色，黄色，红色，绿色，蓝色，黑色]的Softmax输出
2. “type”, shape: [1, 4, 1, 1] — Softmax output across four type classes [car, bus, truck, van]“类型”，形状：[1、4、1、1] –跨四个类型类别[汽车，公共汽车，卡车，货车]的Softmax输出

Since it is a softmax output, we need to map the index of the maximum value with the color and the type.

由于它是softmax输出，因此我们需要将最大值的索引与颜色和类型进行映射。

    color = ['white','grey','yellow','red','green','blue','black']    vehicle = ['car','bus','truck','van']    ### Finding out the color and type    result_color = str(color[np.argmax(results['color'])])    result_type = str(vehicle[np.argmax(results['type'])])### Add details to image    font = cv2.FONT_HERSHEY_SIMPLEX    font_scale = 1    col = (0, 255,0) #BGR    thickness = 2        color_text= 'color: '+result_color    type_text = 'vehicle: '+result_type    cv2.putText(image,color_text,(50,50), font, font_scale, col, thickness, cv2.LINE_AA)    cv2.putText(image,type_text,(50,75), font, font_scale, col, thickness, cv2.LINE_AA)    ### Save the image    cv2.imwrite('path/vehicle.png',image)if __name__=="__main__":    main()

I tried for two vehicles and I got the following output:-

我尝试了两辆车，结果如下：

Well, That’s all folks. I hope by now you have a proper understanding of how to deploy an AI edge application using OpenVINO. OpenVINO has various pre-trained models for several applications. Try implementing different pre-trained models available in the OpenVINO model zoo and create your own edge application. Thank you so much for reading my article.

好吧，这就是所有人。 我希望到目前为止，您对如何使用OpenVINO部署AI边缘应用程序有正确的了解。 OpenVINO具有针对各种应用的各种预训练模型。 尝试实施OpenVINO 模型库中可用的不同的预训练模型，然后创建自己的边缘应用程序。 非常感谢您阅读我的文章。