华为云GPU服务器使用PaddleServing方式部署PaddleClas多个自己训练的识别模型服务

前言

最近公司需要对图片中的不同的货车品牌和车系进行识别，通过PaddleClas进行模型训练后得到一个品牌识别模型和一个车系识别模型，现在对两个模型部署到一台华为云的GPU服务器上，要对多个模型同时进行部署，只能采取PaddleServing中的 Pipeline 服务或者C++ serving服务进行部署，由于C++ serving需要编译源码，比较麻烦，所以下面采用Pipeline 方式对多个模型进行串联部署。

如何在华为云服务器上搭建GPU版本的PaddlePaddle环境请参考以下文章： https://blog.csdn.net/loutengyuan/article/details/126527326

环境准备

需要准备PaddleClas的运行环境和Paddle Serving的运行环境。

准备PaddleClas的运行环境链接

# 克隆代码
git clone https://github.com/PaddlePaddle/PaddleClas

安装PaddleServing的运行环境，步骤如下

# 安装serving，用于启动服务
wget https://paddle-serving.bj.bcebos.com/test-dev/whl/paddle_serving_server_gpu-0.8.3.post102-py3-none-any.whl
pip3 install paddle_serving_server_gpu-0.8.3.post102-py3-none-any.whl# 安装client，用于向服务发送请求
wget https://paddle-serving.bj.bcebos.com/test-dev/whl/paddle_serving_client-0.8.3-cp38-none-any.whl
pip3 install paddle_serving_client-0.8.3-cp38-none-any.whl# 安装serving-app
wget https://paddle-serving.bj.bcebos.com/test-dev/whl/paddle_serving_app-0.8.3-py3-none-any.whl
pip3 install paddle_serving_app-0.8.3-py3-none-any.whl

服务数据准备

使用 PaddleServing 做图像识别服务化部署时，需要将保存的多个 inference 模型都转换为 Serving 模型。

模型转换

进入工作目录：

cd PaddleClas/deploy/

创建并进入models文件夹：

# 创建并进入models文件夹
mkdir models
cd models

将训练好的inference 模型放到该文件夹下，包括检测模型（picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer）、品牌识别模型（rec_brands_v1.0_infer）和车系识别模型（rec_series_v1.0_infer）结构如下：

├── picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer
│   ├── infer_cfg.yml
│   ├── inference.pdiparams
│   ├── inference.pdiparams.info
│   └── inference.pdmodel
├── rec_brands_v1.0_infer
│   ├── inference.pdiparams
│   ├── inference.pdiparams.info
│   └── inference.pdmodel
└── rec_series_v1.0_infer├── inference.pdiparams├── inference.pdiparams.info├── inference.pdmodel└── readme.txt

转换通用检测 inference 模型为 Serving 模型：

# 转换通用检测模型
python3.8 -m paddle_serving_client.convert --dirname ./picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer/ \
--model_filename inference.pdmodel  \
--params_filename inference.pdiparams \
--serving_server ./picodet_PPLCNet_x2_5_mainbody_lite_v1.0_serving/ \
--serving_client ./picodet_PPLCNet_x2_5_mainbody_lite_v1.0_client/

通用检测 inference 模型转换完成后，会在当前文件夹多出 picodet_PPLCNet_x2_5_mainbody_lite_v1.0_serving/和 picodet_PPLCNet_x2_5_mainbody_lite_v1.0_client/ 的文件夹，具备如下结构：

    ├── picodet_PPLCNet_x2_5_mainbody_lite_v1.0_serving/│   ├── inference.pdiparams│   ├── inference.pdmodel│   ├── serving_server_conf.prototxt│   └── serving_server_conf.stream.prototxt│└── picodet_PPLCNet_x2_5_mainbody_lite_v1.0_client/├── serving_client_conf.prototxt└── serving_client_conf.stream.prototxt

转换品牌识别 inference 模型为 Serving 模型：

  # 转换品牌识别模型python3.8 -m paddle_serving_client.convert \--dirname ./rec_brands_v1.0_infer/ \--model_filename inference.pdmodel  \--params_filename inference.pdiparams \--serving_server ./rec_brands_v1.0_serving/ \--serving_client ./rec_brands_v1.0_client/

品牌识别 inference 模型转换完成后，会在当前文件夹多出 rec_brands_v1.0_serving/ 和 rec_brands_v1.0_client/ 的文件夹，具备如下结构：

    ├── rec_brands_v1.0_serving/│   ├── inference.pdiparams│   ├── inference.pdmodel│   ├── serving_server_conf.prototxt│   └── serving_server_conf.stream.prototxt│└── rec_brands_v1.0_client/├── serving_client_conf.prototxt└── serving_client_conf.stream.prototxt

分别修改 rec_brands_v1.0_serving/ 和 rec_brands_v1.0_client/ 目录下的 serving_server_conf.prototxt 中的 alias 名字：将 fetch_var 中的 alias_name 改为 features。修改后的 serving_server_conf.prototxt 内容如下：

feed_var {name: "x"alias_name: "x"is_lod_tensor: falsefeed_type: 1shape: 3shape: 224shape: 224
}
fetch_var {name: "save_infer_model/scale_0.tmp_1"alias_name: "features"is_lod_tensor: falsefetch_type: 1shape: 512
}

转换车系识别 inference 模型为 Serving 模型：

  # 转换车系识别模型python3.8 -m paddle_serving_client.convert \--dirname ./rec_series_v1.0_infer/ \--model_filename inference.pdmodel  \--params_filename inference.pdiparams \--serving_server ./rec_series_v1.0_serving/ \--serving_client ./rec_series_v1.0_client/

车系识别 inference 模型转换完成后，会在当前文件夹多出 rec_series_v1.0_serving/ 和 rec_series_v1.0_client/ 的文件夹，具备如下结构：

    ├── rec_series_v1.0_serving/│   ├── inference.pdiparams│   ├── inference.pdmodel│   ├── serving_server_conf.prototxt│   └── serving_server_conf.stream.prototxt│└── rec_series_v1.0_client/├── serving_client_conf.prototxt└── serving_client_conf.stream.prototxt

分别修改 rec_series_v1.0_serving/ 和 rec_series_v1.0_client/ 目录下的 serving_server_conf.prototxt 中的 alias 名字：将 fetch_var 中的 alias_name 改为 features。修改后的 serving_server_conf.prototxt 内容如下：

feed_var {name: "x"alias_name: "x"is_lod_tensor: falsefeed_type: 1shape: 3shape: 224shape: 224
}
fetch_var {name: "save_infer_model/scale_0.tmp_1"alias_name: "features"is_lod_tensor: falsefetch_type: 1shape: 512
}

上述命令中参数具体含义如下表所示：

参数	类型	默认值	描述
`dirname`	str	-	需要转换的模型文件存储路径，Program结构文件和参数文件均保存在此目录。
`model_filename`	str	None	存储需要转换的模型Inference Program结构的文件名称。如果设置为None，则使用 `__model__` 作为默认的文件名
`params_filename`	str	None	存储需要转换的模型所有参数的文件名称。当且仅当所有模型参数被保>存在一个单独的二进制文件中，它才需要被指定。如果模型参数是存储在各自分离的文件中，设置它的值为None
`serving_server`	str	`"serving_server"`	转换后的模型文件和配置文件的存储路径。默认值为serving_server
`serving_client`	str	`"serving_client"`	转换后的客户端配置文件存储路径。默认值为serving_client

索引库添加

将品牌和车系库放到上一级（deploy）目录

    # 回到deploy目录cd ../

目录结构如下：

    ├── brand_dataset_v1.0/│   └── index│       ├── id_map.pkl│       └── vector.index└── series_dataset_v1.0/└── index├── id_map.pkl└── vector.index

服务部署

注意： 识别服务涉及到多个模型，出于性能考虑采用 PipeLine 部署方式。Pipeline 部署方式当前不支持 windows 平台。
进入到工作目录

  cd ./deploy/paddleserving/recognition

paddleserving 目录包含启动 Python Pipeline 服务、C++ Serving 服务和发送预测请求的代码，包括：

  __init__.pyconfig.yml                  # 启动python pipeline服务的配置文件pipeline_http_client.py     # http方式发送pipeline预测请求的脚本pipeline_rpc_client.py      # rpc方式发送pipeline预测请求的脚本recognition_web_service.py  # 启动pipeline服务端的脚本readme.md                   # 识别模型服务化部署文档run_cpp_serving.sh          # 启动C++ Pipeline Serving部署的脚本test_cpp_serving_client.py  # rpc方式发送C++ Pipeline serving预测请求的脚本

修改config.yml文件如下：

#worker_num, 最大并发数。当build_dag_each_worker=True时, 框架会创建worker_num个进程，每个进程内构建grpcSever和DAG
##当build_dag_each_worker=False时，框架会设置主线程grpc线程池的max_workers=worker_num
worker_num: 1#http端口, rpc_port和http_port不允许同时为空。当rpc_port可用且http_port为空时，不自动生成http_port
http_port: 8899
#rpc_port: 9994dag:#op资源类型, True, 为线程模型；False，为进程模型is_thread_op: False
op:rec_brands:#并发数，is_thread_op=True时，为线程并发；否则为进程并发concurrency: 1#当op配置没有server_endpoints时，从local_service_conf读取本地服务配置local_service_conf:#uci模型路径model_config: ../../models/rec_brands_v1.0_serving#计算硬件类型: 空缺时由devices决定(CPU/GPU)，0=cpu, 1=gpu, 2=tensorRT, 3=arm cpu, 4=kunlun xpudevice_type: 1#计算硬件ID，当devices为""或不写时为CPU预测；当devices为"0", "0,1,2"时为GPU预测，表示使用的GPU卡devices: "0" # "0,1"#client类型，包括brpc, grpc和local_predictor.local_predictor不启动Serving服务，进程内预测client_type: local_predictor#Fetch结果列表，以client_config中fetch_var的alias_name为准fetch_list: ["features"]rec_series:#并发数，is_thread_op=True时，为线程并发；否则为进程并发concurrency: 1#当op配置没有server_endpoints时，从local_service_conf读取本地服务配置local_service_conf:#uci模型路径model_config: ../../models/rec_series_v1.0_serving#计算硬件类型: 空缺时由devices决定(CPU/GPU)，0=cpu, 1=gpu, 2=tensorRT, 3=arm cpu, 4=kunlun xpudevice_type: 1#计算硬件ID，当devices为""或不写时为CPU预测；当devices为"0", "0,1,2"时为GPU预测，表示使用的GPU卡devices: "0" # "0,1"#client类型，包括brpc, grpc和local_predictor.local_predictor不启动Serving服务，进程内预测client_type: local_predictor#Fetch结果列表，以client_config中fetch_var的alias_name为准fetch_list: ["features"]det:concurrency: 1local_service_conf:client_type: local_predictordevice_type: 1devices: '0'fetch_list:- save_infer_model/scale_0.tmp_1model_config: ../../models/picodet_PPLCNet_x2_5_mainbody_lite_v1.0_serving/

修改recognition_web_service.py文件如下：

# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import datetimefrom paddle_serving_server.web_service import WebService, Op
from paddle_serving_server.pipeline import RequestOp, ResponseOp
from paddle_serving_server.pipeline import PipelineServer
from paddle_serving_server.pipeline.proto import pipeline_service_pb2
from paddle_serving_server.pipeline.channel import ChannelDataErrcode
import logging
import numpy as np
import sys
import cv2
from paddle_serving_app.reader import *
import base64
import os
import faiss
import pickle
import jsonclass TestRequestOp(RequestOp):def init_op(self):passdef unpack_request_package(self, request):# print(str(request.method))dict_data = {}log_id = Noneif request is None:raise ValueError("request is None")for idx, key in enumerate(request.key):dict_data[key] = request.value[idx]log_id = request.logidreturn dict_data, log_id, None, ""class DetOp(Op):def init_op(self):self.img_preprocess = Sequential([BGR2RGB(), Div(255.0),Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225], False),Resize((640, 640)), Transpose((2, 0, 1))])self.img_postprocess = RCNNPostprocess("label_list.txt", "output")self.threshold = 0.2self.max_det_results = 5def generate_scale(self, im):"""Args:im (np.ndarray): image (np.ndarray)Returns:im_scale_x: the resize ratio of Xim_scale_y: the resize ratio of Y"""target_size = [640, 640]origin_shape = im.shape[:2]resize_h, resize_w = target_sizeim_scale_y = resize_h / float(origin_shape[0])im_scale_x = resize_w / float(origin_shape[1])return im_scale_y, im_scale_xdef preprocess(self, input_dicts, data_id, log_id):print("{} detect begin --> data_id: {}".format(datetime.datetime.now(), data_id))(_, input_dict), = input_dicts.items()imgs = []raw_imgs = []for key in input_dict.keys():data = base64.b64decode(input_dict[key].encode('utf8'))raw_imgs.append(data)data = np.fromstring(data, np.uint8)raw_im = cv2.imdecode(data, cv2.IMREAD_COLOR)im_scale_y, im_scale_x = self.generate_scale(raw_im)im = self.img_preprocess(raw_im)im_shape = np.array(im.shape[1:]).reshape(-1)scale_factor = np.array([im_scale_y, im_scale_x]).reshape(-1)imgs.append({"image": im[np.newaxis, :],"im_shape": im_shape[np.newaxis, :],"scale_factor": scale_factor[np.newaxis, :],})self.raw_img = raw_imgsfeed_dict = {"image": np.concatenate([x["image"] for x in imgs], axis=0),"im_shape": np.concatenate([x["im_shape"] for x in imgs], axis=0),"scale_factor": np.concatenate([x["scale_factor"] for x in imgs], axis=0)}return feed_dict, False, None, ""def postprocess(self, input_dicts, fetch_dict, data_id, log_id):boxes = self.img_postprocess(fetch_dict, visualize=False)boxes.sort(key=lambda x: x["score"], reverse=True)boxes = filter(lambda x: x["score"] >= self.threshold,boxes[:self.max_det_results])boxes = list(boxes)for i in range(len(boxes)):boxes[i]["bbox"][2] += boxes[i]["bbox"][0] - 1boxes[i]["bbox"][3] += boxes[i]["bbox"][1] - 1result = json.dumps(boxes)res_dict = {"bbox_result": result, "image": self.raw_img}print("{} detect finish --> data_id: {}".format(datetime.datetime.now(), data_id))return res_dict, None, ""class BrandsRecOp(Op):def init_op(self):self.seq = Sequential([BGR2RGB(), Resize((224, 224)), Div(255),Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225],False), Transpose((2, 0, 1))])index_dir = "../../brand_dataset_v1.0/index"assert os.path.exists(os.path.join(index_dir, "vector.index")), "vector.index not found ..."assert os.path.exists(os.path.join(index_dir, "id_map.pkl")), "id_map.pkl not found ... "self.searcher = faiss.read_index(os.path.join(index_dir, "vector.index"))with open(os.path.join(index_dir, "id_map.pkl"), "rb") as fd:self.id_map = pickle.load(fd)self.rec_nms_thresold = 0.05self.rec_score_thres = 0.5self.feature_normalize = Trueself.return_k = 1self.area_ratio_thresold=0.1def preprocess(self, input_dicts, data_id, log_id):(_, input_dict), = input_dicts.items()raw_img = input_dict["image"][0]data = np.frombuffer(raw_img, np.uint8)origin_img = cv2.imdecode(data, cv2.IMREAD_COLOR)dt_boxes = input_dict["bbox_result"]boxes = json.loads(dt_boxes)boxes.append({"category_id": 0,"score": 1.0,"bbox": [0, 0, origin_img.shape[1], origin_img.shape[0]]})self.det_boxes = boxes#construct batch images for recimgs = []for box in boxes:box = [int(x) for x in box["bbox"]]im = origin_img[box[1]:box[3], box[0]:box[2]].copy()img = self.seq(im)imgs.append(img[np.newaxis, :].copy())input_imgs = np.concatenate(imgs, axis=0)return {"x": input_imgs}, False, None, ""def nms_to_rec_results(self, results, thresh=0.1):filtered_results = []x1 = np.array([r["bbox"][0] for r in results]).astype("float32")y1 = np.array([r["bbox"][1] for r in results]).astype("float32")x2 = np.array([r["bbox"][2] for r in results]).astype("float32")y2 = np.array([r["bbox"][3] for r in results]).astype("float32")scores = np.array([r["rec_scores"] for r in results])areas = (x2 - x1 + 1) * (y2 - y1 + 1)order = scores.argsort()[::-1]while order.size > 0:i = order[0]xx1 = np.maximum(x1[i], x1[order[1:]])yy1 = np.maximum(y1[i], y1[order[1:]])xx2 = np.minimum(x2[i], x2[order[1:]])yy2 = np.minimum(y2[i], y2[order[1:]])w = np.maximum(0.0, xx2 - xx1 + 1)h = np.maximum(0.0, yy2 - yy1 + 1)inter = w * hovr = inter / (areas[i] + areas[order[1:]] - inter)inds = np.where(ovr <= thresh)[0]order = order[inds + 1]filtered_results.append(results[i])return filtered_resultsdef check_boxes(self, results, area_ratio_thresh=0.1):filtered_results = []for result in results:if result["area_ratio"]>=area_ratio_thresh:filtered_results.append(result)if len(filtered_results)>0:return filtered_resultselse:return resultsdef postprocess(self, input_dicts, fetch_dict, data_id, log_id):batch_features = fetch_dict["features"]if self.feature_normalize:feas_norm = np.sqrt(np.sum(np.square(batch_features), axis=1, keepdims=True))batch_features = np.divide(batch_features, feas_norm)scores, docs = self.searcher.search(batch_features, self.return_k)origin_img_box = self.det_boxes[len(self.det_boxes) - 1]["bbox"]total_pixes = origin_img_box[2] * origin_img_box[3]results = []for i in range(scores.shape[0]):pred = {}xmin, ymin, xmax, ymax = self.det_boxes[i]["bbox"]area_pix = (xmax - xmin) * (ymax - ymin)ratio = 0.0if total_pixes > 0:ratio = area_pix * 1.0 / total_pixesif scores[i][0] >= self.rec_score_thres:pred["bbox"] = [int(x) for x in self.det_boxes[i]["bbox"]]pred["rec_docs"] = self.id_map[docs[i][0]].split()[1]pred["rec_scores"] = scores[i][0]pred["area_ratio"] = round(ratio, 4)results.append(pred)#do nmsresults = self.nms_to_rec_results(results, self.rec_nms_thresold)print("{} BrandsRecOp data_id: {} --> Nms Result: {}".format(datetime.datetime.now(), data_id, results))results = self.check_boxes(results, self.area_ratio_thresold)print("{} BrandsRecOp data_id: {} --> Out Result: {}".format(datetime.datetime.now(), data_id, results))return {"result": str(results)}, None, ""class SeriesRecOp(Op):def init_op(self):self.seq = Sequential([BGR2RGB(), Resize((224, 224)), Div(255),Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225],False), Transpose((2, 0, 1))])index_dir = "../../series_dataset_v1.0/index"assert os.path.exists(os.path.join(index_dir, "vector.index")), "vector.index not found ..."assert os.path.exists(os.path.join(index_dir, "id_map.pkl")), "id_map.pkl not found ... "self.searcher = faiss.read_index(os.path.join(index_dir, "vector.index"))with open(os.path.join(index_dir, "id_map.pkl"), "rb") as fd:self.id_map = pickle.load(fd)self.rec_nms_thresold = 0.05self.rec_score_thres = 0.5self.feature_normalize = Trueself.return_k = 1self.area_ratio_thresold=0.1def preprocess(self, input_dicts, data_id, log_id):(_, input_dict), = input_dicts.items()raw_img = input_dict["image"][0]data = np.frombuffer(raw_img, np.uint8)origin_img = cv2.imdecode(data, cv2.IMREAD_COLOR)dt_boxes = input_dict["bbox_result"]boxes = json.loads(dt_boxes)boxes.append({"category_id": 0,"score": 1.0,"bbox": [0, 0, origin_img.shape[1], origin_img.shape[0]]})self.det_boxes = boxes#construct batch images for recimgs = []for box in boxes:box = [int(x) for x in box["bbox"]]im = origin_img[box[1]:box[3], box[0]:box[2]].copy()img = self.seq(im)imgs.append(img[np.newaxis, :].copy())input_imgs = np.concatenate(imgs, axis=0)return {"x": input_imgs}, False, None, ""def nms_to_rec_results(self, results, thresh=0.1):filtered_results = []x1 = np.array([r["bbox"][0] for r in results]).astype("float32")y1 = np.array([r["bbox"][1] for r in results]).astype("float32")x2 = np.array([r["bbox"][2] for r in results]).astype("float32")y2 = np.array([r["bbox"][3] for r in results]).astype("float32")scores = np.array([r["rec_scores"] for r in results])areas = (x2 - x1 + 1) * (y2 - y1 + 1)order = scores.argsort()[::-1]while order.size > 0:i = order[0]xx1 = np.maximum(x1[i], x1[order[1:]])yy1 = np.maximum(y1[i], y1[order[1:]])xx2 = np.minimum(x2[i], x2[order[1:]])yy2 = np.minimum(y2[i], y2[order[1:]])w = np.maximum(0.0, xx2 - xx1 + 1)h = np.maximum(0.0, yy2 - yy1 + 1)inter = w * hovr = inter / (areas[i] + areas[order[1:]] - inter)inds = np.where(ovr <= thresh)[0]order = order[inds + 1]filtered_results.append(results[i])return filtered_resultsdef check_boxes(self, results, area_ratio_thresh=0.1):filtered_results = []for result in results:if result["area_ratio"]>=area_ratio_thresh:filtered_results.append(result)if len(filtered_results)>0:return filtered_resultselse:return resultsdef postprocess(self, input_dicts, fetch_dict, data_id, log_id):batch_features = fetch_dict["features"]if self.feature_normalize:feas_norm = np.sqrt(np.sum(np.square(batch_features), axis=1, keepdims=True))batch_features = np.divide(batch_features, feas_norm)scores, docs = self.searcher.search(batch_features, self.return_k)origin_img_box = self.det_boxes[len(self.det_boxes) - 1]["bbox"]total_pixes = origin_img_box[2] * origin_img_box[3]results = []for i in range(scores.shape[0]):pred = {}xmin, ymin, xmax, ymax = self.det_boxes[i]["bbox"]area_pix = (xmax - xmin) * (ymax - ymin)ratio = 0.0if total_pixes > 0:ratio = area_pix * 1.0 / total_pixesif scores[i][0] >= self.rec_score_thres:pred["bbox"] = [int(x) for x in self.det_boxes[i]["bbox"]]pred["rec_docs"] = self.id_map[docs[i][0]].split()[1]pred["rec_scores"] = scores[i][0]pred["area_ratio"] = round(ratio, 4)results.append(pred)#do nmsresults = self.nms_to_rec_results(results, self.rec_nms_thresold)print("{} SeriesRecOp data_id: {} --> Nms Result: {}".format(datetime.datetime.now(), data_id, results))results = self.check_boxes(results, self.area_ratio_thresold)print("{} SeriesRecOp data_id: {} --> Out Result: {}".format(datetime.datetime.now(), data_id, results))return {"result": str(results)}, None, ""class CombineOp(Op):def preprocess(self, input_data, data_id, log_id):return None, False, None, ""def postprocess(self, input_dicts, fetch_dict, data_id, log_id):print("{} CombineOp data_id: {} --> input_dicts: {}".format(datetime.datetime.now(), data_id, input_dicts))results = {}for op_name, data in input_dicts.items():if "brands" in op_name:ret = data["result"]if ret is not None:results["brands"] = json.loads(ret.replace("'", "\""))else:results["brands"] = "[]"elif "series" in op_name:ret = data["result"]if ret is not None:results["series"] = json.loads(ret.replace("'", "\""))else:results["series"] = "[]"print("{} CombineOp data_id: {} --> Out Result: {}".format(datetime.datetime.now(), data_id, results))return {"result": str(results)}, None, ""class RecognitionService(WebService):def get_pipeline_response(self, read_op):read_op2 = TestRequestOp()det_op = DetOp(name="det", input_ops=[read_op2])rec_brands_op = BrandsRecOp(name="rec_brands", input_ops=[det_op])rec_series_op = SeriesRecOp(name="rec_series", input_ops=[det_op])combine_op = CombineOp("combine", input_ops=[rec_brands_op, rec_series_op])return combine_opproduct_recog_service = RecognitionService(name="recognition")
product_recog_service.prepare_pipeline_config("config.yml")
product_recog_service.run_service()

启动服务：

# 启动服务，运行日志保存在 log.txt
nohup python3.8 recognition_web_service.py &>log.txt &

如果出现faiss没找到，请参考这里：https://blog.csdn.net/weixin_43882112/article/details/107614217

查看进程

ps -ef|grep python

关闭进程

kill -9 19913

查看日志

tail -f 1000 log.log

如何查看端口占用

$: netstat -anp | grep 8888
tcp        0      0 127.0.0.1:8888          0.0.0.0:*               LISTEN      13404/python3
tcp        0      1 172.17.0.10:34036       115.42.35.84:8888       SYN_SENT    14586/python3

强制杀掉进程：通过pid

$: kill -9 13404
$: kill -9 14586
$: netstat -anp | grep 8888
$:

服务测试

修改pipeline_http_client.py文件如下：

import requests
import json
import base64
import osimgpath = "图片路径.jpg"def cv2_to_base64(image):return base64.b64encode(image).decode('utf8')if __name__ == "__main__":url = "http://127.0.0.1:8899/recognition/prediction"with open(os.path.join(".",  imgpath), 'rb') as file:image_data1 = file.read()image = cv2_to_base64(image_data1)data = {"key": ["image"], "value": [image]}for i in range(1):r = requests.post(url=url, data=json.dumps(data))print(r.json())

发送请求：

python3.8 pipeline_http_client.py

成功运行后，模型预测的结果会打印在客户端中，如下所示：

{'err_no': 0, 'err_msg': '', 'key': ['result'], 'value': ["{'brands': [{'bbox': [16, 19, 492, 565], 'rec_docs': '1', 'rec_scores': 0.98805684, 'area_ratio': 0.7432}], 'series': [{'bbox': [16, 19, 492, 565], 'rec_docs': '6', 'rec_scores': 0.9267364, 'area_ratio': 0.7432}]}"], 'tensors': []}