<?xml version="1.0" ?>
<annotation><folder>VOC2007</folder><filename>crop_000010.png</filename><source><database>PASperson Database</database><annotation>PASperson</annotation></source><size><width>594</width><height>720</height><depth>3</depth></size><segmented>0</segmented><object><name>person</name><pose>Unspecified</pose><truncated>0</truncated><difficult>0</difficult><bndbox><xmin>194</xmin><ymin>127</ymin><xmax>413</xmax><ymax>647</ymax></bndbox></object>
</annotation>

我们将使用这些注释使模型可识别对象。但是，在继续前进之前，我们首先需要将这些批注解析为csv文件并提取所需的数据。Python提供了用于解析xml文件的ElementTree API。下面是一个函数，可用于轻松加载和解析xml批注文件。

def parse_xml_to_csv(path):xml_list = []#iterate over all files to extract the bounding box for person present in the corresponding imagefor xml_annot in glob.glob(path + '/*.xml'):#load and parse filetree = ET.parse(xml_annot)doc = etree.parse(xml_annot)count = doc.xpath("count(//object)")#getting root of the documentroot = tree.getroot()with open(str(xml_annot)[0:-4]+".csv","w+") as file:file.write(str(int(count)))for person in root.findall('object'):value = (person[4][0].text,person[4][1].text,person[4][2].text,person[4][3].text)coors = " ".join(value)with open(str(xml_annot)[0:-4]+".csv","a") as file:file.write("\n")file.write(coors)

调用上面的函数，将路径传递到注释文件作为参数：

annot_path ="./Annotations"
xml_df = parse_xml_to_csv(annot_path)

功能完成后，您可以查看所有转换后的csv文件。

import os                        # to interact with OS
import cv2                       # to perform selective search on images
import keras                     # to implement neural net
import numpy as np               # to work with arrays
import pandas as pd              # for data manipulation
import tensorflow as tf          # for deep learning models
import matplotlib.pyplot as plt  # for plotting

正如我们之前提到的，搜索感兴趣的区域在计算上已耗费大量力气，因此我们将在此处尝试实现有效的解决方案。选择性搜索根据颜色、纹理、大小或形状计算相似度，并按层次将最相似的区域分组。继续该过程，直到整个图像变为单个区域。OpenCV提供使用该createSelectiveSearchSegmentation函数来实现选择性搜索。将优化和选择性搜索添加到您的解决方案中，如下所示：

# OpenCV optimization
cv2.setUseOptimized(True);
# selective search
selective_search = cv2.ximgproc.segmentation.createSelectiveSearchSegmentation()

现在，如果我们在测试图像上应用选择性搜索，则会导致所需对象周围的边界框。

def compute_iou(box1, box2):x_left = max(box1['x1'], box2['x1'])y_top = max(box1['y1'], box2['y1'])x_right = min(box1['x2'], box2['x2'])y_bottom = min(box1['y2'], box2['y2'])intersection_area = (x_right - x_left) * (y_bottom - y_top)box1_area = (box1['x2'] - box1['x1']) * (box1['y2'] - box1['y1'])box2_area = (box2['x2'] - box2['x1']) * (box2['y2'] - box2['y1'])union_area = box1_area + box2_area - intersection_areaiou = intersection_area / union_areareturn iou

training_images=[]
training_labels=[]
for e,i in enumerate(os.listdir(annot)):try:filename = i.split(".")[0]+".png"img = cv2.imread(os.path.join(path,filename))dataframe = pd.read_csv(os.path.join(annot,i))ground_truth_values=[]for row in dataframe.iterrows():x1 = int(row[1][0].split(" ")[0])y1 = int(row[1][0].split(" ")[1])x2 = int(row[1][0].split(" ")[2])y2 = int(row[1][0].split(" ")[3])ground_truth_values.append({"x1":x1,"x2":x2,"y1":y1,"y2":y2})# setting the image as base image for selective searchselective_search.setBaseImage(img)# initializing fast selective searchselective_search.switchToSelectiveSearchFast()# getting proposed regionsssresults = selective_search.process()imout = img.copy()counter = 0f_counter = 0flag = 0fflag = 0bflag = 0for e,result in enumerate(ssresults):# iterating over the first 2000 results from selective search to colculate IOUif e < 2000 and flag == 0:for val in ground_truth_values:x,y,w,h = resultiou = compute_iou(val,{"x1":x,"x2":x+w,"y1":y,"y2":y+h})# limiting the maximum positive samples to 20if counter < 20:# setting IOU > 0.70 as goodness measure for positive i.e. person detectedif iou > 0.70:image = imout[y:y+h,x:x+w]resized = cv2.resize(image, (224,224), interpolation = cv2.INTER_AREA)training_images.append(resized)training_labels.append(1)counter += 1else :fflag =1# limiting the maximum negative samples to 20if f_counter <20:if iou < 0.3:image = imout[y:y+h,x:x+w]resized = cv2.resizetimage, (224,224), interpolation = cv2.INTER_AREA)training_images.append(resized)training_labels.append(0)f_counter += 1else :bflag = 1if fflag == 1 and bflag == 1:flag = 1except Exception as e:print(e)continue

现在，training_images和training_labels包含我们模型的新x和y坐标。让我们从模型的导入开始。

from keras.layers import Dense
from keras import Model
from keras import optimizers
from keras.optimizers import Adam
from keras.applications.vgg16 import VGG16
from keras.preprocessing.image import ImageDataGenerator
from keras.callbacks import ModelCheckpoint, EarlyStopping

从技术上讲，R-CNN模型可以从头开始拟合，但是它花费了太多时间，并且导致性能不佳。在这里，我们将使用迁移学习来节省时间并获得更好的性能。您可以根据自己的喜好使用imagenet或coco权重进行转移学习。

vggmodel = VGG16(weights='imagenet', include_top=True)
vggmodel.summary()

上面的代码片段导致以下输出：

for layers in (vggmodel.layers)[:10]:
layers.trainable = False

我们只对人类的存在与否感兴趣，这意味着我们只有两个类别可以预测，因此我们将添加两个具有softmax激活的单位密集层。使用softmax激活的原因是它确保输出的总和为1（即，输出是概率）。

X= vggmodel.layers[-2].output
predictions = Dense(2, activation="softmax")(X)

最后，我们将使用Adam优化器来编译模型。

model_final = Model(vggmodel.input, predictions)
model_final.compile(loss = keras.losses.categorical_crossentropy, optimizer = Adam(lr=0.001), metrics=["accuracy"])

我们已经创建了模型。在继续之前，我们需要对数据集进行编码。我们可以使用LabelBinarizer编码。

class Label_Binarizer(LabelBinarizer):def transform(self, y_old):Y = super().transform(y_old)if self.y_type_ == 'binary':return np.hstack((Y, 1-Y))else:return Ydef inverse(self, Y):if self.y_type_ == 'binary':return super().inverse(Y[:, 0])else:return super().inverse(Y)
encoded = Label_Binarizer()
Y =  encoded.fit_transform(y_new)

我们还需要将数据集分为训练集和测试集，可以使用来自sklearn的train_test_split来完成。在这里，我们将数据分为80％的训练和20％的测试比率。

X_train, X_test , y_train, y_test = train_test_split(X_new,Y,test_size=0.20)

Keras提供ImageDataGenerator来将数据集传递给模型。您还可以应用水平或垂直翻转来增加数据集。

train_data_prep = ImageDataGenerator(horizontal_flip=True, vertical_flip=True, rotation_range=90)
trainingdata = train_data_prep.flow(x=X_train, y=y_train)
test_data_prep = ImageDataGenerator(horizontal_flip=True, vertical_flip=True, rotation_range=90)
testingdata = test_data_prep.flow(x=X_test, y=y_test)

添加Keras回调

checkpoint = ModelCheckpoint("rcnn.h5", monitor='val_loss', verbose=1, save_best_only=True, save_weights_only=False, mode='auto', save_freq=1)
early = EarlyStopping(monitor='val_loss', min_delta=0, patience=100, verbose=1, mode='auto')
hist = model_final.fit_generator(generator= traindata, steps_per_epoch= 10, epochs= 500, validation_data= testdata, validation_steps=2, callbacks=[checkpoint,early])

测试我们的模型

count=0
for e,i in enumerate(os.listdir(path)):count += 1image = cv2.imread(os.path.join(path,i))selective_search.setBaseImage(image)selective_search.switchToSelectiveSearchFast()ssresults = selective_search.process()imout = image.copy()for e, res in enumerate(ssresults):if e < 2000:x,y,w,h = restest_image = imout[y:y+h,x:x+w]resized = cv2.resize(test_image, (224,224), interpolation = cv2.INTER_AREA)image = np.expand_dims(resized, axis=0)out= model_final.predict(image)if out[0][0] > 0.65:cv2.rectangle(imout, (x, y), (x+w, y+h), (0, 255, 0), 1, cv2.LINE_AA)plt.figure()plt.imshow(imout)