OpenCV—python 模板匹配与图像特征匹配

文章目录

一、理论介绍与算法
二、算法代码
- 单目标匹配
- 多目标匹配
三多尺度模板匹配

一、理论介绍与算法

模板匹配是在一幅图像中寻找一个特定目标的方法之一，这种方法的原理非常简单，遍历图像中的每一个可能的位置，比较各处与模板是否“相似”，当相似度足够高时，就认为找到了我们的目标

用T表示模板图像，I表示待匹配图像，切模板图像的宽为w高为h，用R表示匹配结果，匹配过程(图示如上)：

通过滑动, 我们的意思是图像块一次移动一个像素 (从左往右,从上往下). 在每一个位置, 都进行一次度量计算来表明它是 “好” 或 “坏” 地与那个位置匹配 (或者说块图像和原图像的特定区域有多么相似).
对于 T 覆盖在 I 上的每个位置,你把度量值保存到 结果图像矩阵 ® 中. 在 R 中的每个位置 ( x . y ) (x.y) (x.y)都包含匹配度量值,红色椭圆框住的位置很可能是结果图像矩阵中的最大数值, 所以这个区域 (以这个点为顶点,长宽和模板图像一样大小的矩阵) 被认为是匹配的。
我们使用函数 minMaxLoc 来定位在矩阵 R 中的最大值点 (或者最小值, 根据函数输入的匹配参数)

OpenCV提供了6种模板匹配算法：

平方差匹配法CV_TM_SQDIFF（最好匹配为0.匹配越差,匹配值越大）
R ( x , y ) = ∑ x ′ , y ′ [ T ( x ′ , y ′ ) − I ( x + x ′ , y + y ′ ) ] 2 \rm R(x,y) = \sum_{x',y'}[T(x',y') -I(x+x',y+y')]^2 R(x,y)=x′,y′∑[T(x′,y′)−I(x+x′,y+y′)]2
归一化平方差匹配法CV_TM_SQDIFF_NORMED
R ( x , y ) = ∑ x ′ , y ′ [ T ( x ′ , y ′ ) − I ( x + x ′ , y + y ′ ) ] 2 ∑ x ′ , y ′ T ( x ′ , y ′ ) 2 ⋅ ∑ x ′ , y ′ I ( x + x ′ , y + y ′ ) 2 \rm R(x,y) = \frac{\sum_{x',y'}[T(x',y') -I(x+x',y+y')]^2}{\sqrt{\sum_{x',y'}T(x',y')^2 ·\sum_{x',y'}I(x+x',y+y')^2}} R(x,y)=∑x′,y′T(x′,y′)2⋅∑x′,y′I(x+x′,y+y′)2 ∑x′,y′[T(x′,y′)−I(x+x′,y+y′)]2
相关匹配法CV_TM_CCORR
这类方法采用模板和图像间的乘法操作,所以较大的数表示匹配程度较高,0标识最坏的匹配效果
R ( x , y ) = ∑ x ′ , y ′ [ T ( x ′ , y ′ ) ⋅ I ( x + x ′ , y + y ′ ) ] \rm R(x,y) = \sum_{x',y'}[T(x',y') ·I(x+x',y+y')] R(x,y)=x′,y′∑[T(x′,y′)⋅I(x+x′,y+y′)]
归一化相关匹配法CV_TM_CCORR_NORMED
R ( x , y ) = ∑ x ′ , y ′ [ T ( x ′ , y ′ ) ⋅ I ′ ( x + x ′ , y + y ′ ) ] ∑ x ′ , y ′ T ( x ′ , y ′ ) 2 ⋅ ∑ x ′ , y ′ I ( x + x ′ , y + y ′ ) 2 \rm R(x,y) = \frac{\sum_{x',y'}[T(x',y') ·I'(x+x',y+y')]}{\sqrt{\sum_{x',y'}T(x',y')^2 ·\sum_{x',y'}I(x+x',y+y')^2}} R(x,y)=∑x′,y′T(x′,y′)2⋅∑x′,y′I(x+x′,y+y′)2 ∑x′,y′[T(x′,y′)⋅I′(x+x′,y+y′)]
相关系数匹配法CV_TM_CCOEFF
这类方法将模版对其均值的相对值与图像对其均值的相关值进行匹配,1表示完美匹配,-1表示糟糕的匹配,0表示没有任何相关性(随机序列)。
R ( x , y ) = ∑ x ′ , y ′ [ T ′ ( x ′ , y ′ ) ⋅ I ( x + x ′ , y + y ′ ) ] \rm R(x,y) = \sum_{x',y'}[T'(x',y') ·I(x+x',y+y')] R(x,y)=x′,y′∑[T′(x′,y′)⋅I(x+x′,y+y′)]
其中： T ′ ( x ′ , y ′ ) = T ( x ′ , y ′ ) − 1 w ⋅ h ⋅ ∑ x ′ ′ , y ′ ′ T ( x ′ ′ , y ′ ′ ) \rm T'(x',y') = T(x',y')-\frac{1}{w·h}·\sum_{x'',y''}T(x'',y'') T′(x′,y′)=T(x′,y′)−w⋅h1⋅∑x′′,y′′T(x′′,y′′)
I ′ ( x + x ′ , y + y ′ ) = I ( x + x ′ , y + y ′ ) − 1 w ⋅ h ⋅ ∑ x ′ ′ , y ′ ′ I ( x + x ′ ′ , y + y ′ ′ ) \rm I'(x+x',y+y') =I(x+x',y+y') -\frac{1}{w·h}·\sum_{x'',y''}I(x+x'',y+y'') I′(x+x′,y+y′)=I(x+x′,y+y′)−w⋅h1⋅∑x′′,y′′I(x+x′′,y+y′′)
归一化相关系数匹配法CV_TM_CCOEFF_NORMED
R ( x , y ) = ∑ x ′ , y ′ [ T ′ ( x ′ , y ′ ) ⋅ I ′ ( x + x ′ , y + y ′ ) ] ∑ x ′ , y ′ T ′ ( x ′ , y ′ ) 2 ⋅ ∑ x ′ , y ′ I ′ ( x + x ′ , y + y ′ ) 2 \rm R(x,y) = \frac{\sum_{x',y'}[T'(x',y') ·I'(x+x',y+y')]}{\sqrt{\sum_{x',y'}T'(x',y')^2 ·\sum_{x',y'}I'(x+x',y+y')^2}} R(x,y)=∑x′,y′T′(x′,y′)2⋅∑x′,y′I′(x+x′,y+y′)2 ∑x′,y′[T′(x′,y′)⋅I′(x+x′,y+y′)]

显然公式越来越复杂，计算量也很大，但是准确度也跟着提高，看如何取舍了。

二、算法代码

如下参考代码中是直接操作像素，显然速度非常慢，我在如上（左）图片中匹配上（右）图。我使用numpy矩阵运算方式重写了以上算法。

import numpy as np
import time
import cv2def EM_EM2(temp):array = temp.reshape(1,-1)EM_sum = np.double(np.sum(array[0]))square_arr = np.square(array[0])EM2_sum = np.double(np.sum(square_arr))return EM_sum,EM2_sumdef EI_EI2(img, u, v,temp):height, width = temp.shape[:2]roi = img[v:v+height, u:u+width]array_roi = roi.reshape(1,-1)EI_sum = np.double(np.sum(array_roi[0]))square_arr = np.square(array_roi[0])EI2_sum = np.double(np.sum(square_arr))return EI_sum,EI2_sumdef EIM(img, u, v, temp):height, width = temp.shape[:2]roi = img[v:v+height, u:u+width]product = temp*roi*1.0product_array = product.reshape(1, -1)sum = np.double(np.sum(product_array[0]))return sumdef Match(img, temp):imgHt, imgWd = img.shape[:2]height, width = temp.shape[:2]uMax = imgWd-widthvMax = imgHt-heighttemp_N = width*heightmatch_len = (uMax+1)*(vMax+1)MatchRec = [0.0 for _ in range(0, match_len)]k = 0EM_sum, EM2_sum = EM_EM2(temp)for u in range(0, uMax+1):for v in range(0, vMax+1):EI_sum, EI2_sum = EI_EI2(img, u, v, temp)IM = EIM(img,u,v,temp)numerator=(  temp_N * IM - EI_sum*EM_sum)*(temp_N * IM - EI_sum * EM_sum)denominator=(temp_N * EI2_sum - EI_sum**2)*(temp_N * EM2_sum - EM_sum**2)ret = numerator/denominatorMatchRec[k]=retk+=1print('进度==》[{}]'.format(u/(vMax+1)))val = 0k = 0x = y = 0for p in range(0, uMax+1):for q in range(0, vMax+1):if MatchRec[k] > val:val = MatchRec[k]x = py = qk+=1print ("val: %f"%val)return (x, y)def main():img = cv2.imread('./gggg/001A12.png', cv2.IMREAD_GRAYSCALE)temp = cv2.imread('./gggg/001A1.png', cv2.IMREAD_GRAYSCALE)tempHt, tempWd = temp.shape(x, y) = Match(img, temp)cv2.rectangle(img, (x, y), (x+tempWd, y+tempHt), (0,0,0), 2)cv2.imshow("temp", temp)cv2.imshow("result", img)cv2.waitKey(0)cv2.destroyAllWindows()if __name__ == '__main__':start = time.time()main()end = time.time()print("Total Spend time：", str((end - start) / 60)[0:6] + "分钟")'''
val: 1.000025
Total Spend time： 0.0866分钟
'''

为了更快的进行算法验证，用上述代码进行验证时请尽量选用较小的匹配图像及模板图像：

单目标匹配

import cv2
from matplotlib import pyplot as pltimg = cv2.imread('./gggg/001.png',0)
template = cv2.imread('./gggg/001A1.png',0)
w, h = template.shape[::-1]methods = ['cv2.TM_SQDIFF', 'cv2.TM_SQDIFF_NORMED','cv2.TM_CCORR','cv2.TM_CCORR_NORMED','cv2.TM_CCOEFF', 'cv2.TM_CCOEFF_NORMED']for meth in methods:method = eval(meth)res = cv2.matchTemplate(img,template,method)min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(res) #找到最大值和最小值print('{}:cv2.minMaxLoc(res):\t{} '.format(meth,cv2.minMaxLoc(res)))if method in [cv2.TM_SQDIFF, cv2.TM_SQDIFF_NORMED]:top_left = min_locelse:top_left = max_locbottom_right = (top_left[0] + w, top_left[1] + h)cv2.rectangle(img,top_left, bottom_right, 255, 2)plt.subplot(121),plt.imshow(res,cmap = 'gray')plt.title('Matching Result'), plt.xticks([]), plt.yticks([])plt.subplot(122),plt.imshow(img,cmap = 'gray')plt.title('Detected Point'), plt.xticks([]), plt.yticks([])plt.suptitle(meth)plt.show()

多目标匹配

import cv2
import numpytarget = cv2.imread("./gggg/001A2.png")     #读取目标图片
template = cv2.imread("./gggg/001A1.png")   #读取模板图片theight, twidth = template.shape[:2]
result = cv2.matchTemplate(target,template,cv2.TM_SQDIFF_NORMED)
print('result:',result)
#归一化处理
#cv2.normalize( result, result, 0, 1, cv2.NORM_MINMAX, -1 )
min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(result)#绘制矩形边框，将匹配区域标注出来
cv2.rectangle(target,min_loc,(min_loc[0]+twidth,min_loc[1]+theight),(0,0,225),2)#对于cv2.TM_SQDIFF及cv2.TM_SQDIFF_NORMED方法min_val越趋近与0匹配度越好，匹配位置取min_loc
#对于其他方法max_val越趋近于1匹配度越好，匹配位置取max_loc
strmin_val = str(min_val)
temp_loc = min_loc
other_loc = min_loc
numOfloc = 1threshold = 0.01                    # 设置匹配阈值为0.01
loc = numpy.where(result<threshold) # 第一次筛选
for other_loc in zip(*loc[::-1]):   # 第二次筛选--将位置偏移小于5个像素的结果舍去if (temp_loc[0]+5<other_loc[0])or(temp_loc[1]+5<other_loc[1]):numOfloc = numOfloc + 1temp_loc = other_loccv2.rectangle(target,other_loc,(other_loc[0]+twidth,other_loc[1]+theight),(0,0,225),2)
str_numOfloc = str(numOfloc)strText = "MatchResult--MatchingValue="+strmin_val+"--NumberOfPosition="+str_numOfloc
cv2.imshow(strText,target)
cv2.waitKey()
cv2.destroyAllWindows()

三多尺度模板匹配

关键点检测器，局部不变描述符（如SIFT，SURF，FREAK等），以及与RANSAC或LMEDs的关键点匹配

由于图片中文字的大小与制作的模板大小以及比例肯定有出入，为了使用opencv 进行模板匹配，以多种比例循环输入图像（即使输入图像逐渐变小和变小）。
使用cv2应用模板匹配。匹配模板并跟踪最大相关系数的匹配（以及具有最大相关系数的区域的 x，y坐标）。
遍历所有尺度后，取相关系数最大的区域作为“匹配”区域。试图匹配旋转的对象或呈现非仿射变换的对象，则模板匹配并不理想。

模块安装
pip install imutils

import numpy as np
import argparse
import imutils
import glob
import cv2

ap = argparse.ArgumentParser()
ap.add_argument("-t", "--template", required=True, help="Path to template image")
ap.add_argument("-i", "--images", required=True,help="Path to images where template will be matched")ap.add_argument("-v", "--visualize",help="Flag indicating whether or not to visualize each iteration")
args = vars(ap.parse_args())template = cv2.imread(args["template"])
template = cv2.cvtColor(template, cv2.COLOR_BGR2GRAY)
template = cv2.Canny(template, 50, 200)(tH, tW) = template.shape[:2]
cv2.imshow("Template", template)

# loop over the images to find the template infor imagePath in glob.glob(args["images"] + "/*.jpg"):image = cv2.imread(imagePath)gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)found = None# loop over the scales of the image 起始值，结束值，以及相等的块片数for scale in np.linspace(0.2, 1.0, 20)[::-1]:resized = imutils.resize(gray, width = int(gray.shape[1] * scale))r = gray.shape[1] / float(resized.shape[1])if resized.shape[0] < tH or resized.shape[1] < tW:breakedged = cv2.Canny(resized, 50, 200)result = cv2.matchTemplate(edged, template, cv2.TM_CCOEFF)(_, maxVal, _, maxLoc) = cv2.minMaxLoc(result)# check to see if the iteration should be visualizedif args.get("visualize", False):clone = np.dstack([edged, edged, edged])cv2.rectangle(clone, (maxLoc[0], maxLoc[1]),(maxLoc[0] + tW, maxLoc[1] + tH), (0, 0, 255), 2)cv2.imshow("Visualize", clone)cv2.waitKey(0)# if we have found a new maximum correlation value, then ipdateif found is None or maxVal > found[0]:found = (maxVal, maxLoc, r)# unpack the bookkeeping varaible and compute the (x, y) coordinates(_, maxLoc, r) = found(startX, startY) = (int(maxLoc[0] * r), int(maxLoc[1] * r))(endX, endY) = (int((maxLoc[0] + tW) * r), int((maxLoc[1] + tH) * r))# draw a bounding box around the detected result and display the imagecv2.rectangle(image, (startX, startY), (endX, endY), (0, 0, 255), 2)cv2.imshow("Image", image)cv2.waitKey(0)

打开终端输入命令

$ python match.py --template cod_logo.png --images images
$ python match.py --template cod_logo.png --images images --visualize 1 #可视化

特别鸣谢
https://www.pyimagesearch.com/2015/01/26/multi-scale-template-matching-using-python-opencv/

模板匹配与文本识别
https://blog.csdn.net/k_shmily/article/details/52002453

Harris角点：二、FAST特征点 SURF特征的描述
https://blog.csdn.net/xzl0318/article/details/70860318
https://blog.csdn.net/robinhjwy/article/details/77620924
https://blog.csdn.net/lql0716/article/details/52491085

鸣谢：
https://blog.csdn.net/HuangZhang_123/article/details/80660688
https://blog.csdn.net/zhuisui_woxin/article/details/84400439
http://www.opencv.org.cn/opencvdoc/2.3.2/html/doc/tutorials/imgproc/histograms/template_matching/template_matching.html