基于opencv的图像拼接
2024-04-24 05:50:02
基于opencv的图像拼接
问题
自动控制创新实践要求合并多机器人的slam地图，既合并有重叠部分的两张或多张图片， 比如说下面两张图片
1. 特征检测
使用SIFT算法进行特征检测，得到特征图
2.对特征进行K近邻算法进行匹配
得到匹配结果
3. 计算其中一张图的透视针对另一张图的透视变换
- 由于无法提前知道两张图片的位置关系，对于透视变换，可能图片会映射到整个选取区域的左边，这样的话，无法正常显示图片，因此，要对透视变换后的图片进行面积检查，如果比原来的图片面积小太多，就用另一张图片来进行透视变换
"""计算两张图的透视关系"""matchCount = len(matches)M = getHomography(kpsA, kpsB, matches, reprojThresh=4)if M is None:print("Error!")(matches, H, status) = M"""将图片A进行透视变换并检查图片位置"""result = cv2.warpPerspective(imageA, H, ((imageA.shape[1] + imageB.shape[1])*2, (imageA.shape[0] + imageB.shape[0]) * 2))resultAfterCut = cutBlack(result)# 检查图片位置if np.size(resultAfterCut) < np.size(imageA) * 0.95:print("图片位置不对,将自动调换")# 调换图片kpsA,kpsB = swap(kpsA, kpsB)imageA, imageB = swap(imageA, imageB)if feature_matching == 'bf':matches = matchKeyPointsBF(featuresB, featuresA, method=feature_extractor)elif feature_matching == 'knn':matches = matchKeyPointsKNN(featuresB, featuresA, ratio=0.75, method=feature_extractor)if len(matches) < 10:return NonematchCount = len(matches)M = getHomography(kpsA, kpsB, matches, reprojThresh=4)if M is None:print("Error!")(matches, H, status) = M4. 合并图片，大功告成
得到结果图，合并效果很完美
试试其他图片
- 图片1
融合效果很棒 - 图片二
这张图透视就有点过
如何进行多张图片的合并
很简单，反复调用合并两张图片就行，但是有些细节问题，无法提前得知多张图片的位置，先合并哪个？
既然我们有特征匹配环节，优先合并匹配特征数多的就行。
多张图片的效果图
第一个迷宫图，在百度图片随便找的
第二个slam地图，用的是fr079数据集
效果还行。
完整代码
- 代码已同步到github:https://github.com/799034552/concat_pic
注释很详细，不用怕看不懂
import cv2
import numpy as np
import matplotlib.pyplot as plt
import math
from numpy.core.defchararray import array# ================================================================== #
# 选择特征提取器函数
# ================================================================== #
def detectAndDescribe(image, method=None):assert method is not None, "You need to define a feature detection method. Values are: 'sift', 'surf'"if method == 'sift':descriptor = cv2.xfeatures2d.SIFT_create()elif method == 'surf':descriptor = cv2.xfeatures2d.SURF_create()elif method == 'brisk':descriptor = cv2.BRISK_create()elif method == 'orb':descriptor = cv2.ORB_create()(kps, features) = descriptor.detectAndCompute(image, None)return (kps, features)
# ================================================================== #
# 暴力检测函数
# ================================================================== #
def matchKeyPointsBF(featuresA, featuresB, method):bf = createMatcher(method, crossCheck=True)best_matches = bf.match(featuresA,featuresB)rawMatches = sorted(best_matches, key = lambda x:x.distance)print("Raw matches (Brute force):", len(rawMatches))return rawMatches
# ================================================================== #
# 使用knn检测函数
# ================================================================== #
def matchKeyPointsKNN(featuresA, featuresB, ratio, method):bf = createMatcher(method, crossCheck=False)rawMatches = bf.knnMatch(featuresA, featuresB, 2)# print("Raw matches (knn):", len(rawMatches))matches = []for m,n in rawMatches:if m.distance < n.distance * ratio:matches.append(m)# print(f"knn匹配的特征点数量:{len(matches)}")return matches
# ================================================================== #
#
# ================================================================== #
def createMatcher(method,crossCheck):"Create and return a Matcher Object"if method == 'sift' or method == 'surf':bf = cv2.BFMatcher(cv2.NORM_L2, crossCheck=crossCheck)elif method == 'orb' or method == 'brisk':bf = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck=crossCheck)return bf
# ================================================================== #
# 计算关键点的透视关系
# ================================================================== #
def getHomography(kpsA, kpsB, matches, reprojThresh):# convert the keypoints to numpy arrayskpsA = np.float32([kp.pt for kp in kpsA])kpsB = np.float32([kp.pt for kp in kpsB])if len(matches) > 4:# construct the two sets of pointsptsA = np.float32([kpsA[m.queryIdx] for m in matches])ptsB = np.float32([kpsB[m.trainIdx] for m in matches])# estimate the homography between the sets of points(H, status) = cv2.findHomography(ptsA, ptsB, cv2.RANSAC,reprojThresh)return (matches, H, status)else:return None
# ================================================================== #
# 去除图像黑边
# ================================================================== #
def cutBlack(pic):rows, cols = np.where(pic[:,:,0] !=0)min_row, max_row = min(rows), max(rows) +1min_col, max_col = min(cols), max(cols) +1return pic[min_row:max_row,min_col:max_col,:]
# ================================================================== #
# 调换
# ================================================================== #
def swap(a, b):return b,a
# ================================================================== #
# 主要的函数
# 合并两张图（合并多张图基于此函数）
# ================================================================== #
def handle(path1, path2, isShow = False):"""定义超参数"""feature_extractor = 'sift'feature_matching = 'knn'"""读取原始图像"""if isinstance(path2,str):imageA = cv2.imread(path2)imageA = cv2.cvtColor(imageA,cv2.COLOR_BGR2RGB)else:imageA = path2imageA_gray = cv2.cvtColor(imageA, cv2.COLOR_RGB2GRAY)if isinstance(path1,str):imageB = cv2.imread(path1)imageB = cv2.cvtColor(imageB,cv2.COLOR_BGR2RGB)else:imageB = path1t = np.size(imageB)imageB_gray = cv2.cvtColor(imageB, cv2.COLOR_RGB2GRAY)"""显示输入的两张图片"""if isShow:f = plt.figure(figsize=(10,4))f.add_subplot(1,2,1)plt.title("imageB")plt.imshow(imageB)plt.xticks([]),plt.yticks([])f.add_subplot(1,2,2)plt.title("imageA")plt.imshow(imageA)plt.xticks([]),plt.yticks([])"""提取两证图片的特征"""kpsA, featuresA = detectAndDescribe(imageA_gray, method=feature_extractor)kpsB, featuresB = detectAndDescribe(imageB_gray, method=feature_extractor)"""显示关键点"""if isShow:fig, (ax1,ax2) = plt.subplots(nrows=1, ncols=2, figsize=(10,4), constrained_layout=False)ax1.imshow(cv2.drawKeypoints(imageA_gray,kpsA,None,color=(0,255,0)))ax1.set_xlabel("(a)key point", fontsize=14)ax2.imshow(cv2.drawKeypoints(imageB_gray,kpsB,None,color=(0,255,0)))ax2.set_xlabel("(b)key point", fontsize=14)"""进行特征匹配"""if feature_matching == 'bf':matches = matchKeyPointsBF(featuresA, featuresB, method=feature_extractor)img3 = cv2.drawMatches(imageA,kpsA,imageB,kpsB,matches[:100],None,flags=cv2.DrawMatchesFlags_NOT_DRAW_SINGLE_POINTS)elif feature_matching == 'knn':matches = matchKeyPointsKNN(featuresA, featuresB, ratio=0.75, method=feature_extractor)if len(matches) < 10:return Noneimg3 = cv2.drawMatches(imageA,kpsA,imageB,kpsB,np.random.choice(matches,100),None,flags=cv2.DrawMatchesFlags_NOT_DRAW_SINGLE_POINTS)"""匹配的特征展示"""if isShow:fig = plt.figure(figsize=(10,4))plt.imshow(img3)plt.title("feature match")plt.axis('off')"""计算两张图的透视关系"""matchCount = len(matches)M = getHomography(kpsA, kpsB, matches, reprojThresh=4)if M is None:print("Error!")(matches, H, status) = M"""将图片A进行透视变换并检查图片位置"""result = cv2.warpPerspective(imageA, H, ((imageA.shape[1] + imageB.shape[1])*2, (imageA.shape[0] + imageB.shape[0]) * 2))resultAfterCut = cutBlack(result)# 检查图片位置if np.size(resultAfterCut) < np.size(imageA) * 0.95:print("图片位置不对,将自动调换")# 调换图片kpsA,kpsB = swap(kpsA, kpsB)imageA, imageB = swap(imageA, imageB)if feature_matching == 'bf':matches = matchKeyPointsBF(featuresB, featuresA, method=feature_extractor)elif feature_matching == 'knn':matches = matchKeyPointsKNN(featuresB, featuresA, ratio=0.75, method=feature_extractor)if len(matches) < 10:return NonematchCount = len(matches)M = getHomography(kpsA, kpsB, matches, reprojThresh=4)if M is None:print("Error!")(matches, H, status) = Mresult = cv2.warpPerspective(imageA, H, ((imageA.shape[1] + imageB.shape[1])*2, (imageA.shape[0] + imageB.shape[0]) * 2))"""合并图片"""# cv2.imshow("Perspective transformation", result)# cv2.waitKey(0)result[0:imageB.shape[0], 0:imageB.shape[1]] = np.maximum(imageB, result[0:imageB.shape[0], 0:imageB.shape[1]]) result = cutBlack(result)return result, matchCount
# ================================================================== #
# 合并多张图
# ================================================================== #
def handleMulti(*args, isShow = False):print(isShow)l = len(args)if isShow:row = math.ceil(l / 3)f = plt.figure(figsize=(10, 4))for i in range(l):f.add_subplot(row, 3, i + 1)plt.title(f"image({i+1})")plt.axis("off")plt.imshow(cv2.cvtColor(cv2.imread(args[i]), cv2.COLOR_BGR2RGB))assert(l > 1)isHandle = [0 for i in range(l - 1)]nowPic = args[0]args = args[1:]for j in range(l - 1):isHas = False # 在一轮中是否找到matchCountList = []resultList = []indexList = []for i in range(l - 1):if (isHandle[i] == 1):continueresult, matchCount = handle(nowPic, args[i])if not result is None:matchCountList.append(matchCount)resultList.append(result)indexList.append(i)isHas = Trueif not isHas: # 一轮找完都没有可以合并的return Noneelse:index = matchCountList.index(max(matchCountList))nowPic = resultList[index]isHandle[indexList[index]] = 1print(f"合并第{indexList[index] + 2}个")# cv2.imshow("temp", nowPic)# cv2.waitKey(0)return nowPic# ================================================================== #
# 主函数
# ================================================================== #
if __name__ == "__main__":"""处理两张图，可以打印特征点与对应关系"""result, _ = handle("./input/222.png", "./input/111.png", isShow=True)if not result is None:cv2.imshow("result", result[:, :, [2, 1, 0]])plt.show()cv2.waitKey(0)else:print("没有找到对应特征点,无法合并")exit()"""处理多张图，不可以打印特征点与对应关系"""# result = handleMulti(# "./input/migong (1).png",# "./input/migong (2).png",# "./input/migong (3).png",# isShow=True)# result = handleMulti("./input/111.png", "./input/222.png","./input/333.png", isShow=True)# result = handleMulti("./input/foto7A.jpg", "./input/foto7B.jpg") #合并的不好的图result = handleMulti("./input/intel_lab (1).png","./input/intel_lab (2).png","./input/intel_lab (3).png","./input/intel_lab (4).png","./input/intel_lab (5).png","./input/intel_lab (6).png",isShow=True)if not result is None:cv2.imshow("result", result[:, :, [2, 1, 0]])plt.show()cv2.waitKey(0)else:print("没有找到对应特征点,无法合并")
基于opencv的图像拼接相关推荐
- 基于OpenCV全景图像拼接
本文转载基于SIFT特征的全景图像拼接http://blog.csdn.net/masibuaa/article/details/9246493 主要分为以下几个步骤: (1) 读入两张图片并分别提取 ...
- 【Opencv探索】基于OpenCV的“图像拼接特效”（这效果很实用啊）
前言 家好!我是梨子同学! 希望大家多多支持我!哈哈 为了感谢每一个关注我的小可爱:
- 图像拼接和图像融合技术（基于Opencv）
图像拼接在实际的应用场景很广,比如无人机航拍,遥感图像等等,图像拼接是进一步做图像理解基础步骤,拼接效果的好坏直接影响接下来的工作,所以一个好的图像拼接算法非常重要. 再举一个身边的例子吧,你用你的手 ...
- OpenCV常用图像拼接方法(四)：基于Stitcher类拼接
OpenCV常用图像拼接方法将分为四个部分与大家共享,这里是第四种方法,至此四种常用方法介绍完毕. OpenCV的常用图像拼接方法(四):基于OpenCV Stitcher类的图像拼接,OpenCV版 ...
- 基于OpenCV＆ORB和特征匹配的双视频图像拼接（源码＆部署教程）
1.双视频拼接效果展示 2.视频演示 [项目分享]Python基于OpenCV&ORB和特征匹配的双视频图像拼接(源码&部署教程)_哔哩哔哩_bilibili 3.背景 随着汽车电子和 ...
- Python基于OpenCV＆ORB和特征匹配的双视频图像拼接（源码＆部署教程）
1.双视频拼接效果展示 2.视频演示 [项目分享]Python基于OpenCV&ORB和特征匹配的双视频图像拼接(源码&部署教程)_哔哩哔哩_bilibili 3.背景 随着汽车电子和 ...
- akaze特征匹配怎么去掉不合适的点_单应性矩阵应用基于特征的图像拼接
点击上方蓝字关注我们 微信公众号:OpenCV学堂 关注获取更多计算机视觉与深度学习知识 前言 前面写了一篇关于单应性矩阵的相关文章,结尾说到基于特征的图像拼接跟对象检测中单应性矩阵应用场景.得到很多 ...
- 利用C++中的opencv进行图像拼接
这篇文章依旧是记录采用C++复现图像拼接过程解决遇到的问题.因为自己没有学过C++,大学学的C考完试立马还给老师了,Python也是现学的,只会一点点MATLAB,所以遇到的问题和解决都很基础,目的是 ...
- SURF原理及基于OPENCV实现
写在前面: 六月,沉云湿雨.黄宁然--Time to say goodbye. 参考文献镇楼: [1]卜珂,基于SURF的图像配准与拼接技术研究 [2]曹君宇,基于SURF的图像拼接算法研究 [3]陈 ...
- 语义分割：基于openCV和深度学习（二）
语义分割:基于openCV和深度学习(二) Semantic segmentation in images with OpenCV 开始吧-打开segment.py归档并插入以下代码: Semanti ...
最新文章
- 面试官再问我如何保证 RocketMQ 不丢失消息,这回我笑了！
- 【原】移动端界面的适配
- [jobdu]调整数组顺序使奇数位于偶数前面
- 谨慎能捕千秋蝉（二）——CSRF
- RuntimeError: Found dtype Double but expected Float”
- 如何发现 Redis 热点 Key ，解决方案有哪些？
- SQL SERVER 如果判断text类型数据不为空
- HDU - 6153 A Secret(KMP的next数组性质/扩展KMP)
- 手机是怎么确定位置信息的？
- 手机发布日期为什么保密？罗永浩这么说...
- 服务器放在机柜_机架式服务器和塔式服务器有区别吗
- Mozilla正修复Firefox遗留8年的漏洞
- 给Fedora11安装五笔
- pg数据库生成随机时间_postgreSQL自动生成随机数值的实例
- 小度计算机笔记,开售告罄、口碑炸裂、高语音交互率的小度耳机全新升级语音笔记...
- 【Yolov5】训练yolov5模型并集成到安卓应用中（中）——模型转化
- html5 答题源码脚本,自动答题脚本教程及源码分享（无视分辨率）
- openssl-genras命令简单入门
- 智能机器人与智能系统（大连理工大学庄严教授）——3.工业机器人
- 2021-05-23：pandas 新增sheet，不覆盖原来已经保存的sheet