鱼眼相机去畸变 Python/C++实现

陆辉东之前做了RealSense相机图像的远程传输，但是带畸变的图像

如果更进一步，可以一只fisheye带畸变，一只fisheye去畸变，这样放在QT界面里视觉感更好些

下午简单尝试了下，没有成功，还是要完成这项工作的

opencv 鱼眼矫正

【opencv】鱼眼图像畸变校正——透视变换

主要参照第一篇博客写了代码，但矫正后没什么效果

redwall@redwall-G3-3500:~$ rostopic list
/camera/accel/imu_info
/camera/accel/metadata
/camera/accel/sample
/camera/fisheye1/camera_info
/camera/fisheye1/image_raw

redwall@redwall-G3-3500:~$ rostopic echo /camera/fisheye1/camera_info
header: seq: 0stamp: secs: 1662554565nsecs: 716393948frame_id: "camera_fisheye1_optical_frame"
height: 800
width: 848
distortion_model: "equidistant"
D: [-0.008326118811964989, 0.04620290920138359, -0.04403631016612053, 0.00837636087089777]
K: [286.1809997558594, 0.0, 421.6383972167969, 0.0, 286.3576965332031, 403.9013977050781, 0.0, 0.0, 1.0]
R: [1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0]
P: [286.1809997558594, 0.0, 421.6383972167969, 0.0, 0.0, 286.3576965332031, 403.9013977050781, 0.0, 0.0, 0.0, 1.0, 0.0]
binning_x: 0
binning_y: 0
roi: x_offset: 0y_offset: 0height: 0width: 0do_rectify: False

可以看到关于鱼眼相机的信息，其中包含distortion_model以及一些畸变矩阵，可能会用到
do_rectify: False这里不知道是否和畸变校正相关

参考了下述GitHub包，已fork到个人仓库

https://github.com/HLearning/fisheye

主要学习了下面这段

def undistort(img_path,K,D,DIM,scale=0.6,imshow=False):img = cv2.imread(img_path)dim1 = img.shape[:2][::-1]  #dim1 is the dimension of input image to un-distortassert dim1[0]/dim1[1] == DIM[0]/DIM[1], "Image to undistort needs to have same aspect ratio as the ones used in calibration"if dim1[0]!=DIM[0]:img = cv2.resize(img,DIM,interpolation=cv2.INTER_AREA)Knew = K.copy()if scale:#change fovKnew[(0,1), (0,1)] = scale * Knew[(0,1), (0,1)]map1, map2 = cv2.fisheye.initUndistortRectifyMap(K, D, np.eye(3), Knew, DIM, cv2.CV_16SC2)undistorted_img = cv2.remap(img, map1, map2, interpolation=cv2.INTER_LINEAR, borderMode=cv2.BORDER_CONSTANT)if imshow:cv2.imshow("undistorted", undistorted_img)return undistorted_imgif __name__ == '__main__':'''DIM, K, D = get_K_and_D((6,9), '')'''DIM=(2560, 1920)K=np.array([[652.8609862494474, 0.0, 1262.1021584894233], [0.0, 653.1909758659955, 928.0871455436396], [0.0, 0.0, 1.0]])D=np.array([[-0.024092199861108887], [0.002745976275100771], [0.002545415522352827], [-0.0014366825722748522]])img = undistort('../imgs/pig.jpg',K,D,DIM)cv2.imwrite('../imgs/pig_checkerboard.jpg', img)

结合实验室相机参数做了一些简化，”/drag_video/scripts/fisheye_undistort.py”

#!/usr/bin/env python
import cv2
import numpy as np
import globdef undistort(img_path,K,D,DIM,scale=1.0,imshow=False):img = cv2.imread(img_path)dim1 = img.shape[:2][::-1]  #dim1 is the dimension of input image to un-distortassert dim1[0]/dim1[1] == DIM[0]/DIM[1], "Image to undistort needs to have same aspect ratio as the ones used in calibration"if dim1[0]!=DIM[0]:img = cv2.resize(img,DIM,interpolation=cv2.INTER_AREA)Knew = K.copy()if scale:#change fovKnew[(0,1), (0,1)] = scale * Knew[(0,1), (0,1)]map1, map2 = cv2.fisheye.initUndistortRectifyMap(K, D, np.eye(3), Knew, DIM, cv2.CV_16SC2)undistorted_img = cv2.remap(img, map1, map2, interpolation=cv2.INTER_LINEAR, borderMode=cv2.BORDER_CONSTANT)if imshow:cv2.imshow("undistorted", undistorted_img)return undistorted_imgif __name__ == '__main__':DIM=(848, 800)K=np.array([[286.1809997558594, 0.0, 421.6383972167969,], [0.0, 286.3576965332031, 403.9013977050781], [0.0, 0.0, 1.0]])D=np.array([[-0.008326118811964989], [0.04620290920138359], [-0.04403631016612053], [0.00837636087089777]])img = undistort('../image/raw.jpg',K,D,DIM)cv2.imwrite('../image/processed.jpg', img)

校正效果还可以

校正前

校正后

但这是基于Python实现的，我Python不太行，使用时还参考了一些博客

ROS——创建工作空间和功能包并成功运行一个基本python文件

linux python错误解决：import: not authorized `xx’ @ error/constitute.c/WriteImage/1028.

在Linux中Python文件最前面需要加上下面这行才能正常运行

#!/usr/bin/env python

用C++重新实现了一下

#include <iostream>
#include <opencv2/opencv.hpp>using namespace std;
using namespace cv;int main(int argc, char *argv[])
{Mat K = (cv::Mat_<double>(3, 3) << 286.1809997558594, 0.0, 421.6383972167969,0.0, 286.3576965332031, 403.9013977050781,0.0, 0.0, 1.0);Mat D = (cv::Mat_<double>(4, 1) << -0.008326118811964989, 0.04620290920138359, -0.04403631016612053, 0.00837636087089777);Mat raw_image = imread("/home/redwall/catkin_ws/src/push_video/image/raw.jpg");cout << raw_image.cols  << " " << raw_image.rows << endl;int width = 848;int height = 800;Mat map1, map2;Mat undistortImg;cv::Size imageSize(width, height);cv::fisheye::initUndistortRectifyMap(K, D, cv::Mat(), K, imageSize, CV_16SC2, map1, map2);// cout << "map1:" << map1 << "\nmap2:" << map2 << endl;remap(raw_image, undistortImg, map1, map2, cv::INTER_LINEAR, cv::BORDER_CONSTANT);imwrite("/home/redwall/catkin_ws/src/push_video/image/cpp_processed.jpg", undistortImg);return 0;
}

主要就是两个函数

void cv::fisheye::initUndistortRectifyMap(cv::InputArray K, cv::InputArray D, cv::InputArray R, cv::InputArray P, const cv::Size &size, int m1type, cv::OutputArray map1, cv::OutputArray map2)
Computes undistortion and rectification maps for image transform by cv::remap(). If D is empty zero distortion is used, if R or P is empty identity matrixes are used.参数:
K – Camera matrix
D – Input vector of distortion coefficients
R – Rectification transformation in the object space: 3x3 1-channel, or vector: 3x1/1x3 1-channel or 1x1 3-channel
P – New camera matrix (3x3) or new projection matrix (3x4)
size – Undistorted image size.
m1type – Type of the first output map that can be CV_32FC1 or CV_16SC2 . See convertMaps() for details.
map1 – The first output map.
map2 – The second output map.

void cv::remap(cv::InputArray src, cv::OutputArray dst, cv::InputArray map1, cv::InputArray map2, int interpolation, int borderMode = 0, const cv::Scalar &borderValue = cv::Scalar())
Applies a generic geometrical transformation to an image. The function remap transforms the source image using the specified map:参数:
src – Source image.
dst – Destination image. It has the same size as map1 and the same type as src .
map1 – The first map of either (x,y) points or just x values having the type CV_16SC2 , CV_32FC1, or CV_32FC2. See convertMaps for details on converting a floating point representation to fixed-point for speed.
map2 – The second map of y values having the type CV_16UC1, CV_32FC1, or none (empty map if map1 is (x,y) points), respectively.
interpolation – Interpolation method (see cv::InterpolationFlags). The method INTER_AREA is not supported by this function.
borderMode – Pixel extrapolation method (see cv::BorderTypes). When borderMode=BORDER_TRANSPARENT, it means that the pixels in the destination image that corresponds to the "outliers" in the source image are not modified by the function.
borderValue – Value used in case of a constant border. By default, it is 0.

再看一眼相机的参数信息，在前面有记录

redwall@redwall-G3-3500:~$ rostopic echo /camera/fisheye1/camera_info
header: seq: 0stamp: secs: 1662554565nsecs: 716393948frame_id: "camera_fisheye1_optical_frame"
height: 800
width: 848
distortion_model: "equidistant"
D: [-0.008326118811964989, 0.04620290920138359, -0.04403631016612053, 0.00837636087089777]
K: [286.1809997558594, 0.0, 421.6383972167969, 0.0, 286.3576965332031, 403.9013977050781, 0.0, 0.0, 1.0]
R: [1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0]
P: [286.1809997558594, 0.0, 421.6383972167969, 0.0, 0.0, 286.3576965332031, 403.9013977050781, 0.0, 0.0, 0.0, 1.0, 0.0]
binning_x: 0
binning_y: 0
roi: x_offset: 0y_offset: 0height: 0width: 0do_rectify: False

注意到initUndistortRectifyMap()函数中if R or P is empty identity matrixes are used，因此参数cv::Mat()就是相机参数中的R阵，校正后效果如下图，和Python实现的基本一致

单张照片能够完成畸变校正，那么视频流也就很好实现了，将代码写成一个函数嵌入进现有推拉流框架即可

注意initUndistortRectifyMap() 只需计算一次即可，不需要每次循环都计算，因此可以将initUndistortRectifyMap() 放在循环外面

一开始设计拉流端鱼眼图像校正时，将initUndistortRectifyMap()函数放在循环中重复执行，因此造成了很大的延迟，将initUndistortRectifyMap()函数放到循环外仅执行一次后，延迟很小，可以忽略不计

同样，一开始设计推流端鱼眼图像校正时也翻了上述错误，虽然运行和推拉流延迟均较小，但还是浪费了计算资源，处理同上

鱼眼相机去畸变 Python/C++实现相关推荐

AVM环视系统：鱼眼相机去畸变算法及实战
作者丨中投靓仔@知乎来源丨https://zhuanlan.zhihu.com/p/603296375 编辑丨自动驾驶与AI 点击进入->3D视觉工坊学习交流群前言在近一年的AVM算法开发 ...
相机去畸变软件OCamCalib的使用方法
OCamCalib软件的使用方法标签:相机标定转载请说明:http://blog.csdn.net/hust_sheng/article/details/77976619 此软件方便易用,且精度高 ...
鱼眼相机图像畸变校正
0.前言有关鱼眼相机成像模型相关知识,参考我的这篇文章.通过对鱼眼相机做内参标定,可以得到相机的内参和畸变参数.利用上述参数,可以对鱼眼相机获取的原始畸变图像做畸变校正. 1.畸变校正原理简单回顾 ...
VS2019+OpenCV4.5 鱼眼相机图像畸变矫正
一.鱼眼相机概述鱼眼镜头是定焦镜头中的一种视野范围很大的镜头,它视角范围通常大于等于180度.鱼眼相机虽然能获得较大的视角范围,但是其拍摄的图像存在较大的畸变,为了后续任务的需要,往往需要对原始图像 ...
一文讲透鱼眼相机畸变矫正，及目标检测项目应用
点击上方"3D视觉工坊",选择"星标" 干货第一时间送达 1 个人介绍大家好,我是潘大强. 目前博士毕业4年,主要从事智能安防行业. 之前也分享过AI从业的一 ...
从普通相机模型到鱼眼畸变模型--标定与去畸变
目录普通相机模型普通成像模型针孔模型与透镜组: 径向畸变切向畸变现代手机摄像头的基本组成内参矩阵畸变模型张氏标定法鱼眼模型透镜组 r和f的进一步说明等距投影模型(OpenCV:: ...
多摄像机标定和去畸变
Table of Contents 1.kalibr多摄像机标定 1.1 系统安装,环境配置:实测Ubuntu 16.04 1.2 多摄像机标定 2.OpenCV双目标定 3.Matlab多摄像机标定 ...
slam去畸变（径向畸变）
目录畸变径向畸变切向畸变去畸变原理代码参考畸变产生原因:透镜不能完全满足针孔模型假设径向畸变畸变程度都是从中心开始,用一个半径画圆的话,半径越大,圆周上的畸变程度也越大.这个就是由 ...
鱼眼相机相关知识及其标定矫正
首先列一下参考资料: Fisheye camera model 鱼眼相机标定以及OpenCV实现鱼眼相机畸变矫正的汇总链接鱼眼相机成像模型鱼眼相机矫正原理鱼眼相机畸变矫正的总结鱼眼镜头的选型 ...

鱼眼相机去畸变 Python/C++实现

鱼眼相机去畸变 Python/C++实现相关推荐

最新文章

热门文章