 本文是用python实现上位机开发，使机械臂与视觉结合进行拾取物体。这个过程对我来说是不容易的，因此我是分步实现局部功能，再结合，这样难度会降低。我把整个过程分成了界面，相机，图像处理，标定四个小部分。机械臂用的开塔米罗机械臂，相机是海康相机。

#准备

在编写代码之前，必须要做的就是通读SDK文件，了解它在python中是用何代码进行编写二次开发的。读SDK文件一定要仔细，不然在编写程序时，一个很小的问题能让你搞半天。机械臂连接的话一定要看是否需要通讯协议，我使用的这款不需要。

此外，为更好了解机械臂如何实现，最好先用机械臂自带软件进行操作，熟悉其运动模式和过程。

#界面

我是用tk进行上位机开发。界面设计首先设计好按键位置，例如复位按钮宽度为10，高度为1，执行的命令是home()，只需要在def home()编写对应的代码就能通过复位按钮进行调用。机械臂是有限位的，运动范围是有限的的，使用的机械臂在每次使用前都必须先进行复位操作，且每动作都要“休息”一下，因而需要time.sleep()，定义的步长movelength为5，即每按一次按钮，机械臂移动5个单位距离，都是这个代码只是大概的限位了，各个关节直接的限位我并没有定义。

from tkinter import *
from wlkata_mirobot import WlkataMirobot
import time
arm = WlkataMirobot()
arm.home()
arm.set_speed(2000)#界面控制类
class Window:def __init__(self, win, ww, wh):self.win = winself.win.geometry("%dx%d+%d+%d" % (ww, wh, 50, 50))  # 界面启动时的初始位置self.win.title("上位机")self.btn_home = Button(self.win, text='复位', width=10, height=1, command=self.home)self.btn_home.place(x=350, y=100)self.btn_go_to_zero = Button(self.win, text='回零', width=10, height=1, command=self.go_to_zero)self.btn_go_to_zero.place(x=500, y=100)self.btn_gripper_stop = Button(self.win, text='打开爪夹', width=10, height=1, command=self.gripper_open)self.btn_gripper_stop.place(x=350, y=200)self.btn_clear_close = Button(self.win, text='关闭爪夹', width=10, height=1,command=self.gripper_close)self.btn_clear_close.place(x=500, y=200)self.bj1a = Button(self.win, text='x+', width=10, height=1, command=self.load_bj1a)self.bj1a.place(x=25, y=300)self.bj1d = Button(self.win, text='x-', width=10, height=1, command=self.load_bj1d)self.bj1d.place(x=25, y=400)self.bj2a = Button(self.win, text='y+', width=10, height=1, command=self.load_bj2a)self.bj2a.place(x=125, y=300)self.bj2d = Button(self.win, text='y-', width=10, height=1, command=self.load_bj2d)self.bj2d.place(x=125, y=400)self.bj3a = Button(self.win, text='z+', width=10, height=1, command=self.load_bj3a)self.bj3a.place(x=225, y=300)self.bj3d = Button(self.win, text='z-', width=10, height=1, command=self.load_bj3d)self.bj3d.place(x=225, y=400)self.bj4a = Button(self.win, text='a+', width=10, height=1, command=self.load_bj4a)self.bj4a.place(x=325, y=300)self.bj4d = Button(self.win, text='a-', width=10, height=1, command=self.load_bj4d)self.bj4d.place(x=325, y=400)self.bj5a = Button(self.win, text='b+', width=10, height=1, command=self.load_bj5a)self.bj5a.place(x=425, y=300)self.bj5d = Button(self.win, text='b-', width=10, height=1, command=self.load_bj5d)self.bj5d.place(x=425, y=400)self.bj6a = Button(self.win, text='c+', width=10, height=1, command=self.load_bj6a)self.bj6a.place(x=525, y=300)self.bj6d = Button(self.win, text='c-', width=10, height=1, command=self.load_bj6d)self.bj6d.place(x=525, y=400)self.can_pred = Canvas(self.win, width=450, height=40, bg='white', relief='solid', borderwidth=1)self.can_pred.place(x=25, y=460)self.movelenght = 20.0def home(self):arm.home()print('复位')def go_to_zero(self):arm.go_to_zero()print('回零')def gripper_open(self):arm.gripper_open()print('打开爪夹')def gripper_close(self):arm.gripper_close()print('关闭爪夹')def load_bj1a(self):aa = arm.pose.xtime.sleep(1)bb = aa + self.movelenghtif 125 <= bb<=275:print("运动前:%s 运动后:%s" % (aa, bb))else:print("超过允许限位")time.sleep(1)arm.set_tool_pose(bb, arm.pose.y, arm.pose.z, arm.pose.roll, arm.pose.pitch, arm.pose.yaw)time.sleep(1)def load_bj1d(self):aa = arm.pose.xtime.sleep(1)bb = aa - self.movelenghtif 125 <= bb<=275:print("运动前:%s 运动后:%s" % (aa, bb))else:print("超过允许限位")arm.set_tool_pose(bb,arm.pose.y, arm.pose.z, arm.pose.roll, arm.pose.pitch, arm.pose.yaw)def load_bj2a(self):aa = arm.pose.ytime.sleep(1)bb = aa + self.movelenghtif -195 <= bb <= 190:print("运动前:%s 运动后:%s" % (aa, bb))else:print("超过允许限位")arm.set_tool_pose(arm.pose.x, bb, arm.pose.z, arm.pose.roll, arm.pose.pitch, arm.pose.yaw)def load_bj2d(self):aa = arm.pose.ytime.sleep(1)bb = aa - self.movelenghtif -195 <= bb <= 190:print("运动前:%s 运动后:%s" % (aa, bb))else:print("超过允许限位")arm.set_tool_pose(arm.pose.x, bb, arm.pose.z, arm.pose.roll, arm.pose.pitch, arm.pose.yaw)def load_bj3a(self):aa = arm.pose.ztime.sleep(1)bb = aa + self.movelenghtif 55 <= bb <= 315:print("运动前:%s 运动后:%s" % (aa, bb))else:print("超过允许限位")arm.set_tool_pose(arm.pose.x, arm.pose.y, bb, arm.pose.roll, arm.pose.pitch, arm.pose.yaw)def load_bj3d(self):aa = arm.pose.ztime.sleep(1)bb = aa - self.movelenghtif 55 <= bb <= 315:print("运动前:%s 运动后:%s" % (aa, bb))else:print("超过允许限位")arm.set_tool_pose(arm.pose.x, arm.pose.y, bb, arm.pose.roll, arm.pose.pitch, arm.pose.yaw)def load_bj4a(self):aa = arm.pose.rolltime.sleep(1)bb = aa + self.movelenghtprint("运动前:%s 运动后:%s" % (aa, bb))arm.set_tool_pose(arm.pose.x, arm.pose.y, arm.pose.z, bb, arm.pose.pitch, arm.pose.yaw)def load_bj4d(self):aa = arm.pose.rolltime.sleep(1)bb = aa - self.movelenghtprint("运动前:%s 运动后:%s" % (aa, bb))arm.set_tool_pose(arm.pose.x, arm.pose.y, arm.pose.z, bb, arm.pose.pitch, arm.pose.yaw)def load_bj5a(self):aa = arm.pose.pitchtime.sleep(1)bb = aa + self.movelenghtif -4855 <= bb <=4720:print("运动前:%s 运动后:%s" % (aa, bb))else:print("超过允许限位")arm.set_tool_pose(arm.pose.x, arm.pose.y, arm.pose.z, arm.pose.roll, bb, arm.pose.yaw)def load_bj5d(self):aa = arm.pose.pitchtime.sleep(1)bb = aa - self.movelenghtif -4855 <= bb < 4720:print("运动前:%s 运动后:%s" % (aa, bb))else:print("超过允许限位")arm.set_tool_pose(arm.pose.x, arm.pose.y, arm.pose.z, arm.pose.roll, bb, arm.pose.yaw)def load_bj6a(self):aa = arm.pose.yawbb = aa + self.movelenghtprint("运动前:%s 运动后:%s" % (aa, bb))arm.set_tool_pose(arm.pose.x, arm.pose.y, arm.pose.z, arm.pose.roll, arm.pose.pitch, bb)def load_bj6d(self):aa = arm.pose.yawbb = aa - self.movelenghtprint("运动前:%s 运动后:%s" % (aa, bb))arm.set_tool_pose(arm.pose.x, arm.pose.y, arm.pose.z, arm.pose.roll, arm.pose.pitch, bb)#主函数
if __name__ == '__main__':win = Tk()ww =700wh = 510Window(win, ww, wh)win.mainloop()

机械臂的这是运行后的界面，x,y,z,a,b,c对应机械臂的坐标控制模式的六个维度

#相机

sys.path.append("./MvImport")添加相机路径，我的在c盘，直接这样添加报错了，所以我在File-settings-project:ttyuyin.py-projectstructure中add路径MvImport，这样就不会报错了（也可以试下添加完整的路径）。在运行这一部分代码之前，先用海康相机自带软件进行连接，改好IP地址。

def Init_Cam()：相机初始化，枚举子网内指定的传输协议对应的所有设备，具体代码含义看相机sdk使用手册获取。通过设备型号来查找可用的相机，用于后续的相机连接。

这是图像采集显示的流程图，代码SDK使用手册上有，可以参考。

from tkinter import *
import numpy as np
import cv2
import threading
from PIL import Image, ImageTk
import syssys.path.append("./MvImport")
from MvCameraControl_class import *class Window:# 初始化界面def __init__(self, win, ww, wh):self.win = winself.ww = wwself.wh = whself.win.geometry("%dx%d+%d+%d" % (ww, wh, 50, 50))  # 界面启动时的初始位置self.win.title("实时图像采集")self.vidLabel = Label(win, anchor=NW)self.vidLabel.pack(expand=YES, fill=BOTH)self.vidLabel2 = Label(win, anchor=NW)self.vidLabel2.pack(expand=YES, fill=BOTH)self.yunxing = Button(self.win, text='运行相机', width=10, height=1, command=self.load_yunxing)self.yunxing.place(x=565, y=185)self.close_cam = Button(self.win, text='释放相机', width=10, height=1, command=self.close_cam)self.close_cam.place(x=765, y=185)self.close_cam = Button(self.win, text='图像处理', width=10, height=1, command=self.deal_pic_but)self.close_cam.place(x=565, y=285)self.can_pred1_x = Canvas(self.win, width=150, height=40, bg='white', relief='solid', borderwidth=1)self.can_pred1_x.place(x=535, y=385)self.can_pred1_y = Canvas(self.win, width=150, height=40, bg='white', relief='solid', borderwidth=1)self.can_pred1_y.place(x=745, y=385)self.can_pred2_x = Canvas(self.win, width=150, height=40, bg='white', relief='solid', borderwidth=1)self.can_pred2_x.place(x=535, y=485)self.can_pred2_y = Canvas(self.win, width=150, height=40, bg='white', relief='solid', borderwidth=1)self.can_pred2_y.place(x=745, y=485)self.can_pred3_x = Canvas(self.win, width=150, height=40, bg='white', relief='solid', borderwidth=1)self.can_pred3_x.place(x=535, y=585)self.can_pred3_y = Canvas(self.win, width=150, height=40, bg='white', relief='solid', borderwidth=1)self.can_pred3_y.place(x=745, y=585)self.BOOL = Trueself.cam = MvCamera()self.nConnectionNum = 0self.g_bExit = Falseself.g_bdeal_pic = Falseself.i_iIndex_deal = 0# 初始化相机def Init_Cam(self):deviceList = MV_CC_DEVICE_INFO_LIST()tlayerType = MV_GIGE_DEVICE | MV_USB_DEVICE# 枚举设备ret = MvCamera.MV_CC_EnumDevices(tlayerType, deviceList)if ret != 0:print("enum devices fail! ret[0x%x]" % ret)sys.exit()#未找到相机设备则退出if deviceList.nDeviceNum == 0:print("find no device!")sys.exit()print("Find %d devices!" % deviceList.nDeviceNum)# 找到则循环遍历，获取相机设备信息并打印for i in range(0, deviceList.nDeviceNum):mvcc_dev_info = cast(deviceList.pDeviceInfo[i], POINTER(MV_CC_DEVICE_INFO)).contentsif mvcc_dev_info.nTLayerType == MV_GIGE_DEVICE:print("\ngige device: [%d]" % i)strModeName = ""for per in mvcc_dev_info.SpecialInfo.stGigEInfo.chModelName:strModeName = strModeName + chr(per)print("device model name: %s" % strModeName)nip1 = ((mvcc_dev_info.SpecialInfo.stGigEInfo.nCurrentIp & 0xff000000) >> 24)nip2 = ((mvcc_dev_info.SpecialInfo.stGigEInfo.nCurrentIp & 0x00ff0000) >> 16)nip3 = ((mvcc_dev_info.SpecialInfo.stGigEInfo.nCurrentIp & 0x0000ff00) >> 8)nip4 = (mvcc_dev_info.SpecialInfo.stGigEInfo.nCurrentIp & 0x000000ff)print("current ip: %d.%d.%d.%d\n" % (nip1, nip2, nip3, nip4))elif mvcc_dev_info.nTLayerType == MV_USB_DEVICE:print("\nu3v device: [%d]" % i)strModeName = ""for per in mvcc_dev_info.SpecialInfo.stUsb3VInfo.chModelName:if per == 0:breakstrModeName = strModeName + chr(per)print("device model name: %s" % strModeName)strSerialNumber = ""for per in mvcc_dev_info.SpecialInfo.stUsb3VInfo.chSerialNumber:if per == 0:breakstrSerialNumber = strSerialNumber + chr(per)print("user serial number: %s" % strSerialNumber)# 若相机设备被占用无法连接退出if int(self.nConnectionNum) >= deviceList.nDeviceNum:print("intput error!")sys.exit()# 选择设备stDeviceList = cast(deviceList.pDeviceInfo[int(self.nConnectionNum)], POINTER(MV_CC_DEVICE_INFO)).contentsret = self.cam.MV_CC_CreateHandle(stDeviceList)if ret != 0:print("create handle fail! ret[0x%x]" % ret)sys.exit()# 打开设备ret = self.cam.MV_CC_OpenDevice(MV_ACCESS_Exclusive, 0)if ret != 0:print("open device fail! ret[0x%x]" % ret)sys.exit()# 探测网络最佳包大小if stDeviceList.nTLayerType == MV_GIGE_DEVICE:nPacketSize = self.cam.MV_CC_GetOptimalPacketSize()if int(nPacketSize) > 0:ret = self.cam.MV_CC_SetIntValue("GevSCPSPacketSize", nPacketSize)if ret != 0:print("Warning: Set Packet Size fail! ret[0x%x]" % ret)else:print("Warning: Get Packet Size fail! ret[0x%x]" % nPacketSize)# 设置触发模式为offret = self.cam.MV_CC_SetEnumValue("TriggerMode", MV_TRIGGER_MODE_OFF)if ret != 0:print("set trigger mode fail! ret[0x%x]" % ret)sys.exit()# 获取数据包大小stParam = MVCC_INTVALUE()memset(byref(stParam), 0, sizeof(MVCC_INTVALUE))ret = self.cam.MV_CC_GetIntValue("PayloadSize", stParam)if ret != 0:print("get payload size fail! ret[0x%x]" % ret)sys.exit()nPayloadSize = stParam.nCurValuereturn nPayloadSize# 相机开始采集图像def Start(self):ret = self.cam.MV_CC_StartGrabbing()if ret != 0:print("start grabbing fail! ret[0x%x]" % ret)sys.exit()# 判读图像格式是彩色还是黑白def IsImageColor(self, enType):dates = {PixelType_Gvsp_RGB8_Packed: 'color',PixelType_Gvsp_BGR8_Packed: 'color',PixelType_Gvsp_YUV422_Packed: 'color',PixelType_Gvsp_YUV422_YUYV_Packed: 'color',PixelType_Gvsp_BayerGR8: 'color',PixelType_Gvsp_BayerRG8: 'color',PixelType_Gvsp_BayerGB8: 'color',PixelType_Gvsp_BayerBG8: 'color',PixelType_Gvsp_BayerGB10: 'color',PixelType_Gvsp_BayerGB10_Packed: 'color',PixelType_Gvsp_BayerBG10: 'color',PixelType_Gvsp_BayerBG10_Packed: 'color',PixelType_Gvsp_BayerRG10: 'color',PixelType_Gvsp_BayerRG10_Packed: 'color',PixelType_Gvsp_BayerGR10: 'color',PixelType_Gvsp_BayerGR10_Packed: 'color',PixelType_Gvsp_BayerGB12: 'color',PixelType_Gvsp_BayerGB12_Packed: 'color',PixelType_Gvsp_BayerBG12: 'color',PixelType_Gvsp_BayerBG12_Packed: 'color',PixelType_Gvsp_BayerRG12: 'color',PixelType_Gvsp_BayerRG12_Packed: 'color',PixelType_Gvsp_BayerGR12: 'color',PixelType_Gvsp_BayerGR12_Packed: 'color',PixelType_Gvsp_Mono8: 'mono',PixelType_Gvsp_Mono10: 'mono',PixelType_Gvsp_Mono10_Packed: 'mono',PixelType_Gvsp_Mono12: 'mono',PixelType_Gvsp_Mono12_Packed: 'mono'}return dates.get(enType, '未知')# 图像显示def Image_show(self, image):# 利用CV_BGR2GRAY将原图src转换为灰度图bgr2grayImgcvimage = cv2.cvtColor(image, cv2.COLOR_BGR2RGBA)pilImage = Image.fromarray(cvimage)pilImage = pilImage.resize((400, 400), Image.ANTIALIAS)# 将图片显示到界面tkImage = ImageTk.PhotoImage(image=pilImage)self.vidLabel.configure(image=tkImage)self.vidLabel.image = tkImageif self.g_bdeal_pic==True:self.g_bdeal_pic=Falsex,y=self.deal_pic(image)#得到正方体中心坐标#中心坐标显示到上位机界面if self.i_iIndex_deal==1:self.can_pred1_x.create_text(0, 0, text=str(x), anchor='nw', font=('黑体', 20))self.can_pred1_y.create_text(0, 0, text=str(y), anchor='nw', font=('黑体', 20))if self.i_iIndex_deal==2:self.can_pred2_x.create_text(0, 0, text=str(x), anchor='nw', font=('黑体', 20))self.can_pred2_y.create_text(0, 0, text=str(y), anchor='nw', font=('黑体', 20))if self.i_iIndex_deal==3:self.can_pred3_x.create_text(0, 0, text=str(x), anchor='nw', font=('黑体', 20))self.can_pred3_y.create_text(0, 0, text=str(y), anchor='nw', font=('黑体', 20))print(x)print(y)# 利用CV_BGR2GRAY将原图src转换为灰度图bgr2grayImgcvimage = cv2.cvtColor(image, cv2.COLOR_BGR2RGBA)pilImage = Image.fromarray(cvimage)pilImage = pilImage.resize((400, 400), Image.ANTIALIAS)# 将图片显示到界面tkImage = ImageTk.PhotoImage(image=pilImage)self.vidLabel2.configure(image=tkImage)self.vidLabel2.image = tkImagedef Work_thread(self, cam=0, pData=0, nDataSize=0):stFrameInfo = MV_FRAME_OUT_INFO_EX()memset(byref(stFrameInfo), 0, sizeof(stFrameInfo))img_buff = Nonewhile self.BOOL:ret = cam.MV_CC_GetOneFrameTimeout(pData, nDataSize, stFrameInfo, 1000)if ret == 0:stConvertParam = MV_CC_PIXEL_CONVERT_PARAM()memset(byref(stConvertParam), 0, sizeof(stConvertParam))if self.IsImageColor(stFrameInfo.enPixelType) == 'mono':stConvertParam.enDstPixelType = PixelType_Gvsp_Mono8nConvertSize = stFrameInfo.nWidth * stFrameInfo.nHeightelif self.IsImageColor(stFrameInfo.enPixelType) == 'color':stConvertParam.enDstPixelType = PixelType_Gvsp_BGR8_PackednConvertSize = stFrameInfo.nWidth * stFrameInfo.nHeight * 3else:print("not support!!!")if img_buff is None:img_buff = (c_ubyte * stFrameInfo.nFrameLen)()# ---stConvertParam.nWidth = stFrameInfo.nWidthstConvertParam.nHeight = stFrameInfo.nHeightstConvertParam.pSrcData = cast(pData, POINTER(c_ubyte))stConvertParam.nSrcDataLen = stFrameInfo.nFrameLenstConvertParam.enSrcPixelType = stFrameInfo.enPixelTypestConvertParam.pDstBuffer = (c_ubyte * nConvertSize)()stConvertParam.nDstBufferSize = nConvertSizeret = cam.MV_CC_ConvertPixelType(stConvertParam)if ret != 0:print("convert pixel fail! ret[0x%x]" % ret)del stConvertParam.pSrcDatasys.exit()else:# 黑白处理if self.IsImageColor(stFrameInfo.enPixelType) == 'mono':img_buff = (c_ubyte * stConvertParam.nDstLen)()memmove(byref(img_buff), stConvertParam.pDstBuffer, stConvertParam.nDstLen)img_buff = np.frombuffer(img_buff, count=int(stConvertParam.nDstLen), dtype=np.uint8)img_buff = img_buff.reshape((stFrameInfo.nHeight, stFrameInfo.nWidth))self.Image_show(image=img_buff)# 彩色处理if self.IsImageColor(stFrameInfo.enPixelType) == 'color':img_buff = (c_ubyte * stConvertParam.nDstLen)()memmove(byref(img_buff), stConvertParam.pDstBuffer, stConvertParam.nDstLen)img_buff = np.frombuffer(img_buff, count=int(stConvertParam.nDstBufferSize), dtype=np.uint8)img_buff = img_buff.reshape(stFrameInfo.nHeight, stFrameInfo.nWidth, 3)self.Image_show(image=img_buff)else:print("no data[0x%x]" % ret)if self.g_bExit == True:breakdef load_yunxing(self):# 相机初始化nPayloadSize = self.Init_Cam()self.Start()data_buf = (c_ubyte * nPayloadSize)()try:hThreadHandle = threading.Thread(target=self.Work_thread, args=(self.cam, byref(data_buf), nPayloadSize))hThreadHandle.start()except:print("error: unable to start thread")def close_cam(self):# 停止取流ret = self.cam.MV_CC_StopGrabbing()if ret != 0:print("stop grabbing fail! ret[0x%x]" % ret)sys.exit()# 关闭设备ret = self.cam.MV_CC_CloseDevice()if ret != 0:print("close deivce fail! ret[0x%x]" % ret)sys.exit()# 销毁句柄ret = self.cam.MV_CC_DestroyHandle()print("close deivce succ" )self.BOOL=False# 图像处理def deal_pic_but(self):self.i_iIndex_deal=self.i_iIndex_deal+1self.g_bdeal_pic=Truedef deal_pic(self,image):(R, G, B) = cv2.split(image)diff_RG = cv2.absdiff(R, G)diff_GB = cv2.absdiff(G, B)diff_RB = cv2.absdiff(R, B)mean_np = np.zeros(3, dtype=np.double)mean_np[0] = cv2.mean(diff_RG)[0]mean_np[1] = cv2.mean(diff_GB)[0]mean_np[2] = cv2.mean(diff_RB)[0]list_a_max_list = max(mean_np)max_index = np.argmax(mean_np)if int(max_index) == 0:ret, thresh1 = cv2.threshold(diff_RG, list_a_max_list + 20, 255,cv2.THRESH_BINARY)elif int(max_index) == 1:ret, thresh1 = cv2.threshold(diff_GB, list_a_max_list + 20, 255, cv2.THRESH_BINARY)elif int(max_index) == 2:ret, thresh1 = cv2.threshold(diff_RB, list_a_max_list + 20, 255, cv2.THRESH_BINARY)contours, hier = cv2.findContours(thresh1.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)# 筛选的尺寸min_size = 100max_size = 1800lengths = list()save_list = []for i in range(len(contours)):length = cv2.arcLength(contours[i], True)lengths.append(length)if (length < max_size and length > min_size):save_list.append(contours[i])new_contours = save_listfor c in new_contours:M = cv2.moments(c)x = int(M["m10"] / M["m00"])y = int(M["m01"] / M["m00"])cv2.circle(image, (x, y), 2, (0, 0, 255), -1)cv2.drawContours(image, c, -1, (0, 0, 255), 3)need_x = xneed_y = yreturn need_x, need_y
#主函数
if __name__ == '__main__':·win = Tk()ww = 1000wh = 820Window(win, ww, wh)win.mainloop()

#图像处理

图像处理是在相机代码的基础上添加的。首先，将图片转为转换为RGB模式，因为cv2是BGR模式；再将array转换成image，并把图片裁剪成400*400的大小，显示到界面上。然后，调用图像处理函数对原图进行处理，计算相机中正方体的中心坐标（正方体最上面一面），并将图像显示到上位机程序中，并显示中心点位置，此位置是在相机坐标中的数值。最后，将处理过后的图片显示处理，包括正方体轮廓及其中心点位置。

from tkinter import *
import numpy as np
import cv2
import threading
from PIL import Image, ImageTk
import syssys.path.append("./MvImport")
from MvCameraControl_class import *class Window:# 初始化界面def __init__(self, win, ww, wh):self.win = winself.ww = wwself.wh = whself.win.geometry("%dx%d+%d+%d" % (ww, wh, 50, 50))  # 界面启动时的初始位置self.win.title("实时图像采集")self.vidLabel = Label(win, anchor=NW)self.vidLabel.pack(expand=YES, fill=BOTH)self.vidLabel2 = Label(win, anchor=NW)self.vidLabel2.pack(expand=YES, fill=BOTH)self.yunxing = Button(self.win, text='运行相机', width=10, height=1, command=self.load_yunxing)self.yunxing.place(x=565, y=185)self.close_cam = Button(self.win, text='释放相机', width=10, height=1, command=self.close_cam)self.close_cam.place(x=765, y=185)self.close_cam = Button(self.win, text='图像处理', width=10, height=1, command=self.deal_pic_but)self.close_cam.place(x=565, y=285)self.can_pred1_x = Canvas(self.win, width=150, height=40, bg='white', relief='solid', borderwidth=1)self.can_pred1_x.place(x=535, y=385)self.can_pred1_y = Canvas(self.win, width=150, height=40, bg='white', relief='solid', borderwidth=1)self.can_pred1_y.place(x=745, y=385)self.can_pred2_x = Canvas(self.win, width=150, height=40, bg='white', relief='solid', borderwidth=1)self.can_pred2_x.place(x=535, y=485)self.can_pred2_y = Canvas(self.win, width=150, height=40, bg='white', relief='solid', borderwidth=1)self.can_pred2_y.place(x=745, y=485)self.can_pred3_x = Canvas(self.win, width=150, height=40, bg='white', relief='solid', borderwidth=1)self.can_pred3_x.place(x=535, y=585)self.can_pred3_y = Canvas(self.win, width=150, height=40, bg='white', relief='solid', borderwidth=1)self.can_pred3_y.place(x=745, y=585)self.BOOL = Trueself.cam = MvCamera()self.nConnectionNum = 0self.g_bExit = Falseself.g_bdeal_pic = Falseself.i_iIndex_deal = 0# 初始化相机def Init_Cam(self):deviceList = MV_CC_DEVICE_INFO_LIST()tlayerType = MV_GIGE_DEVICE | MV_USB_DEVICE# 枚举设备ret = MvCamera.MV_CC_EnumDevices(tlayerType, deviceList)if ret != 0:print("enum devices fail! ret[0x%x]" % ret)sys.exit()if deviceList.nDeviceNum == 0:print("find no device!")sys.exit()print("Find %d devices!" % deviceList.nDeviceNum)for i in range(0, deviceList.nDeviceNum):mvcc_dev_info = cast(deviceList.pDeviceInfo[i], POINTER(MV_CC_DEVICE_INFO)).contentsif mvcc_dev_info.nTLayerType == MV_GIGE_DEVICE:print("\ngige device: [%d]" % i)strModeName = ""for per in mvcc_dev_info.SpecialInfo.stGigEInfo.chModelName:strModeName = strModeName + chr(per)print("device model name: %s" % strModeName)nip1 = ((mvcc_dev_info.SpecialInfo.stGigEInfo.nCurrentIp & 0xff000000) >> 24)nip2 = ((mvcc_dev_info.SpecialInfo.stGigEInfo.nCurrentIp & 0x00ff0000) >> 16)nip3 = ((mvcc_dev_info.SpecialInfo.stGigEInfo.nCurrentIp & 0x0000ff00) >> 8)nip4 = (mvcc_dev_info.SpecialInfo.stGigEInfo.nCurrentIp & 0x000000ff)print("current ip: %d.%d.%d.%d\n" % (nip1, nip2, nip3, nip4))elif mvcc_dev_info.nTLayerType == MV_USB_DEVICE:print("\nu3v device: [%d]" % i)strModeName = ""for per in mvcc_dev_info.SpecialInfo.stUsb3VInfo.chModelName:if per == 0:breakstrModeName = strModeName + chr(per)print("device model name: %s" % strModeName)strSerialNumber = ""for per in mvcc_dev_info.SpecialInfo.stUsb3VInfo.chSerialNumber:if per == 0:breakstrSerialNumber = strSerialNumber + chr(per)print("user serial number: %s" % strSerialNumber)if int(self.nConnectionNum) >= deviceList.nDeviceNum:print("intput error!")sys.exit()# 选择设备stDeviceList = cast(deviceList.pDeviceInfo[int(self.nConnectionNum)], POINTER(MV_CC_DEVICE_INFO)).contentsret = self.cam.MV_CC_CreateHandle(stDeviceList)if ret != 0:print("create handle fail! ret[0x%x]" % ret)sys.exit()# 打开设备ret = self.cam.MV_CC_OpenDevice(MV_ACCESS_Exclusive, 0)if ret != 0:print("open device fail! ret[0x%x]" % ret)sys.exit()# 探测网络最佳包大小if stDeviceList.nTLayerType == MV_GIGE_DEVICE:nPacketSize = self.cam.MV_CC_GetOptimalPacketSize()if int(nPacketSize) > 0:ret = self.cam.MV_CC_SetIntValue("GevSCPSPacketSize", nPacketSize)if ret != 0:print("Warning: Set Packet Size fail! ret[0x%x]" % ret)else:print("Warning: Get Packet Size fail! ret[0x%x]" % nPacketSize)# 设置触发模式为offret = self.cam.MV_CC_SetEnumValue("TriggerMode", MV_TRIGGER_MODE_OFF)if ret != 0:print("set trigger mode fail! ret[0x%x]" % ret)sys.exit()# 获取数据包大小stParam = MVCC_INTVALUE()memset(byref(stParam), 0, sizeof(MVCC_INTVALUE))ret = self.cam.MV_CC_GetIntValue("PayloadSize", stParam)if ret != 0:print("get payload size fail! ret[0x%x]" % ret)sys.exit()nPayloadSize = stParam.nCurValuereturn nPayloadSize# 相机开始采集图像def Start(self):ret = self.cam.MV_CC_StartGrabbing()if ret != 0:print("start grabbing fail! ret[0x%x]" % ret)sys.exit()# 判读图像格式是彩色还是黑白def IsImageColor(self, enType):dates = {PixelType_Gvsp_RGB8_Packed: 'color',PixelType_Gvsp_BGR8_Packed: 'color',PixelType_Gvsp_YUV422_Packed: 'color',PixelType_Gvsp_YUV422_YUYV_Packed: 'color',PixelType_Gvsp_BayerGR8: 'color',PixelType_Gvsp_BayerRG8: 'color',PixelType_Gvsp_BayerGB8: 'color',PixelType_Gvsp_BayerBG8: 'color',PixelType_Gvsp_BayerGB10: 'color',PixelType_Gvsp_BayerGB10_Packed: 'color',PixelType_Gvsp_BayerBG10: 'color',PixelType_Gvsp_BayerBG10_Packed: 'color',PixelType_Gvsp_BayerRG10: 'color',PixelType_Gvsp_BayerRG10_Packed: 'color',PixelType_Gvsp_BayerGR10: 'color',PixelType_Gvsp_BayerGR10_Packed: 'color',PixelType_Gvsp_BayerGB12: 'color',PixelType_Gvsp_BayerGB12_Packed: 'color',PixelType_Gvsp_BayerBG12: 'color',PixelType_Gvsp_BayerBG12_Packed: 'color',PixelType_Gvsp_BayerRG12: 'color',PixelType_Gvsp_BayerRG12_Packed: 'color',PixelType_Gvsp_BayerGR12: 'color',PixelType_Gvsp_BayerGR12_Packed: 'color',PixelType_Gvsp_Mono8: 'mono',PixelType_Gvsp_Mono10: 'mono',PixelType_Gvsp_Mono10_Packed: 'mono',PixelType_Gvsp_Mono12: 'mono',PixelType_Gvsp_Mono12_Packed: 'mono'}return dates.get(enType, '未知')# 图像显示def Image_show(self, image):cvimage = cv2.cvtColor(image, cv2.COLOR_BGR2RGBA)   #将BGR转换为RGBApilImage = Image.fromarray(cvimage)                 #array转换成imagepilImage = pilImage.resize((400, 400), Image.ANTIALIAS)#修改图片尺寸为400*400# 将原图片显示到界面tkImage = ImageTk.PhotoImage(image=pilImage)self.vidLabel.configure(image=tkImage)              #配置样式self.vidLabel.image = tkImage#显示中心点坐标if self.g_bdeal_pic==True:self.g_bdeal_pic=Falsex,y=self.deal_pic(image)if self.i_iIndex_deal==1:self.can_pred1_x.create_text(0, 0, text=str(x), anchor='nw', font=('黑体', 20))self.can_pred1_y.create_text(0, 0, text=str(y), anchor='nw', font=('黑体', 20))if self.i_iIndex_deal==2:self.can_pred2_x.create_text(0, 0, text=str(x), anchor='nw', font=('黑体', 20))self.can_pred2_y.create_text(0, 0, text=str(y), anchor='nw', font=('黑体', 20))if self.i_iIndex_deal==3:self.can_pred3_x.create_text(0, 0, text=str(x), anchor='nw', font=('黑体', 20))self.can_pred3_y.create_text(0, 0, text=str(y), anchor='nw', font=('黑体', 20))print(x)print(y)cvimage = cv2.cvtColor(image, cv2.COLOR_BGR2RGBA)pilImage = Image.fromarray(cvimage)pilImage = pilImage.resize((400, 400), Image.ANTIALIAS)# 将处理后的图片显示到界面tkImage = ImageTk.PhotoImage(image=pilImage)self.vidLabel2.configure(image=tkImage)self.vidLabel2.image = tkImagedef Work_thread(self, cam=0, pData=0, nDataSize=0):stFrameInfo = MV_FRAME_OUT_INFO_EX()memset(byref(stFrameInfo), 0, sizeof(stFrameInfo))img_buff = Nonewhile self.BOOL:ret = cam.MV_CC_GetOneFrameTimeout(pData, nDataSize, stFrameInfo, 1000)if ret == 0:stConvertParam = MV_CC_PIXEL_CONVERT_PARAM()memset(byref(stConvertParam), 0, sizeof(stConvertParam))if self.IsImageColor(stFrameInfo.enPixelType) == 'mono':stConvertParam.enDstPixelType = PixelType_Gvsp_Mono8nConvertSize = stFrameInfo.nWidth * stFrameInfo.nHeightelif self.IsImageColor(stFrameInfo.enPixelType) == 'color':stConvertParam.enDstPixelType = PixelType_Gvsp_BGR8_PackednConvertSize = stFrameInfo.nWidth * stFrameInfo.nHeight * 3else:print("not support!!!")if img_buff is None:img_buff = (c_ubyte * stFrameInfo.nFrameLen)()# ---stConvertParam.nWidth = stFrameInfo.nWidthstConvertParam.nHeight = stFrameInfo.nHeightstConvertParam.pSrcData = cast(pData, POINTER(c_ubyte))stConvertParam.nSrcDataLen = stFrameInfo.nFrameLenstConvertParam.enSrcPixelType = stFrameInfo.enPixelTypestConvertParam.pDstBuffer = (c_ubyte * nConvertSize)()stConvertParam.nDstBufferSize = nConvertSizeret = cam.MV_CC_ConvertPixelType(stConvertParam)if ret != 0:print("convert pixel fail! ret[0x%x]" % ret)del stConvertParam.pSrcDatasys.exit()else:# 黑白处理if self.IsImageColor(stFrameInfo.enPixelType) == 'mono':img_buff = (c_ubyte * stConvertParam.nDstLen)()memmove(byref(img_buff), stConvertParam.pDstBuffer, stConvertParam.nDstLen)img_buff = np.frombuffer(img_buff, count=int(stConvertParam.nDstLen), dtype=np.uint8)img_buff = img_buff.reshape((stFrameInfo.nHeight, stFrameInfo.nWidth))self.Image_show(image=img_buff)# 彩色处理if self.IsImageColor(stFrameInfo.enPixelType) == 'color':img_buff = (c_ubyte * stConvertParam.nDstLen)()memmove(byref(img_buff), stConvertParam.pDstBuffer, stConvertParam.nDstLen)img_buff = np.frombuffer(img_buff, count=int(stConvertParam.nDstBufferSize), dtype=np.uint8)img_buff = img_buff.reshape(stFrameInfo.nHeight, stFrameInfo.nWidth, 3)self.Image_show(image=img_buff)else:print("no data[0x%x]" % ret)if self.g_bExit == True:breakdef load_yunxing(self):# 相机初始化nPayloadSize = self.Init_Cam()self.Start()data_buf = (c_ubyte * nPayloadSize)()try:hThreadHandle = threading.Thread(target=self.Work_thread, args=(self.cam, byref(data_buf), nPayloadSize))hThreadHandle.start()except:print("error: unable to start thread")def close_cam(self):# 停止取流ret = self.cam.MV_CC_StopGrabbing()if ret != 0:print("stop grabbing fail! ret[0x%x]" % ret)sys.exit()# 关闭设备ret = self.cam.MV_CC_CloseDevice()if ret != 0:print("close deivce fail! ret[0x%x]" % ret)sys.exit()# 销毁句柄ret = self.cam.MV_CC_DestroyHandle()print("close deivce succ" )self.BOOL=False# 图像处理def deal_pic_but(self):self.i_iIndex_deal=self.i_iIndex_deal+1self.g_bdeal_pic=Truedef deal_pic(self,image):(R, G, B) = cv2.split(image)diff_RG = cv2.absdiff(R, G)             #计算绝对差值diff_GB = cv2.absdiff(G, B)diff_RB = cv2.absdiff(R, B)mean_np = np.zeros(3, dtype=np.double)mean_np[0] = cv2.mean(diff_RG)[0]       #计算均值mean_np[1] = cv2.mean(diff_GB)[0]mean_np[2] = cv2.mean(diff_RB)[0]list_a_max_list = max(mean_np)max_index = np.argmax(mean_np)          #返回最大值if int(max_index) == 0:ret, thresh1 = cv2.threshold(diff_RG, list_a_max_list + 20, 255,cv2.THRESH_BINARY)                 #阈值处理灰度图像elif int(max_index) == 1:ret, thresh1 = cv2.threshold(diff_GB, list_a_max_list + 20, 255, cv2.THRESH_BINARY)elif int(max_index) == 2:ret, thresh1 = cv2.threshold(diff_RB, list_a_max_list + 20, 255, cv2.THRESH_BINARY)contours, hier = cv2.findContours(thresh1.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)#图像轮廓检测# 筛选的尺寸min_size = 100max_size = 1800lengths = list()save_list = []for i in range(len(contours)):length = cv2.arcLength(contours[i], True)lengths.append(length)if (length < max_size and length > min_size):save_list.append(contours[i])new_contours = save_listfor c in new_contours:M = cv2.moments(c)              #重心计算x = int(M["m10"] / M["m00"])y = int(M["m01"] / M["m00"])cv2.circle(image, (x, y), 2, (0, 0, 255), -1)cv2.drawContours(image, c, -1, (0, 0, 255), 3)need_x = xneed_y = yreturn need_x, need_y               #得到中心点位置（相机坐标）
#主函数
if __name__ == '__main__':win = Tk()ww = 1000wh = 820Window(win, ww, wh)win.mainloop()

运行结果：

#标定

此部分内容最主要的就是仿射矩阵的求取，从相机坐标到世界坐标的转换。先用图像处理得到中心点的坐标，固定好机械臂底座保持不动之后，用机械臂自带程库控制机械臂移动到中心点位置，记录下此时坐标控制模式下的坐标，如此反复三组数据（构成一个三角形），就可求出仿射矩阵。利用仿射矩阵的逆变换就可以在知道正方体位置的情况下，使机械臂移动到正方体中心点，完成智能拾物。z轴的位置大致是不变的，取一个平均值即可，为了使标定更为准确，末轴没有放置爪夹，若想更加完整实现抓取物体，可以设置一个偏置量（添加爪夹后，末端工具中点点不在末端中心），完成打开爪夹-下降-关闭爪夹-上升。

from tkinter import *
import numpy as np
import cv2
import syssys.path.append("./MvImport")
class Window:def __init__(self, win, ww, wh):#相机坐标self.point1_x =441self.point1_y =552self.point2_x =431self.point2_y =280self.point3_x =289self.point3_y =261#世界坐标（机械臂坐标）self.point1_X =14.6self.point1_Y =-171self.point2_X =-27.4self.point2_Y =-171self.point3_X =-27.4self.point3_Y =-191pts1 = np.float32([[self.point1_x, self.point1_y], [self.point2_x, self.point2_y], [self.point3_x, self.point3_y]])pts2 = np.float32([[self.point1_X, self.point1_Y], [self.point2_X, self.point2_Y], [self.point3_X, self.point3_Y]])self.M = cv2.getAffineTransform(pts1, pts2)#仿射变化，仿射矩阵为2*3print(self.M)x=np.dot(self.M, [523,681,1])               #仿射逆变换，得到坐标（x,y)print(x)
if __name__ == '__main__':win = Tk()ww =700wh = 510Window(win, ww, wh)win.mainloop()

完整代码

#头文件
from wlkata_mirobot import WlkataMirobot
import time
from tkinter import *
import numpy as np
import cv2
import threading
from PIL import Image, ImageTk
import sys
sys.path.append("./MvImport")
from MvCameraControl_class import *#界面控制类
class Window:def __init__(self, win, ww, wh):self.win = winself.win.geometry("%dx%d+%d+%d" % (ww, wh, 50, 50))  # 界面启动时的初始位置self.win.title("上位机")self.vidLabel = Label(win, anchor=NW)self.vidLabel.pack(expand=YES, fill=BOTH)self.vidLabel2 = Label(win, anchor=NW)self.vidLabel2.pack(expand=YES, fill=BOTH)self.btn_home = Button(self.win, text='复位', width=10, height=1, command=self.home)self.btn_home.place(x=800, y=100)self.btn_go_to_zero = Button(self.win, text='回零', width=10, height=1, command=self.go_to_zero)self.btn_go_to_zero.place(x=900, y=100)self.btn_gripper_stop = Button(self.win, text='打开爪夹', width=10, height=1, command=self.gripper_open)self.btn_gripper_stop.place(x=800, y=200)self.btn_clear_close = Button(self.win, text='关闭爪夹', width=10, height=1,command=self.gripper_close)self.btn_clear_close.place(x=900, y=200)self.bj1a = Button(self.win, text='x+', width=10, height=1, command=self.load_bj1a)self.bj1a.place(x=800, y=250)self.bj1d = Button(self.win, text='x-', width=10, height=1, command=self.load_bj1d)self.bj1d.place(x=900, y=250)self.bj2a = Button(self.win, text='y+', width=10, height=1, command=self.load_bj2a)self.bj2a.place(x=800, y=300)self.bj2d = Button(self.win, text='y-', width=10, height=1, command=self.load_bj2d)self.bj2d.place(x=900, y=300)self.bj3a = Button(self.win, text='z+', width=10, height=1, command=self.load_bj3a)self.bj3a.place(x=800, y=350)self.bj3d = Button(self.win, text='z-', width=10, height=1, command=self.load_bj3d)self.bj3d.place(x=900, y=350)self.bj4a = Button(self.win, text='a+', width=10, height=1, command=self.load_bj4a)self.bj4a.place(x=800, y=400)self.bj4d = Button(self.win, text='a-', width=10, height=1, command=self.load_bj4d)self.bj4d.place(x=900, y=400)self.bj5a = Button(self.win, text='b+', width=10, height=1, command=self.load_bj5a)self.bj5a.place(x=800, y=450)self.bj5d = Button(self.win, text='b-', width=10, height=1, command=self.load_bj5d)self.bj5d.place(x=900, y=450)self.bj6a = Button(self.win, text='c+', width=10, height=1, command=self.load_bj6a)self.bj6a.place(x=800, y=500)self.bj6d = Button(self.win, text='c-', width=10, height=1, command=self.load_bj6d)self.bj6d.place(x=900, y=500)self.yunxing = Button(self.win, text='运行相机', width=10, height=1, command=self.load_yunxing)self.yunxing.place(x=800, y=0)self.close_cam = Button(self.win, text='释放相机', width=10, height=1, command=self.close_cam)self.close_cam.place(x=900, y=0)self.close_cam = Button(self.win, text='图像处理', width=10, height=1, command=self.deal_pic_but)self.close_cam.place(x=570, y=5)self.can_pred1_x = Canvas(self.win, width=150, height=40, bg='white', relief='solid', borderwidth=1)self.can_pred1_x.place(x=450, y=40)self.can_pred1_y = Canvas(self.win, width=150, height=40, bg='white', relief='solid', borderwidth=1)self.can_pred1_y.place(x=600, y=40)self.can_pred2_x = Canvas(self.win, width=150, height=40, bg='white', relief='solid', borderwidth=1)self.can_pred2_x.place(x=450, y=90)self.can_pred2_y = Canvas(self.win, width=150, height=40, bg='white', relief='solid', borderwidth=1)self.can_pred2_y.place(x=600, y=90)self.can_pred3_x = Canvas(self.win, width=150, height=40, bg='white', relief='solid', borderwidth=1)self.can_pred3_x.place(x=450, y=140)self.can_pred3_y = Canvas(self.win, width=150, height=40, bg='white', relief='solid', borderwidth=1)self.can_pred3_y.place(x=600, y=140)self.btn_zhauqugongjian = Button(self.win, text='抓取工件', width=10, height=1, command=self.deal_getworkpiece)self.btn_zhauqugongjian.place(x=570, y=220)self.can_pred1_X = Canvas(self.win, width=150, height=40, bg='white', relief='solid', borderwidth=1)self.can_pred1_X.place(x=450, y=255)self.can_pred1_Y = Canvas(self.win, width=150, height=40, bg='white', relief='solid', borderwidth=1)self.can_pred1_Y.place(x=600, y=255)self.can_pred2_X = Canvas(self.win, width=150, height=40, bg='white', relief='solid', borderwidth=1)self.can_pred2_X.place(x=450, y=305)self.can_pred2_Y = Canvas(self.win, width=150, height=40, bg='white', relief='solid', borderwidth=1)self.can_pred2_Y.place(x=600, y=305)self.can_pred3_X = Canvas(self.win, width=150, height=40, bg='white', relief='solid', borderwidth=1)self.can_pred3_X.place(x=450, y=355)self.can_pred3_Y = Canvas(self.win, width=150, height=40, bg='white', relief='solid', borderwidth=1)self.can_pred3_Y.place(x=600, y=355)self.btn_teach.place(x=570, y=420)self.BOOL = Trueself.cam = MvCamera()self.nConnectionNum = 0self.g_bExit = Falseself.g_bdeal_pic = Falseself.g_bdeal_getworkpiece = Falseself.i_iIndex_deal = 0self.movelenght = 20.0self.i_iIndex_dealrobot = 0self.point1_x = 633self.point1_y = 646self.point2_x = 232self.point2_y = 612self.point3_x = 640self.point3_y = 158self.point1_X = 47.6self.point1_Y = -141self.point2_X = 51.6self.point2_Y = -195self.point3_X = -24.4self.point3_Y = -153pts1 = np.float32([[self.point1_x, self.point1_y], [self.point2_x, self.point2_y], [self.point3_x, self.point3_y]])pts2 = np.float32([[self.point1_X, self.point1_Y], [self.point2_X, self.point2_Y], [self.point3_X, self.point3_Y]])self.M = cv2.getAffineTransform(pts1, pts2)self.arm = WlkataMirobot()#界面def home(self):self.arm.home()print('复位')def go_to_zero(self):self.arm.go_to_zero()print('回零')def gripper_open(self):self.arm.gripper_open()print('打开爪夹')def gripper_close(self):self.arm.gripper_close()print('关闭爪夹')def load_bj1a(self):aa = self.arm.pose.xtime.sleep(1)bb = aa + self.movelenghtif 125 <= bb<=275:print("运动前:%s 运动后:%s" % (aa, bb))else:print("超过允许限位")time.sleep(1)self.arm.set_tool_pose(bb, self.arm.pose.y, self.arm.pose.z, self.arm.pose.roll, self.arm.pose.pitch, self.arm.pose.yaw)time.sleep(1)def load_bj1d(self):aa = self.arm.pose.xtime.sleep(1)bb = aa - self.movelenghtif 125 <= bb<=275:print("运动前:%s 运动后:%s" % (aa, bb))else:print("超过允许限位")self.arm.set_tool_pose(bb,self.arm.pose.y, self.arm.pose.z, self.arm.pose.roll, self.arm.pose.pitch, self.arm.pose.yaw)def load_bj2a(self):aa = self.arm.pose.ytime.sleep(1)bb = aa + self.movelenghtif -195 <= bb <= 190:print("运动前:%s 运动后:%s" % (aa, bb))else:print("超过允许限位")self.arm.set_tool_pose(self.arm.pose.x, bb, self.arm.pose.z, self.arm.pose.roll, self.arm.pose.pitch, self.arm.pose.yaw)def load_bj2d(self):aa = self.arm.pose.ytime.sleep(1)bb = aa - self.movelenghtif -195 <= bb <= 190:print("运动前:%s 运动后:%s" % (aa, bb))else:print("超过允许限位")self.arm.set_tool_pose(self.arm.pose.x, bb, self.arm.pose.z, self.arm.pose.roll, self.arm.pose.pitch, self.arm.pose.yaw)def load_bj3a(self):aa = self.arm.pose.ztime.sleep(1)bb = aa + self.movelenghtif 55 <= bb <= 315:print("运动前:%s 运动后:%s" % (aa, bb))else:print("超过允许限位")self.arm.set_tool_pose(self.arm.pose.x, self.arm.pose.y, bb, self.arm.pose.roll, self.arm.pose.pitch, self.arm.pose.yaw)def load_bj3d(self):aa = self.arm.pose.ztime.sleep(1)bb = aa - self.movelenghtif 55 <= bb <= 315:print("运动前:%s 运动后:%s" % (aa, bb))else:print("超过允许限位")self.arm.set_tool_pose(self.arm.pose.x, self.arm.pose.y, bb, self.arm.pose.roll, self.arm.pose.pitch, self.arm.pose.yaw)def load_bj4a(self):aa = self.arm.pose.rolltime.sleep(1)bb = aa + self.movelenghtprint("运动前:%s 运动后:%s" % (aa, bb))self.arm.set_tool_pose(self.arm.pose.x, self.arm.pose.y, self.arm.pose.z, bb, self.arm.pose.pitch, self.arm.pose.yaw)def load_bj4d(self):aa = self.arm.pose.rolltime.sleep(1)bb = aa - self.movelenghtprint("运动前:%s 运动后:%s" % (aa, bb))self.arm.set_tool_pose(self.arm.pose.x, self.arm.pose.y, self.arm.pose.z, bb, self.arm.pose.pitch, self.arm.pose.yaw)def load_bj5a(self):aa = self.arm.pose.pitchtime.sleep(1)bb = aa + self.movelenghtif -4855 <= bb <=4720:print("运动前:%s 运动后:%s" % (aa, bb))else:print("超过允许限位")self.arm.set_tool_pose(self.arm.pose.x, self.arm.pose.y, self.arm.pose.z, self.arm.pose.roll, bb, self.arm.pose.yaw)def load_bj5d(self):aa = self.arm.pose.pitchtime.sleep(1)bb = aa - self.movelenghtif -4855 <= bb < 4720:print("运动前:%s 运动后:%s" % (aa, bb))else:print("超过允许限位")self.arm.set_tool_pose(self.arm.pose.x, self.arm.pose.y, self.arm.pose.z, self.arm.pose.roll, bb, self.arm.pose.yaw)def load_bj6a(self):aa = self.arm.pose.yawbb = aa + self.movelenghtprint("运动前:%s 运动后:%s" % (aa, bb))self.arm.set_tool_pose(self.arm.pose.x, self.arm.pose.y, self.arm.pose.z, self.arm.pose.roll, self.arm.pose.pitch, bb)def load_bj6d(self):aa = self.arm.pose.yawbb = aa - self.movelenghtprint("运动前:%s 运动后:%s" % (aa, bb))self.arm.set_tool_pose(self.arm.pose.x, self.arm.pose.y, self.arm.pose.z, self.arm.pose.roll, self.arm.pose.pitch, bb)#相机def Init_Cam(self):deviceList = MV_CC_DEVICE_INFO_LIST()tlayerType = MV_GIGE_DEVICE | MV_USB_DEVICE# 枚举设备ret = MvCamera.MV_CC_EnumDevices(tlayerType, deviceList)if ret != 0:print("enum devices fail! ret[0x%x]" % ret)sys.exit()if deviceList.nDeviceNum == 0:print("find no device!")sys.exit()print("Find %d devices!" % deviceList.nDeviceNum)for i in range(0, deviceList.nDeviceNum):mvcc_dev_info = cast(deviceList.pDeviceInfo[i], POINTER(MV_CC_DEVICE_INFO)).contentsif mvcc_dev_info.nTLayerType == MV_GIGE_DEVICE:print("\ngige device: [%d]" % i)strModeName = ""for per in mvcc_dev_info.SpecialInfo.stGigEInfo.chModelName:strModeName = strModeName + chr(per)print("device model name: %s" % strModeName)nip1 = ((mvcc_dev_info.SpecialInfo.stGigEInfo.nCurrentIp & 0xff000000) >> 24)nip2 = ((mvcc_dev_info.SpecialInfo.stGigEInfo.nCurrentIp & 0x00ff0000) >> 16)nip3 = ((mvcc_dev_info.SpecialInfo.stGigEInfo.nCurrentIp & 0x0000ff00) >> 8)nip4 = (mvcc_dev_info.SpecialInfo.stGigEInfo.nCurrentIp & 0x000000ff)print("current ip: %d.%d.%d.%d\n" % (nip1, nip2, nip3, nip4))elif mvcc_dev_info.nTLayerType == MV_USB_DEVICE:print("\nu3v device: [%d]" % i)strModeName = ""for per in mvcc_dev_info.SpecialInfo.stUsb3VInfo.chModelName:if per == 0:breakstrModeName = strModeName + chr(per)print("device model name: %s" % strModeName)strSerialNumber = ""for per in mvcc_dev_info.SpecialInfo.stUsb3VInfo.chSerialNumber:if per == 0:breakstrSerialNumber = strSerialNumber + chr(per)print("user serial number: %s" % strSerialNumber)if int(self.nConnectionNum) >= deviceList.nDeviceNum:print("intput error!")sys.exit()# 选择设备stDeviceList = cast(deviceList.pDeviceInfo[int(self.nConnectionNum)],POINTER(MV_CC_DEVICE_INFO)).contentsret = self.cam.MV_CC_CreateHandle(stDeviceList)if ret != 0:print("create handle fail! ret[0x%x]" % ret)sys.exit()# 打开设备ret = self.cam.MV_CC_OpenDevice(MV_ACCESS_Exclusive, 0)if ret != 0:print("open device fail! ret[0x%x]" % ret)sys.exit()# 探测网络最佳包大小if stDeviceList.nTLayerType == MV_GIGE_DEVICE:nPacketSize = self.cam.MV_CC_GetOptimalPacketSize()if int(nPacketSize) > 0:ret = self.cam.MV_CC_SetIntValue("GevSCPSPacketSize", nPacketSize)if ret != 0:print("Warning: Set Packet Size fail! ret[0x%x]" % ret)else:print("Warning: Get Packet Size fail! ret[0x%x]" % nPacketSize)# 设置触发模式为offret = self.cam.MV_CC_SetEnumValue("TriggerMode", MV_TRIGGER_MODE_OFF)if ret != 0:print("set trigger mode fail! ret[0x%x]" % ret)sys.exit()# 获取数据包大小stParam = MVCC_INTVALUE()memset(byref(stParam), 0, sizeof(MVCC_INTVALUE))ret = self.cam.MV_CC_GetIntValue("PayloadSize", stParam)if ret != 0:print("get payload size fail! ret[0x%x]" % ret)sys.exit()nPayloadSize = stParam.nCurValuereturn nPayloadSize# 相机开始采集图像def Start(self):ret = self.cam.MV_CC_StartGrabbing()if ret != 0:print("start grabbing fail! ret[0x%x]" % ret)sys.exit()# 判读图像格式是彩色还是黑白def IsImageColor(self, enType):dates = {PixelType_Gvsp_RGB8_Packed: 'color',PixelType_Gvsp_BGR8_Packed: 'color',PixelType_Gvsp_YUV422_Packed: 'color',PixelType_Gvsp_YUV422_YUYV_Packed: 'color',PixelType_Gvsp_BayerGR8: 'color',PixelType_Gvsp_BayerRG8: 'color',PixelType_Gvsp_BayerGB8: 'color',PixelType_Gvsp_BayerBG8: 'color',PixelType_Gvsp_BayerGB10: 'color',PixelType_Gvsp_BayerGB10_Packed: 'color',PixelType_Gvsp_BayerBG10: 'color',PixelType_Gvsp_BayerBG10_Packed: 'color',PixelType_Gvsp_BayerRG10: 'color',PixelType_Gvsp_BayerRG10_Packed: 'color',PixelType_Gvsp_BayerGR10: 'color',PixelType_Gvsp_BayerGR10_Packed: 'color',PixelType_Gvsp_BayerGB12: 'color',PixelType_Gvsp_BayerGB12_Packed: 'color',PixelType_Gvsp_BayerBG12: 'color',PixelType_Gvsp_BayerBG12_Packed: 'color',PixelType_Gvsp_BayerRG12: 'color',PixelType_Gvsp_BayerRG12_Packed: 'color',PixelType_Gvsp_BayerGR12: 'color',PixelType_Gvsp_BayerGR12_Packed: 'color',PixelType_Gvsp_Mono8: 'mono',PixelType_Gvsp_Mono10: 'mono',PixelType_Gvsp_Mono10_Packed: 'mono',PixelType_Gvsp_Mono12: 'mono',PixelType_Gvsp_Mono12_Packed: 'mono'}return dates.get(enType, '未知')#图像显示函数def Image_show(self, image):cvimage = cv2.cvtColor(image, cv2.COLOR_BGR2RGBA)       # 利用CV_BGR2GRAY将原图src转换为灰度图bgr2grayImgpilImage = Image.fromarray(cvimage)                     # 实现array到image的转换pilImage = pilImage.resize((400, 400), Image.ANTIALIAS) # resize(InputArray src, OutputArray dst, Size dsize,tkImage = ImageTk.PhotoImage(image=pilImage)            # 将图片显示到界面self.vidLabel.configure(image=tkImage)self.vidLabel.image = tkImageif self.g_bdeal_getworkpiece == True:self.g_bdeal_getworkpiece = Falsex, y = self.deal_pic(image)c = np.array([[x], [y], [1]])dd = np.dot(self.M, c)print(dd[0])print(dd[1])self.arm.set_tool_pose(int(dd[0]), int(dd[1]), 33, 0, 0, 0)time.sleep(2)self.i_iIndex_dealrobot = self.i_iIndex_dealrobot + 1if self.i_iIndex_dealrobot == 1:self.can_pred1_X.create_text(0, 0, text=int(dd[0]), anchor='nw', font=('黑体', 20))self.can_pred1_Y.create_text(0, 0, text=int(dd[1]), anchor='nw', font=('黑体', 20))self.point1_X = int(dd[0])self.point1_Y = int(dd[1])if self.i_iIndex_dealrobot == 2:self.can_pred2_X.create_text(0, 0, text=int(dd[0]), anchor='nw', font=('黑体', 20))self.can_pred2_Y.create_text(0, 0, text=int(dd[1]), anchor='nw', font=('黑体', 20))self.point2_X = int(dd[0])self.point2_Y = int(dd[1])if self.i_iIndex_dealrobot == 3:self.can_pred3_X.create_text(0, 0, text=int(dd[0]), anchor='nw', font=('黑体', 20))self.can_pred3_Y.create_text(0, 0, text=int(dd[1]), anchor='nw', font=('黑体', 20))self.point3_X = int(dd[0])self.point3_Y = int(dd[1])if self.g_bdeal_pic==True:self.g_bdeal_pic=Falsex,y=self.deal_pic(image)if self.i_iIndex_deal==1:self.can_pred1_x.create_text(0, 0, text=str(x), anchor='nw', font=('黑体', 20))self.can_pred1_y.create_text(0, 0, text=str(y), anchor='nw', font=('黑体', 20))self.point1_x = xself.point1_y = yif self.i_iIndex_deal==2:self.can_pred2_x.create_text(0, 0, text=str(x), anchor='nw', font=('黑体', 20))self.can_pred2_y.create_text(0, 0, text=str(y), anchor='nw', font=('黑体', 20))self.point2_x = xself.point2_y = yif self.i_iIndex_deal==3:self.can_pred3_x.create_text(0, 0, text=str(x), anchor='nw', font=('黑体', 20))self.can_pred3_y.create_text(0, 0, text=str(y), anchor='nw', font=('黑体', 20))self.point3_x = xself.point3_y = y# 利用CV_BGR2GRAY将原图src转换为灰度图bgr2grayImgcvimage = cv2.cvtColor(image, cv2.COLOR_BGR2RGBA)pilImage = Image.fromarray(cvimage)pilImage = pilImage.resize((400, 400), Image.ANTIALIAS)# 将图片显示到界面tkImage = ImageTk.PhotoImage(image=pilImage)self.vidLabel2.configure(image=tkImage)self.vidLabel2.image = tkImagedef Work_thread(self, cam=0, pData=0, nDataSize=0):stFrameInfo = MV_FRAME_OUT_INFO_EX()memset(byref(stFrameInfo), 0, sizeof(stFrameInfo))img_buff = Nonewhile self.BOOL:ret = cam.MV_CC_GetOneFrameTimeout(pData, nDataSize, stFrameInfo, 1000)if ret == 0:stConvertParam = MV_CC_PIXEL_CONVERT_PARAM()memset(byref(stConvertParam), 0, sizeof(stConvertParam))if self.IsImageColor(stFrameInfo.enPixelType) == 'mono':stConvertParam.enDstPixelType = PixelType_Gvsp_Mono8nConvertSize = stFrameInfo.nWidth * stFrameInfo.nHeightelif self.IsImageColor(stFrameInfo.enPixelType) == 'color':stConvertParam.enDstPixelType = PixelType_Gvsp_BGR8_PackednConvertSize = stFrameInfo.nWidth * stFrameInfo.nHeight * 3else:print("not support!!!")if img_buff is None:img_buff = (c_ubyte * stFrameInfo.nFrameLen)()# ---stConvertParam.nWidth = stFrameInfo.nWidthstConvertParam.nHeight = stFrameInfo.nHeightstConvertParam.pSrcData = cast(pData, POINTER(c_ubyte))stConvertParam.nSrcDataLen = stFrameInfo.nFrameLenstConvertParam.enSrcPixelType = stFrameInfo.enPixelTypestConvertParam.pDstBuffer = (c_ubyte * nConvertSize)()stConvertParam.nDstBufferSize = nConvertSizeret = cam.MV_CC_ConvertPixelType(stConvertParam)if ret != 0:print("convert pixel fail! ret[0x%x]" % ret)del stConvertParam.pSrcDatasys.exit()else:# 黑白处理if self.IsImageColor(stFrameInfo.enPixelType) == 'mono':img_buff = (c_ubyte * stConvertParam.nDstLen)()memmove(byref(img_buff), stConvertParam.pDstBuffer, stConvertParam.nDstLen)img_buff=np.frombuffer(img_buff, count=int(stConvertParam.nDstLen), dtype=np.uint8)img_buff=img_buff.reshape((stFrameInfo.nHeight,stFrameInfo.nWidth))self.Image_show(image=img_buff)# 彩色处理if self.IsImageColor(stFrameInfo.enPixelType) == 'color':img_buff = (c_ubyte * stConvertParam.nDstLen)()memmove(byref(img_buff),stConvertParam.pDstBuffer, stConvertParam.nDstLen)img_buff=np.frombuffer(img_buff, count=int(stConvertParam.nDstBufferSize), dtype=np.uint8)img_buff = img_buff.reshape(stFrameInfo.nHeight, stFrameInfo.nWidth, 3)self.Image_show(image=img_buff)else:print("no data[0x%x]" % ret)if self.g_bExit == True:breakdef load_yunxing(self):nPayloadSize = self.Init_Cam()  # 相机初始化self.Start()data_buf = (c_ubyte * nPayloadSize)()try:hThreadHandle = threading.Thread(target=self.Work_thread, args=(self.cam, byref(data_buf), nPayloadSize))hThreadHandle.start()except:print("error: unable to start thread")def close_cam(self):# 停止取流ret = self.cam.MV_CC_StopGrabbing()if ret != 0:print("stop grabbing fail! ret[0x%x]" % ret)sys.exit()# 关闭设备ret = self.cam.MV_CC_CloseDevice()if ret != 0:print("close deivce fail! ret[0x%x]" % ret)sys.exit()# 销毁句柄ret = self.cam.MV_CC_DestroyHandle()print("close deivce succ")self.BOOL = False
# 图像处理def deal_pic_but(self):self.i_iIndex_deal=self.i_iIndex_deal+1self.g_bdeal_pic=Truedef deal_getworkpiece(self):print(self.M)self.g_bdeal_getworkpiece = Truedef deal_pic(self,image):(R, G, B) = cv2.split(image)diff_RG = cv2.absdiff(R, G)diff_GB = cv2.absdiff(G, B)diff_RB = cv2.absdiff(R, B)mean_np = np.zeros(3, dtype=np.double)mean_np[0] = cv2.mean(diff_RG)[0]mean_np[1] = cv2.mean(diff_GB)[0]mean_np[2] = cv2.mean(diff_RB)[0]list_a_max_list = max(mean_np)max_index = np.argmax(mean_np)if int(max_index) == 0:ret, thresh1 = cv2.threshold(diff_RG, list_a_max_list + 20, 255,cv2.THRESH_BINARY)elif int(max_index) == 1:ret, thresh1 = cv2.threshold(diff_GB, list_a_max_list + 20, 255, cv2.THRESH_BINARY)elif int(max_index) == 2:ret, thresh1 = cv2.threshold(diff_RB, list_a_max_list + 20, 255, cv2.THRESH_BINARY)contours, hier = cv2.findContours(thresh1.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)# 筛选的尺寸min_size = 100max_size = 1800lengths = list()save_list = []for i in range(len(contours)):length = cv2.arcLength(contours[i], True)lengths.append(length)if (length < max_size and length > min_size):save_list.append(contours[i])new_contours = save_listfor c in new_contours:# find bounding box coordinatesnew_c = c.reshape((c.shape[0], c.shape[2]))x, y = new_c.sum(axis=0) / new_c.shape[0]x = int(x)y = int(y)cv2.circle(image, (x, y), 2, (0, 0, 255), -1)cv2.drawContours(image, c, -1, (0, 0, 255), 3)need_x = xneed_y = yreturn need_x, need_y#主函数
if __name__ == '__main__':win = Tk()ww = 1200wh = 820Window(win, ww, wh)win.mainloop()

   max_index = np.argmax(mean_np)if int(max_index) == 0:ret, thresh1 = cv2.threshold(diff_RG, list_a_max_list + 20, 255,cv2.THRESH_BINARY)elif int(max_index) == 1:ret, thresh1 = cv2.threshold(diff_GB, list_a_max_list + 20, 255, cv2.THRESH_BINARY)elif int(max_index) == 2:ret, thresh1 = cv2.threshold(diff_RB, list_a_max_list + 20, 255, cv2.THRESH_BINARY)contours, hier = cv2.findContours(thresh1.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)# 筛选的尺寸min_size = 100max_size = 1800lengths = list()save_list = []for i in range(len(contours)):length = cv2.arcLength(contours[i], True)lengths.append(length)if (length < max_size and length > min_size):save_list.append(contours[i])new_contours = save_listfor c in new_contours:# find bounding box coordinatesnew_c = c.reshape((c.shape[0], c.shape[2]))x, y = new_c.sum(axis=0) / new_c.shape[0]x = int(x)y = int(y)cv2.circle(image, (x, y), 2, (0, 0, 255), -1)cv2.drawContours(image, c, -1, (0, 0, 255), 3)need_x = xneed_y = yreturn need_x, need_y

#主函数
if name == ‘main’:
win = Tk()
ww = 1200
wh = 820
Window(win, ww, wh)
win.mainloop()

![在这里插入图片描述](https://img-blog.csdnimg.cn/99a912eeee964dd9b6a08d0e29321723.png?x-oss-process=image/watermark,type_d3F5LXplbmhlaQ,shadow_50,text_Q1NETiBAcXFfNDI3MTkyMTc=,size_20,color_FFFFFF,t_70,g_se,x_16#pic_center)

python上位机实现机械臂拾物相关推荐

“Xilinx ZYNQ+TCP通信+Python上位机”实现实时视频传输系统
笔者在CSDN的第一篇万字长文,请多多支持. 本文是笔者的公众号 IC设计者笔记文章的转载.很多优质原创内容都会第一时间发布在公众号,欢迎关注公众号,一起交流学习.公众号后台回复"ZYNQ ...
python3中利用serial模块实现单片机与python上位机的通信（串口调试助手）
1.指标: python上位机向单片机发送字符,单片机如果收到的字符为'1',则点亮灯1,如果收到的字符为'2',则点亮灯2:单片机若接受到字符,读取字符后,向python上位机发送字符(1-& ...
python上位机开发实例-python上位机
广告关闭腾讯云双11爆品提前享,精选热门产品助力上云,云服务器首年88元起,买的越多返的越多,最高满返5000元! 若python上位机接受到的字符为"1',则print出ok,如果字符是 ...
python上位机开发经验总结01
文章目录 python上位机开发经验总结01 python变量与文件的处理全局变量与局部变量文件间的变量处理 threading模块使用经验管理线程定义线程 tkinter使用经验 tkint ...
用python做一个上位机串口通信_【教程】简易Python上位机之LED控制
电子爱好者应该不会对"上位机"这个词感到陌生,毕竟或多或少有过接触.但若是说到上位机的开发的话,大家就不一定熟悉了.很多电子爱好者完全没有接触过上位机的开发工作,他们真的没有相应的 ...
python led屏控制_【教程】简易Python上位机之LED控制
电子爱好者应该不会对"上位机"这个词感到陌生,毕竟或多或少有过接触.但若是说到上位机的开发的话,大家就不一定熟悉了.很多电子爱好者完全没有接触过上位机的开发工作,他们真的没有相应的 ...
C# / VB / LabVIEW / VC / Python 上位机使用S7-TCP协议与西门子PLC进行网口通信的教程 (Win/Linux)
现在越来越多的项目开始使用上位机了,在上位机实现数据存储.曲线绘制时,使用高级语言自行开发程序,比各种组态软件更加自由,更加强大.在进行上位机软件开发时,第一步就是要跟PLC取得通信,能够读写PLC内 ...
Python上位机软件图形界面实战——PyQt
转载:https://blog.csdn.net/qq_25939803/article/details/97894219 文章目录引言 1 环境配置 2 新建一个软件窗口 3 QtDesigner ...
python 上位机直接与西门子变频器建立通信
利用python直接读取西门子变频器参数,省去变频器与PLC的连接,代码如下. 硬件连接:一根网线连接电脑与变频器,变频器必须要有PN口. 电脑的网络需与变频器的网络ip在同一个网段.如变频器为192 ...
python上位机界面设计_用Python写界面--上位机开发
Python真的可以说是无所不能,上到人工智能.图像识别.下到控制电机.爬虫.数据处理,前不久发现Python还可以做界面,虽然比较丑,但是还是可以一试. Python内置图形界面库--Tkinter ...

python上位机实现机械臂拾物

#准备

#界面

#相机

#图像处理

#标定

完整代码

python上位机实现机械臂拾物相关推荐

最新文章

热门文章