opencv和mediapipe实现手势识别

本篇文章只是手势识别的一个demo，想要识别的精度更高，还需要添加其他的约束条件，这里只是根据每个手指关键点和手掌根部的距离来判断手指是伸展开还是弯曲的。关于mediapi pe的简介，可以去看官网：Home - mediapipe，官网有现成的demo程序，直接拷贝应用就可以实现手掌21个关键点的识别，这21个关键点的分布如下：

而且，检测的实时性也非常的不错：

当然，mediapipe不止可以检测手势，面部检测，姿态检测都可以：

下面说下这个手势识别的demo的大体思路：

首先，要import必要的库和导入必要的函数方法：

import cv2 as cv
import numpy as np
import mediapipe as mp
from numpy import linalg#手部检测函数
mpHands = mp.solutions.hands
hands = mpHands.Hands()#绘制关键点和连接线函数
mpDraw = mp.solutions.drawing_utils
handLmsStyle = mpDraw.DrawingSpec(color=(0, 0, 255), thickness=int(5))
handConStyle = mpDraw.DrawingSpec(color=(0, 255, 0), thickness=int(10))

其中，handLmsStyle和handConStyle分别是关键点和连接线的特征，包括颜色和关键点（连接线）的宽度。

如果画面中有手，就可以通过如下函数将关键点和连接线表示出来

if result.multi_hand_landmarks:#同时出现两只手都可以检测for i, handLms in enumerate(result.multi_hand_landmarks):mpDraw.draw_landmarks(frame, handLms, mpHands.HAND_CONNECTIONS,landmark_drawing_spec=handLmsStyle,connection_drawing_spec=handConStyle)

有个这21个关键点，可以做的事情就太多了，比如控制电脑的音量，鼠标、键盘，如果有个完善的手势姿态库，还可以做比如手语识别等等。因为实际生活中，手的摆放不一定是正好手心面向摄像头的，所以约束条件越苛刻，精度就会越高，这里的做法就没有考虑这么多，就只是用手指不同姿态时的向量L2范数（就是向量的模，忘记了就看线性代数或者机器学习）不同，来粗略的检测，比如说食指，伸直的时候和弯曲的时候，指尖（点8）到手掌根部（点0）的向量模dist1肯定是大于点6到点0的向量模dist2的，如果食指弯曲的时候，则有dist1 < dist2，食指、中指、无名指和小拇指的判断都是如此，仅大拇指就点17代替的点0，代码如下：

for k in range (5):if k == 0:figure_ = finger_stretch_detect(landmark[17],landmark[4*k+2],landmark[4*k+4])else:    figure_ = finger_stretch_detect(landmark[0],landmark[4*k+2],landmark[4*k+4])

然后通过五个手指的状态，来判断当前的手势，我这里列举了一些，简单粗暴：

def detect_hands_gesture(result):if (result[0] == 1) and (result[1] == 0) and (result[2] == 0) and (result[3] == 0) and (result[4] == 0):gesture = "good"elif (result[0] == 0) and (result[1] == 1)and (result[2] == 0) and (result[3] == 0) and (result[4] == 0):gesture = "one"elif (result[0] == 0) and (result[1] == 0)and (result[2] == 1) and (result[3] == 0) and (result[4] == 0):gesture = "please civilization in testing"elif (result[0] == 0) and (result[1] == 1)and (result[2] == 1) and (result[3] == 0) and (result[4] == 0):gesture = "two"elif (result[0] == 0) and (result[1] == 1)and (result[2] == 1) and (result[3] == 1) and (result[4] == 0):gesture = "three"elif (result[0] == 0) and (result[1] == 1)and (result[2] == 1) and (result[3] == 1) and (result[4] == 1):gesture = "four"elif (result[0] == 1) and (result[1] == 1)and (result[2] == 1) and (result[3] == 1) and (result[4] == 1):gesture = "five"elif (result[0] == 1) and (result[1] == 0)and (result[2] == 0) and (result[3] == 0) and (result[4] == 1):gesture = "six"elif (result[0] == 0) and (result[1] == 0)and (result[2] == 1) and (result[3] == 1) and (result[4] == 1):gesture = "OK"elif(result[0] == 0) and (result[1] == 0) and (result[2] == 0) and (result[3] == 0) and (result[4] == 0):gesture = "stone"else:gesture = "not in detect range..."return gesture

然后根据判断的结果输出即可，效果如下：

完整代码如下：

import cv2 as cv
import numpy as np
import mediapipe as mp
from numpy import linalg#视频设备号
DEVICE_NUM = 0# 手指检测
# point1-手掌0点位置，point2-手指尖点位置，point3手指根部点位置
def finger_stretch_detect(point1, point2, point3):result = 0#计算向量的L2范数dist1 = np.linalg.norm((point2 - point1), ord=2)dist2 = np.linalg.norm((point3 - point1), ord=2)if dist2 > dist1:result = 1return result# 检测手势
def detect_hands_gesture(result):if (result[0] == 1) and (result[1] == 0) and (result[2] == 0) and (result[3] == 0) and (result[4] == 0):gesture = "good"elif (result[0] == 0) and (result[1] == 1)and (result[2] == 0) and (result[3] == 0) and (result[4] == 0):gesture = "one"elif (result[0] == 0) and (result[1] == 0)and (result[2] == 1) and (result[3] == 0) and (result[4] == 0):gesture = "please civilization in testing"elif (result[0] == 0) and (result[1] == 1)and (result[2] == 1) and (result[3] == 0) and (result[4] == 0):gesture = "two"elif (result[0] == 0) and (result[1] == 1)and (result[2] == 1) and (result[3] == 1) and (result[4] == 0):gesture = "three"elif (result[0] == 0) and (result[1] == 1)and (result[2] == 1) and (result[3] == 1) and (result[4] == 1):gesture = "four"elif (result[0] == 1) and (result[1] == 1)and (result[2] == 1) and (result[3] == 1) and (result[4] == 1):gesture = "five"elif (result[0] == 1) and (result[1] == 0)and (result[2] == 0) and (result[3] == 0) and (result[4] == 1):gesture = "six"elif (result[0] == 0) and (result[1] == 0)and (result[2] == 1) and (result[3] == 1) and (result[4] == 1):gesture = "OK"elif(result[0] == 0) and (result[1] == 0) and (result[2] == 0) and (result[3] == 0) and (result[4] == 0):gesture = "stone"else:gesture = "not in detect range..."return gesturedef detect():# 接入USB摄像头时，注意修改cap设备的编号cap = cv.VideoCapture(DEVICE_NUM) # 加载手部检测函数mpHands = mp.solutions.handshands = mpHands.Hands()# 加载绘制函数，并设置手部关键点和连接线的形状、颜色mpDraw = mp.solutions.drawing_utilshandLmsStyle = mpDraw.DrawingSpec(color=(0, 0, 255), thickness=int(5))handConStyle = mpDraw.DrawingSpec(color=(0, 255, 0), thickness=int(10))figure = np.zeros(5)landmark = np.empty((21, 2))if not cap.isOpened():print("Can not open camera.")exit()while True:ret, frame = cap.read()if not ret:print("Can not receive frame (stream end?). Exiting...")break#mediaPipe的图像要求是RGB，所以此处需要转换图像的格式frame_RGB = cv.cvtColor(frame, cv.COLOR_BGR2RGB)result = hands.process(frame_RGB)#读取视频图像的高和宽frame_height = frame.shape[0]frame_width  = frame.shape[1]#print(result.multi_hand_landmarks)#如果检测到手if result.multi_hand_landmarks:#为每个手绘制关键点和连接线for i, handLms in enumerate(result.multi_hand_landmarks):mpDraw.draw_landmarks(frame, handLms, mpHands.HAND_CONNECTIONS,landmark_drawing_spec=handLmsStyle,connection_drawing_spec=handConStyle)for j, lm in enumerate(handLms.landmark):xPos = int(lm.x * frame_width)yPos = int(lm.y * frame_height)landmark_ = [xPos, yPos]landmark[j,:] = landmark_# 通过判断手指尖与手指根部到0位置点的距离判断手指是否伸开(拇指检测到17点的距离)for k in range (5):if k == 0:figure_ = finger_stretch_detect(landmark[17],landmark[4*k+2],landmark[4*k+4])else:    figure_ = finger_stretch_detect(landmark[0],landmark[4*k+2],landmark[4*k+4])figure[k] = figure_print(figure,'\n')gesture_result = detect_hands_gesture(figure)cv.putText(frame, f"{gesture_result}", (30, 60*(i+1)), cv.FONT_HERSHEY_COMPLEX, 2, (255 ,255, 0), 5)cv.imshow('frame', frame)if cv.waitKey(1) == ord('q'):breakcap.release()cv.destroyAllWindows()if __name__ == '__main__':detect()

我的公众号：

opencv和mediapipe实现手势识别相关推荐

基于Unity引擎利用OpenCV和MediaPipe的面部表情和人体运动捕捉系统
基于Unity引擎利用OpenCV和MediaPipe的面部表情和人体运动捕捉系统前言项目概述项目实现效果 2D面部表情实时捕捉 3D人体动作实时捕捉补充引用前言之前做的一个项目--使用 ...
基于OpenCV的简易实时手势识别（含代码）
基于OpenCV的简易实时手势识别 1.基本信息介绍 1.1实验步骤 1.2效果展示 2.肤色检测+二值化+开运算+高斯模糊 2.1 flip()函数原型 2.2cvtColor()函数原型 2.3s ...
毕业设计 - 题目：基于机器视觉opencv的手势检测手势识别算法 - 深度学习卷积神经网络 opencv python
文章目录 1 简介 2 传统机器视觉的手势检测 2.1 轮廓检测法 2.2 算法结果 2.3 整体代码实现 2.3.1 算法流程 3 深度学习方法做手势识别 3.1 经典的卷积神经网络 3.2 YOL ...
Python使用Opencv图像处理方法完成手势识别(一)
Opencv完成手势识别 HSV的提取特征提取轮廓绘制完整代码由于是使用Opencv完成手势识别,所以首先利用颜色特征是对手的颜色进行提取,获得HSV的最小值与最大值. HSV的提取 HSV颜 ...
基于MediaPipe的手势识别 --安卓部分
基于MediaPipe的手势识别 --安卓部分 mediapipe网址:可能会有点儿慢 https://google.github.io/mediapipe/solutions/hands.html# ...
基于python+opencv+mediapipe实现手势识别详细讲解
目录运行环境: 一.opencv 二.meidapipe配置三.实现手部的识别并标注 1.参数分析 1.multi_hand_landmarks 2.multi_hand_world_landma ...
基于Opencv和Mediapipe实现手势控制音量
前言在先前的博客中已经实现过了手势追踪的基本功能,由于最近项目需要,开始学习封装操作,也为了更简洁的调用手势追踪模块,所以参照了Youtube上一位大佬的教程,把之前的追踪模块整理了一下,将代码封装 ...
opencv 手部识别_手势识别结合到VR头显中，有哪些难点？
文/知乎用户mysunnytime "VR 头盔要做到「手势识别」,有哪些需要攻克的难点?" 工业场景下,我们可以把这个问题转化为,如何利用更少的软硬件条件,对VR使用场景的手势识 ...
opencv 实现的静态手势识别进而玩剪刀石头布
要想运行该代码,请确保安装了:python 2.7,opencv 2.4.9 效果如下: 算法如下: 把图片先进行处理,处理过程: 1.用膨胀图像与腐蚀图像相减的方法获得轮廓. 2.用二 ...

opencv和mediapipe实现手势识别

opencv和mediapipe实现手势识别相关推荐

最新文章

热门文章