1.Grasp Lable Format

1.1 Raw Lable Format （原始标签格式）
- （1）Labels on Objects
- （2）Collision Masks on Each Scene
1.2 API Loaded Labels
- （1）Labels示例
- （2）6D Grasp
- （3）6D Grasp Group
- （4）Rectangle Grasp
- （5）Rectangle Grasp Group
1.3 Grasp and GraspGroup 转换

GraspNet

1.1 Raw Lable Format （原始标签格式）

原始标签由两部分组成，每个物体的所有候选抓取标签and每个场景的collision masks。

（1）Labels on Objects

The raw label on each object is a list of numpy arrays.
每个物体的初始标签均为一个np.arry列表。

import numpy as np
import osroot_dir = '/media/robotics/data/datasets/GraspNet/'data_dir1 = os.path.join(root_dir, 'grasp_label/000_labels.npz')
label = np.load(data_dir1) # GRASPNET_ROOT/grasp_label/000_labels.npz
print(label.files)                 # ['points', 'offsets', 'collision', 'scores']print(label['points'].shape)       # (3459, 3)print(label['offsets'].shape)      # (3459, 300, 12, 4, 3)print(label['collision'].shape,    # (3459, 300, 12, 4)label['collision'].dtype)    # dtype('bool')print(label['scores'].shape,       # (3459, 300, 12, 4)label['scores'][0][0][0][0]) # -1.0

name	含义	翻译
‘points’	`records the grasp center point coordinates in model frame.`	记录抓取中心坐标。
‘offsets’	`records the in-plane rotation, depth and width of the gripper respectively in the last dimension.`	记录gripper的旋转、深度和宽度。
‘collision’	`records the bool mask for if the grasp pose collides with the model.`	记录布尔mask。
‘scores’	`records the minimum coefficient of friction between the gripper and object to achieve a stable grasp.`	记录gripper与物体之间的最小摩擦系数to实现稳定抓取。

Notes：

在原始标签中，grasp的分数越低越好，
但是，score = -1.0表示grasp pose完全不能被接受。

（2）Collision Masks on Each Scene

每个场景的collision mask均为一个np.arry列表。

import numpy as npdata_dir2 = os.path.join(root_dir, 'collision_label/scene_0000/collision_labels.npz')
c_labels = np.load(data_dir2) # GRASPNET_ROOT/collision_label/scene_0000/collision_labels.npz
print(c_labels.files)
#['arr_0', 'arr_4', 'arr_5', 'arr_2', 'arr_3', 'arr_7', 'arr_1', 'arr_8', 'arr_6']print(c_labels['arr_0'].shape,      # (487, 300, 12, 4)c_labels['arr_0'].dtype)      # dtype('bool')
print(c_labels['arr_0'][10][20][3]) # array([ True, True, True, True])

‘arr_i’是每个场景的object_id_list.txt中第i个物体的collision mask，其shape = (num points, 300, 12, 4)。
num_points, 300, 12, 4分别为点数、视角ID、内旋ID、深度ID。

更多细节请参考：graspnetAPI.GraspNet.loadGrasp()

1.2 API Loaded Labels

通过调用graspnetAPI.GraspNet.loadGrasp()，可以方便的get场景中的所以grasp labels、params、scores.

加载grasp labels有四种数据结构：Grasp, GraspGroup, RectGrasp and RectGraspGroup.
每个类的内部数据格式是np.arry数组，这比Python列表运算更高效。
它们的定义见 grasp.py

（1）Labels示例

加载一个GraspGroup实例

from graspnetAPI import GraspNet
import open3d as o3d
import cv2# loading grasp for a scene.
graspnet_root = '/media/robotics/data/datasets/GraspNet' # ROOT PATH FOR GRASPNET
sceneId = 1
annId = 3# 1.initialize a GraspNet instance
g = GraspNet(graspnet_root, camera='kinect', split='test_novel')# load grasps of scene 1 with annotation id = 3, camera = kinect and fric_coef_thresh = 0.2
_6d_grasp = g.loadGrasp(sceneId = sceneId, annId = annId, format = '6d', camera = 'kinect', fric_coef_thresh = 0.2)
print('6d grasp:\n{}'.format(_6d_grasp))

Grasp Group, Number=90332:Grasp: score:0.9000000357627869, width:0.11247877031564713, height:0.019999999552965164, depth:0.029999999329447746, translation:[-0.09166837 -0.16910084  0.39480919]
rotation:
[[-0.81045675 -0.57493848  0.11227506][ 0.49874267 -0.77775514 -0.38256073][ 0.30727136 -0.25405255  0.91708326]]
object id:66Grasp: score:0.9000000357627869, width:0.10030215978622437, height:0.019999999552965164, depth:0.019999999552965164, translation:[-0.09166837 -0.16910084  0.39480919]
rotation:
[[-0.73440629 -0.67870212  0.0033038 ][ 0.64608938 -0.70059127 -0.3028869 ][ 0.20788456 -0.22030747  0.95302087]]
object id:66Grasp: score:0.9000000357627869, width:0.08487851172685623, height:0.019999999552965164, depth:0.019999999552965164, translation:[-0.10412319 -0.13797761  0.38312319]
rotation:
[[ 0.03316294  0.78667933 -0.61647028][-0.47164679  0.55612749  0.68430358][ 0.88116372  0.26806271  0.38947761]]
object id:66

通过index or slice访问元素

# index（索引）
print('_6d_grasp[0](grasp):\n{}'.format(_6d_grasp[0]))
# slice（切片）
print('_6d_grasp[0:2]:\n{}'.format(_6d_grasp[0:2]))
print('_6d_grasp[[0,1]]:\n{}'.format(_6d_grasp[[0,1]]))

_6d_grasp[0] (grasp):

Grasp: score:0.9000000357627869, width:0.11247877031564713, height:0.019999999552965164, depth:0.029999999329447746, translation:[-0.09166837 -0.16910084  0.39480919]
rotation:
[[-0.81045675 -0.57493848  0.11227506][ 0.49874267 -0.77775514 -0.38256073][ 0.30727136 -0.25405255  0.91708326]]
object id:66

_6d_grasp[0:2]:

Grasp Group, Number=2:Grasp: score:0.9000000357627869, width:0.11247877031564713, height:0.019999999552965164, depth:0.029999999329447746, translation:[-0.09166837 -0.16910084  0.39480919]
rotation:
[[-0.81045675 -0.57493848  0.11227506][ 0.49874267 -0.77775514 -0.38256073][ 0.30727136 -0.25405255  0.91708326]]
object id:66Grasp: score:0.9000000357627869, width:0.10030215978622437, height:0.019999999552965164, depth:0.019999999552965164, translation:[-0.09166837 -0.16910084  0.39480919]
rotation:
[[-0.73440629 -0.67870212  0.0033038 ][ 0.64608938 -0.70059127 -0.3028869 ][ 0.20788456 -0.22030747  0.95302087]]
object id:66

_6d_grasp[[0,1]]:

Grasp Group, Number=2:Grasp: score:0.9000000357627869, width:0.11247877031564713, height:0.019999999552965164, depth:0.029999999329447746, translation:[-0.09166837 -0.16910084  0.39480919]
rotation:
[[-0.81045675 -0.57493848  0.11227506][ 0.49874267 -0.77775514 -0.38256073][ 0.30727136 -0.25405255  0.91708326]]
object id:66Grasp: score:0.9000000357627869, width:0.10030215978622437, height:0.019999999552965164, depth:0.019999999552965164, translation:[-0.09166837 -0.16910084  0.39480919]
rotation:
[[-0.73440629 -0.67870212  0.0033038 ][ 0.64608938 -0.70059127 -0.3028869 ][ 0.20788456 -0.22030747  0.95302087]]
object id:66

GraspGroup的每个元素都是一个Grasp实例，
Grasp的属性可以通过提供的方法访问

# grasp is a Grasp instance defined in grasp.py
# access and set properties.
grasp = _6d_grasp[0]print('grasp.translation={}'.format(grasp.translation))grasp.translation = np.array([1.0, 2.0, 3.0])
print('After modification, grasp.translation={}'.format(grasp.translation))
print('grasp.rotation_matrix={}'.format(grasp.rotation_matrix))grasp.rotation_matrix = np.eye(3).reshape((9))
print('After modification, grasp.rotation_matrix={}'.format(grasp.rotation_matrix))
print('grasp.width={}, height:{}, depth:{}, score:{}'.format(grasp.width, grasp.height, grasp.depth, grasp.score))

grasp.translation=[-0.09166837 -0.16910084  0.39480919]
After modification, grasp.translation=[1. 2. 3.]grasp.rotation_matrix=[[-0.81045675 -0.57493848  0.11227506][ 0.49874267 -0.77775514 -0.38256073][ 0.30727136 -0.25405255  0.91708326]]
After modification, grasp.rotation_matrix=[[1. 0. 0.][0. 1. 0.][0. 0. 1.]]grasp.width=0.11247877031564713, height:0.019999999552965164, depth:0.029999999329447746, score:0.9000000357627869

RectGrasp是矩形抓取的类，
格式与Grasp不同，但提供的API类似。

# load rectangle grasps of scene 1 with annotation id = 3, camera = realsense and fric_coef_thresh = 0.2
rect_grasp_group = g.loadGrasp(sceneId = sceneId, annId = annId, format = 'rect', camera = 'realsense', fric_coef_thresh = 0.2)
print('rectangle grasp group:\n{}'.format(rect_grasp_group))# rect_grasp is an RectGraspGroup instance defined in grasp.py
print('rect_grasp_group:\n{}'.format(rect_grasp_group))# index
rect_grasp = rect_grasp_group[0]
print('rect_grasp_group[0](rect_grasp):\n{}'.format(rect_grasp))# slice
print('rect_grasp_group[0:2]:\n{}'.format(rect_grasp_group[0:2]))
print('rect_grasp_group[[0,1]]:\n{}'.format(rect_grasp_group[[0,1]]))# properties of rect_grasp
print('rect_grasp.center_point:{}, open_point:{}, height:{}, score:{}'.format(rect_grasp.center_point, rect_grasp.open_point, rect_grasp.height, rect_grasp.score))

rectangle grasp group:
----------
Rectangle Grasp Group, Number=29036:
Rectangle Grasp: score:0.40000003576278687, height:52.59913635253906, open point:(677.3685, -87.56467), center point:(560.52704, -127.61005), object id:66
Rectangle Grasp: score:0.40000003576278687, height:52.5867805480957, open point:(682.6998, -85.73743), center point:(560.52704, -127.61005), object id:66
Rectangle Grasp: score:0.6000000238418579, height:52.63461685180664, open point:(649.1359, -68.900116), center point:(560.52704, -127.61005), object id:66
Rectangle Grasp: score:0.30000001192092896, height:52.629859924316406, open point:(651.5441, -67.304474), center point:(560.52704, -127.61005), object id:66
Rectangle Grasp: score:0.800000011920929, height:52.84688186645508, open point:(625.3405, -55.22821), center point:(560.52704, -127.61005), object id:66
......
Rectangle Grasp: score:0.30000001192092896, height:40.82650375366211, open point:(613.7542, 459.35962), center point:(668.34296, 521.0328), object id:14
Rectangle Grasp: score:0.40000003576278687, height:40.82828140258789, open point:(749.2127, 549.03204), center point:(668.34296, 521.0328), object id:14
Rectangle Grasp: score:0.6000000238418579, height:39.991939544677734, open point:(584.6618, 410.68518), center point:(658.4677, 494.0827), object id:14
Rectangle Grasp: score:0.5, height:39.60657501220703, open point:(596.4554, 400.3207), center point:(670.0193, 483.46442), object id:14
Rectangle Grasp: score:0.40000003576278687, height:39.93021774291992, open point:(602.8676, 408.1247), center point:(673.44824, 487.8963), object id:14
----------rect_grasp_group:
----------
Rectangle Grasp Group, Number=29036:
Rectangle Grasp: score:0.40000003576278687, height:52.59913635253906, open point:(677.3685, -87.56467), center point:(560.52704, -127.61005), object id:66
Rectangle Grasp: score:0.40000003576278687, height:52.5867805480957, open point:(682.6998, -85.73743), center point:(560.52704, -127.61005), object id:66
Rectangle Grasp: score:0.6000000238418579, height:52.63461685180664, open point:(649.1359, -68.900116), center point:(560.52704, -127.61005), object id:66
Rectangle Grasp: score:0.30000001192092896, height:52.629859924316406, open point:(651.5441, -67.304474), center point:(560.52704, -127.61005), object id:66
Rectangle Grasp: score:0.800000011920929, height:52.84688186645508, open point:(625.3405, -55.22821), center point:(560.52704, -127.61005), object id:66
......
Rectangle Grasp: score:0.30000001192092896, height:40.82650375366211, open point:(613.7542, 459.35962), center point:(668.34296, 521.0328), object id:14
Rectangle Grasp: score:0.40000003576278687, height:40.82828140258789, open point:(749.2127, 549.03204), center point:(668.34296, 521.0328), object id:14
Rectangle Grasp: score:0.6000000238418579, height:39.991939544677734, open point:(584.6618, 410.68518), center point:(658.4677, 494.0827), object id:14
Rectangle Grasp: score:0.5, height:39.60657501220703, open point:(596.4554, 400.3207), center point:(670.0193, 483.46442), object id:14
Rectangle Grasp: score:0.40000003576278687, height:39.93021774291992, open point:(602.8676, 408.1247), center point:(673.44824, 487.8963), object id:14
----------rect_grasp_group[0](rect_grasp):
Rectangle Grasp: score:0.40000003576278687, height:52.59913635253906, open point:(677.3685, -87.56467), center point:(560.52704, -127.61005), object id:66rect_grasp_group[0:2]:
----------
Rectangle Grasp Group, Number=2:
Rectangle Grasp: score:0.40000003576278687, height:52.59913635253906, open point:(677.3685, -87.56467), center point:(560.52704, -127.61005), object id:66
Rectangle Grasp: score:0.40000003576278687, height:52.5867805480957, open point:(682.6998, -85.73743), center point:(560.52704, -127.61005), object id:66
----------rect_grasp_group[[0,1]]:
----------
Rectangle Grasp Group, Number=2:
Rectangle Grasp: score:0.40000003576278687, height:52.59913635253906, open point:(677.3685, -87.56467), center point:(560.52704, -127.61005), object id:66
Rectangle Grasp: score:0.40000003576278687, height:52.5867805480957, open point:(682.6998, -85.73743), center point:(560.52704, -127.61005), object id:66
----------rect_grasp.center_point:(560.52704, -127.61005), open_point:(677.3685, -87.56467), height:52.59913635253906, score:0.40000003576278687

（2）6D Grasp

17 个浮点数用于定义通用的 6d grasp，width、height、 depth、 score and 附加object id也是定义的一部分。

Notes：
抓取的分数越高越好，这与原始标签不同。
实际上，score = 1.1 - raw_score（摩擦系数）

class Grasp():def __init__(self, *args):'''**Input:**- args can be a numpy array or tuple of the score, width, height, depth, rotation_matrix, translation, object_id- the format of numpy array is [score, width, height, depth, rotation_matrix(9), translation(3), object_id]- the length of the numpy array is 17.'''if len(args) == 0:self.grasp_array = np.array([0, 0.02, 0.02, 0.02, 1, 0, 0, 0, 1 ,0 , 0, 0, 1, 0, 0, 0, -1], dtype = np.float64)elif len(args) == 1:if type(args[0]) == np.ndarray:self.grasp_array = copy.deepcopy(args[0])else:raise TypeError('if only one arg is given, it must be np.ndarray.')elif len(args) == 7:score, width, height, depth, rotation_matrix, translation, object_id = argsself.grasp_array = np.concatenate([np.array((score, width, height, depth)),rotation_matrix.reshape(-1), translation, np.array((object_id)).reshape(-1)]).astype(np.float64)else:raise ValueError('only 1 or 7 arguments are accepted')

（3）6D Grasp Group

通常，一个场景中有很多grasps，这些抓点组成一个类GraspGroup。
相比Grasp，GraspGroup包含一个二维 numpy 数组，额外的维度是每个grasp的索引。

class GraspGroup():def __init__(self, *args):'''**Input:**- args can be (1) nothing (2) numpy array of grasp group array (3) str of the npy file.'''if len(args) == 0:self.grasp_group_array = np.zeros((0, GRASP_ARRAY_LEN), dtype=np.float64)elif len(args) == 1:if isinstance(args[0], np.ndarray):self.grasp_group_array = args[0]elif isinstance(args[0], str):self.grasp_group_array = np.load(args[0])else:raise ValueError('args must be nothing, numpy array or string.')else:raise ValueError('args must be nothing, numpy array or string.')

实现了对列表的常见操作，例如s indexing, slicing and sorting。
此外，一个重要的功能是用户可以将GraspGroup转储到一个 numpy 文件中，并通过调用GraspGroup.save_npy()和GraspGroup.from_npy()将其加载到另一个程序中。

（4）Rectangle Grasp

7个浮点数用于定义通用的矩形抓取，即the center point, the open point, height, score and the
attached object id.。每个参数的详细定义如上图所示，中心点和开放点的坐标在像素框中。

class RectGrasp():def __init__(self, *args):'''**Input:**- args can be a numpy array or tuple of the center_x, center_y, open_x, open_y, height, score, object_id- the format of numpy array is [center_x, center_y, open_x, open_y, height, score, object_id]- the length of the numpy array is 7.'''if len(args) == 1:if type(args[0]) == np.ndarray:self.rect_grasp_array = copy.deepcopy(args[0])else:raise TypeError('if only one arg is given, it must be np.ndarray.')elif len(args) == RECT_GRASP_ARRAY_LEN:self.rect_grasp_array = np.array(args).astype(np.float64)else:raise ValueError('only one or six arguments are accepted')

（5）Rectangle Grasp Group

RectGraspGroup的格式类似于RectGrasp和GraspGroup

class RectGraspGroup():def __init__(self, *args):'''**Input:**- args can be (1) nothing (2) numpy array of rect_grasp_group_array (3) str of the numpy file.'''if len(args) == 0:self.rect_grasp_group_array = np.zeros((0, RECT_GRASP_ARRAY_LEN), dtype=np.float64)elif len(args) == 1:if isinstance(args[0], np.ndarray):self.rect_grasp_group_array = args[0]elif isinstance(args[0], str):self.rect_grasp_group_array = np.load(args[0])else:raise ValueError('args must be nothing, numpy array or string.')else:raise ValueError('args must be nothing, numpy array or string.')

1.3 Grasp and GraspGroup 转换

可以转换 Grasp 或 GraspGroup 得到 4x4 矩阵。

from graspnetAPI import *
# transform grasp
g = Grasp() # simple Grasp
frame = o3d.geometry.TriangleMesh.create_coordinate_frame(0.1)# Grasp before transformation
o3d.visualization.draw_geometries([g.to_open3d_geometry(), frame])
g.translation = np.array((0,0,0.01))# setup a transformation matrix
T = np.eye(4)
T[:3,3] = np.array((0.01, 0.02, 0.03))
T[:3,:3] = np.array([[0,0,1.0],[1,0,0],[0,1,0]])
g.transform(T)# Grasp after transformation
o3d.visualization.draw_geometries([g.to_open3d_geometry(), frame])g1 = Grasp()
gg = GraspGroup()
gg.add(g)
gg.add(g1)# GraspGroup before transformation
o3d.visualization.draw_geometries([*gg.to_open3d_geometry_list(), frame])gg.transform(T)# GraspGroup after transformation
o3d.visualization.draw_geometries([*gg.to_open3d_geometry_list(), frame])

1.GraspNet-API之Grasp Lable Format相关推荐

解读 ESP32 API参考-system-App Image Format
ESP32 API参考 - system App Image Format Application Description Adding a Custom Structure to an Applic ...
【微信小程序】NodeJs调用云开发HTTP API错误代码47001 / data format error错误
问题描述服务器使用nodejs发送http请求调用云开发api报错47001 //返回结果如下 {errcode: 47001,errmsg: 'data format error hint: [x ...
Marketing Cloud API消费entity unsupported format错误消息的处理
在消费SAP Marketing Cloud API时,遇到如下错误: 解决方案是在发起http请求的头部添加content-type字段,类型为multipart/mixed: 要获取更多Jerry ...
GraspNet 通用物品抓取数据集
GraspNet 通用物品抓取数据集 GraspNet 是一个通用物品抓取的开源项目,现包含 GraspNet-1Biliion:使用平行爪进行抓取 SuctionNet-1Billion:使用吸盘进 ...
rails 构建 API
我是来鼓吹使用 Rails 写 API 的. 原文在此: https://labs.kollegorna.se/blog/2015/04/build-an-api-now/ 原文有一个很大的缺陷就是读 ...
【Python爬虫】微信公众号历史文章和文章评论API分析
上一篇文章爬取微信公众号文章信息准备工作介绍了微信公众号历史文章和文章评论API的组成情况,历史文章API格式:https://mp.weixin.qq.com/mp/profile_ext?acti ...
JAVA day13，14 API、Object类、日期时间类（long，Date，Calendar，DateFormat）、String类（字符串，可变长字符串）、正则表达式、包装类
1.API API(Application Programming Interface),应⽤程序编程接⼝.Java API是⼀本程序员的"字典",是JDK提供给我们使⽤的类的说 ...
好玩免费的api接口
本期介绍一些好玩免费的api接口, 点击标题可以直接查看结果图片类API 淘宝买家秀随机图片 https://api.66mz8.com/api/rand.tbimg.php?format=jpg ...
php酷狗音乐API接口,酷狗音乐抓取api
关键词抓取歌曲列表,获取hash http://mobilecdn.kugou.com/api/v3/search/song?format=json&keyword=%E6%88%91%E4% ...

1.GraspNet-API之Grasp Lable Format