Welcome to the first post in this series of blogs on extracting features from images using OpenCV and Python.

欢迎使用本系列博客的第一篇有关使用OpenCV和Python从图像中提取特征的文章。

Feature extraction from images and videos is a common problem in the field of Computer Vision. In this post, we will consider the task of identifying balls and table edges on a pool table.

从图像和视频中提取特征是计算机视觉领域的普遍问题。 在这篇文章中，我们将考虑识别台球桌上的球和桌子边缘的任务。

Consider the example image below from an online pool game.

考虑下面的在线撞球游戏示例图像。

Let's say we want to mark the positions of every ball in this image and also the four inner edges of the table. There are multiple ways in which this can be done and some methods work better than others for a given image. However, a useful approach is to try and separate out the contents of an image based on their color composition.

假设我们要标记此图像中每个球的位置以及桌子的四个内边缘。 有多种方法可以完成此操作，对于给定的图像，某些方法比其他方法效果更好。 但是，一种有用的方法是尝试根据图像的颜色成分来分离图像的内容。

For example, in the above image, we can see that the tabletop, the balls and the image background all have different colors.

例如，在上图中，我们可以看到桌面，球和图像背景都有不同的颜色。

Hence if we can separate out the colors in the image, we would be closer to solving our problem.

因此，如果我们可以分离出图像中的颜色，那么我们将更接近解决问题。

An easy way to do this is to convert the RBG image into HSV format and then find out the range of H, S and V values corresponding to the object of interest.

一种简单的方法是将RBG图像转换为HSV格式，然后找出与感兴趣对象相对应的H，S和V值的范围。

For details on this step refer to my blog (coming soon) on HSV based extraction.

有关此步骤的详细信息，请参阅我的博客(即将发布)有关基于HSV的提取。

Once we have the HSV color map for the table top, we can use the OpenCV “inRange()” function to obtain a visualization of the extracted mask as below.

一旦获得了桌面的HSV颜色图，就可以使用OpenCV“ inRange()”函数来获取提取的蒙版的可视化效果，如下所示。

As we can see, this step has helped achieve the following objectives:

如我们所见，这一步骤有助于实现以下目标：

1. the table object (white) is clearly distinguishable from the image background (black)表格对象(白色)可与图像背景(黑色)明显区分开
2. the balls (black) are clearly distinguishable from the table surface (white)球(黑色)与桌面(白色)明显区分开

As a first step, we need to extract the table object from the image in order to focus on the table and its contents and ignore other objects in the image e.g. background, external objects etc.

第一步，我们需要从图像中提取表格对象，以便专注于表格及其内容，而忽略图像中的其他对象，例如背景，外部对象等。

Now is the correct time to apply Edge Detection techniques to identify and extract desired components from the image. There are multiple options available such as Canny and Sobel functions and each has its merits and demerits.

现在是应用边缘检测技术从图像中识别和提取所需成分的正确时机。 有多个选项，例如Canny和Sobel函数，每个都有其优点和缺点。

We will use the OpenCV “findContours()” function for edge detection to extract all contours in the mask image. The contour with the largest area is the one corresponding to the table itself.

我们将使用OpenCV“ findContours()”函数进行边缘检测，以提取蒙版图像中的所有轮廓。 面积最大的轮廓就是与表格本身相对应的轮廓。

In order to implement a smooth extraction of the table, we will find the bounding rectangle (OpenCV “boundingRect()” function) of the table contour and use its coordinates to extract the sub-image from the original image containing only the object of interest, in this case, the table surface and balls as shown in the image below.

为了实现表格的平滑提取，我们将找到表格轮廓的边界矩形(OpenCV “ boundingRect()”函数)，并使用其坐标从仅包含感兴趣对象的原始图像中提取子图像。 ，在这种情况下，工作台表面和球如下图所示。

Now the remaining task is to extract the individual balls and identify the inner edges of the table.

现在剩下的任务是提取单个球并识别桌子的内边缘。

To achieve this, we will again obtain the mask using HSV based extraction method used earlier, first focusing on the balls and then on the table edges.

为此，我们将再次使用之前使用的基于HSV的提取方法来获得蒙版，首先将注意力集中在球上，然后再关注桌面边缘。

检测球 (Detecting balls)

The mask image for the balls will look the same as the one we used earlier for the table. From the obtained mask image, we will extract the ball contours using the OpenCV “findContours()” function once again. This time we are interested in only those contours which resemble a circle and are of a given size.

球的蒙版图像将与我们之前在表格中使用的蒙版图像相同。 从获得的蒙版图像中，我们将再次使用OpenCV “ findContours()”函数提取球的轮廓。 这次，我们仅对那些类似于圆形且具有给定尺寸的轮廓感兴趣。

Again there are many ways to detect the ball contours, but one method which works best is to find the minimum bounding rectangle for each detected contour and chose the ones which best resemble a square and also lie within the desired range of area. We will use the OpenCV function “minAreaRect()” in this case.

同样，有很多方法可以检测到球的轮廓，但是最有效的一种方法是为每个检测到的轮廓找到最小的边界矩形，然后选择最类似于正方形并且也位于所需区域范围内的矩形。 在这种情况下，我们将使用OpenCV函数“ minAreaRect()” 。

On the selected set of contours, we will further apply the OpenCV “minEnclosingCircle()” function to obtain uniform sized circles over each of the balls.

在所选的轮廓集上，我们将进一步应用OpenCV “ minEnclosingCircle() ”函数，以在每个球上获得大小一致的圆。

Now we just need to use OpenCV “circle()” function to draw over each of the detected balls with any color of our choice.

现在，我们只需要使用OpenCV “ circle()”函数以我们选择的任何颜色绘制每个检测到的球。

检测桌子边缘 (Detecting table edges)

This is a two-step approach since the table has both an outer and inner edge and we are interested in only the latter.

这是一种两步方法，因为桌子同时具有外边缘和内边缘，我们仅对后者感兴趣。

The first step is to get a mask for the table edges using the HSV based approach. The obtained mask looks like below in which all four sides can be easily distinguished.

第一步是使用基于HSV的方法为表格边缘获取遮罩。 所获得的掩模如下所示，在其中可以容易地区分所有四个侧面。

With this mask we can now extract the inner edges by locating the two horizontal and two vertical lines which are closest from the center of the image.

使用此蒙版，我们现在可以通过定位距图像中心最近的两条水平线和两条垂直线来提取内部边缘。

We will use the OpenCV “HoughLines()” function to find all lines in the image and select only the 4 of our interest.

我们将使用OpenCV“ HoughLines()”函数查找图像中的所有行，并仅选择我们感兴趣的4条。

Once the 4 lines are detected we just need to use the OpenCV “line()” function to draw the corresponding table edges.

一旦检测到4条线，我们只需要使用OpenCV “ line()”函数绘制相应的表格边缘。

The obtained image can then be overlaid on top of the original image to complete the task as shown below.

然后，可以将获得的图像叠加在原始图像上以完成任务，如下所示。

Hope you have enjoyed reading this post.

希望您喜欢阅读这篇文章。

In my next post, I will cover another interesting example of feature extraction so stay tuned.

在我的下一篇文章中，我将介绍另一个有趣的特征提取示例，请继续关注。

Important Disclaimer:

重要免责声明：

The method used in this blog post especially the HSV values used for detecting balls and table edges will not necessarily work for every image. Every image is unique in its characteristics and needs the right set of parameters in order for feature extraction to work as desired. This is precisely what makes Computer Vision such an interesting and challenging field.

本博客文章中使用的方法，尤其是用于检测球和桌子边缘的HSV值，不一定适用于所有图像。 每张图像的特征都是独特的，并且需要正确的参数集，以便按需进行特征提取。 这正是使Computer Vision成为一个有趣而充满挑战的领域的原因。