介绍 (Introduction)

In this post, I will show how I detect and track players using Yolov3, Opencv and SORT from video clip, and turn the detections to the bird’s-eye view as shown above.

在这篇文章中，我将展示如何使用视频片段中的Yolov3，Opencv和SORT来检测和跟踪播放器，并将检测结果变为鸟瞰图，如上所示。

Inspired by Sam Blake’s great work (https://medium.com/hal24k-techblog/how-to-track-objects-in-the-real-world-with-tensorflow-sort-and-opencv-a64d9564ccb1), I will do the following steps for this project:

受山姆·布雷克 ( Sam Blake )的出色著作( https://medium.com/hal24k-techblog/how-to-track-objects-in-the-real-world-with-tensorflow-sort-and-opencv-a64d9564ccb1 )的启发，我将为此项目执行以下步骤：

1. Object Detection (Yolo and Opencv)对象检测(Yolo和Opencv)
2. Object Tracking (SORT)对象跟踪(SORT)
3. Perspective Transform (Opencv)透视变换(Opencv)

足球视频数据集 (Football video dataset)

In order to have a stable tracking and perspective transform, I need a video clip without camera moving around. I downloaded the video from IPL Ball Detection Datasets. Please be noted that the ball is not tracked in this project, it was already tracked (green bounding box) from the source.

为了获得稳定的跟踪和透视变换，我需要一个没有相机四处移动的视频剪辑。 我从IPL球检测数据集下载了视频。 请注意，此项目未跟踪该球，它已从源中被跟踪(绿色边框)。

物体检测 (Object Detection)

The first step is to load the video and detect the players.

第一步是加载视频并检测播放器。

I used the pre-trained Yolov3 weight and used Opencv’s dnn module and only selected detections classified as ‘person’.

我使用了预先训练的Yolov3权重，并使用了Opencv的dnn模块，并且只选择了归类为“人”的检测项。

I drew bounding boxes for detected players and their tails for previous ten frames.

我为检测到的球员画了边框，并为前十帧画了他们的尾巴。

Looks like the pre-trained model is doing quite okay.

看起来像经过预训练的模型还算不错。

对象追踪 (Object Tracking)

Next I want to track the player and assign unique IDs to them. I used Alex Bewley’s SORT algorithm(simple online and realtime tracking), which I applied to my previous work.

接下来，我要跟踪播放器并为其分配唯一的ID。 我使用了Alex Bewley的SORT算法(简单的在线和实时跟踪)，该算法已应用于我以前的工作中。

Now each player has a unique ID assigned and displayed in the video.

现在，每个播放器都有一个唯一的ID，该ID已分配并显示在视频中。

透视变换 (Perspective Transform)

The video looks good now, but I still want to have players’ motion in bird’s-eye view. It can be done by doing perspective transform. There are a little bit math involved, fortunately Opencv’s getPerspectiveTransform function make it a lot easier.

该视频现在看起来不错，但我仍想以鸟瞰视角让玩家运动。 可以通过执行透视变换来完成。 涉及到一些数学运算，幸运的是， Opencv的getPerspectiveTransform函数使它变得更加容易。

I need to find 4 fixed points as reference and identify the coordinations from the video and also from the bird’s-eye view image.

我需要找到4个固定点作为参考，并从视频和鸟瞰图图像中识别出坐标。

First I identify 4 reference points from the video as show in in red spot and get the pixel coordinations.

首先，我从视频中识别出4个参考点，如红色点所示，并获得像素坐标。

np.array([        [1, 47], # Upper left        [878, 54], # Upper right        [1019, 544], # Lower right        [1, 546] # Lower left                                   ])

I did not really see very solid reference points from the video so I roughly identified 4 points and marked these locations on the bird’s-eye view and got the corresponding pixel coordinations. It will be more precise if the reference points are more robust.

我没有从视频中真正看到非常坚实的参考点，因此我大致确定了4个点，并在鸟瞰图上标记了这些位置，并获得了相应的像素坐标。 如果参考点更健壮，它将更加精确。

np.array([        [871, 37], # Upper left        [1490, 39], # Upper right        [1458, 959], # Lower right        [1061, 955] # Lower left                                   ])

Then by applying Opencv’s getPerspectiveTransform using these reference points, we can transform the detections from video to bird’s-eye view.

然后，通过使用这些参考点应用Opencv的getPerspectiveTransform ，我们可以将检测结果从视频转换为鸟瞰图。

With player’s movement information, it is possible to do further analysis such as players’ running distance and velocity.

利用运动员的运动信息，可以进行进一步分析，例如运动员的跑步距离和速度。

The speed for running this player tracking is around 0.3 second per frame on my 2016 Macbook Pro Intel i5 CPU. It is possible to do this real time by using GPU for some applications if necessary.

在我的2016 Macbook Pro Intel i5 CPU上，运行此播放器跟踪的速度约为每帧0.3秒。 如有必要，可以通过将GPU用于某些应用程序来实时执行此操作。

Thanks for reading, comments and suggestions are welcome!

感谢您的阅读，欢迎提出意见和建议！

In my next post, I used OpenCV to identify players’s team based on their jersey color. Feel free to take a look!

在我的下一篇文章中，我使用OpenCV根据球员的球衣颜色来识别他们的球队。 随意看看！