I am of the strong opinion that Reinforcement Learning (RL) is the future of game-playing AI agents. However, to transition to a world where we can use RL in real-life games, there are many sub-problems that we need to take care of first. One such sub-problem is that it is very difficult to make a generalized AI that is good at playing more than one game which it isn’t trained on, even if the game genre is exactly the same.

我坚信强化学习(RL)是玩游戏的AI代理的未来。 但是，要过渡到可以在现实游戏中使用RL的世界，我们首先需要解决许多子问题。 这样的子问题之一是，即使游戏类型完全相同，也很难制作出一个擅长于玩多个未经训练的游戏的通用AI。

For example, if we train a bot that is excellent at driving a car in Gran Turismo, this bot will not perform very well on other car racing games like Forza Horizon. This means for every game we need to train the RL agents from scratch, which is not ideal because the current RL algorithms are not very sample efficient and take a lot of resources to train.

例如，如果我们在Gran Turismo中训练出一个擅长驾驶汽车的机器人，那么该机器人在诸如Forza Horizo​​n之类的其他赛车游戏中的表现将不会很好。 这意味着对于每个游戏，我们都需要从头开始训练RL代理，这是不理想的，因为当前的RL算法效率不高，并且需要大量资源进行训练。

This is why today I want to cover a review paper from the robotics world which could also be applied to solve our problem in the game development. It is titled “Crossing The Gap: A Deep Dive into Zero-Shot Sim-to-Real Transfer for Dynamics” and is published by Robotics Lab at Imperial College London.

这就是为什么我今天想报道机器人世界的一篇评论论文，该论文也可以用于解决我们在游戏开发中的问题。 它的标题是“ 跨越鸿沟：深入研究零动态模拟到真实的动力学转换 ”，由伦敦帝国学院的机器人实验室出版。

This paper focuses on various techniques that are used in robotics research to transfer a trained model from a simulation to real life. The problem here is that various robot and environment dynamics like friction of surface, wind resistance, speed of robotic arm, etc. are ever so slightly different between the simulation and real-life, making it difficult to successfully transfer a model between the two. This means a robot arm that performs well in simulation may not perform well in real life due to these differences in dynamics, which is very similar to our problem in game AI development stated earlier.

本文重点研究机器人技术中使用的各种技术，以将经过训练的模型从仿真转移到现实生活。 这里的问题是，各种机器人和环境动力学(例如表面摩擦，风阻，机械臂的速度等)在模拟和现实生活之间相差甚远，因此很难在两者之间成功地传递模型。 这意味着，由于动力学方面的这些差异，在仿真中表现出色的机械臂在现实生活中可能无法获得良好的表现，这与我们前面提到的游戏AI开发中的问题非常相似。

It seems one of the better approaches to train robust RL policies is to introduce variations and randomness during training time, instead of adapting the learned policies at deployment time. By adding small randomness to the environment dynamics, the RL policy learns to generalize well due to the added noise and works much better when transferred to real-life. Similarly, adding Random Forces to this environment also seems to work just as well, if not better, than tweaking the simulation dynamics.

训练健壮的RL策略的更好方法之一似乎是在训练时引入变化和随机性 ，而不是在部署时适应所学的策略。 通过为环境动力学增加较小的随机性，RL策略可以学习到归因于噪声的一般性，并且在转移到现实生活中时效果更好。 同样，将随机力添加到该环境中似乎也比调整模拟动力学效果更好，甚至更好。

Something similar should be done when training game bots on a particular game. By tweaking parameters like reaction time of agents, we could build a more generalized game AI that can also be transferred to other games of similar genre.

在特定游戏上训练游戏机器人时，应执行类似的操作。 通过调整诸如代理的React时间之类的参数，我们可以构建更通用的游戏AI，也可以将其转移到类似类型的其他游戏中。

有用的链接 (Useful Links)

1. Paper Full-Text (PDF)

   论文全文(PDF)

2. Authors’ Video

   作者的视频

Thank you for reading. If you liked this article, you may follow more of my work on Medium, GitHub, or subscribe to my YouTube channel.

感谢您的阅读。 如果您喜欢这篇文章，可以关注我在Medium ， GitHub上的更多工作，或者订阅我的YouTube频道 。