麻省理工学院MAX TEGMARK实验室的一个新AI图书馆 (A NEW AI LIBRARY FROM MAX TEGMARK’S LAB AT MIT)

Table of Contents

目录

1. Introduction2. Code3. Their Example4. Our Own Easy Example5. Symbolic Regression on Noisy Data

1. 简介 2. 代码 3. 他们的示例 4. 我们自己的简单示例 5. 嘈杂数据的符号回归

1.一个新的符号回归库 (1. A New Symbolic Regression Library)

I recently saw a post on LinkedIn from MIT professor Max Tegmark about a new ML library his lab released. I decided to try it out. The paper is AI Feynman 2.0: Pareto-optimal symbolic regression exploiting graph modularity, submitted June 18th, 2020. The first author is Silviu-Marian Udrescu, who was generous enough to hop on a call with me and explain the backstory of this new machine learning library. The library, called AI Feynman 2.0, helps to fit regression formulas to data. More specifically, it helps to fit formulas to data at different levels of complexity (defined in bits). The user can select the operators that the solver will use from sets of operators, and the solver does its thing. Operators are things like exponentiation, cos, arctan, and so on.

最近，我在麻省理工学院教授马克斯·泰格马克(Max Tegmark)的 LinkedIn上看到了他的实验室发布的新ML库的帖子。 我决定尝试一下。 论文为AI Feynman 2.0：利用图形模块化的帕累托最优符号回归 ，于2020年6月18日提交。第一作者是Silviu-Marian Udrescu ，他慷慨地希望与我联系，并解释这台新机器的背景知识学习图书馆。 该库名为AI Feynman 2.0，有助于将回归公式拟合到数据中。 更具体地说，它有助于使公式适合不同复杂程度(以位定义)的数据。 用户可以从一组运算符中选择求解器将使用的运算符，然后求解器将执行其操作。 运算符是幂运算，cos，反正切运算等。

Symbolic regression is a way of stringing together the user-specified mathematical functions to build an equation for the output “y” that best fits the provided dataset. That provided dataset takes the form of sample points (or observations) for each input variable x0, x1, and so forth, along with the corresponding “y”. Since we don’t want to overfit on the data, we need to limit the allowed complexity of the equation or at least have the ability to solve under a complexity constraint. Unlike a neural network, learning one formula with just a few short expressions in it gives you a highly interpretable model, and can lead to insights that you might not get from a neural network model with millions of weights and biases.

符号回归是一种将用户指定的数学函数串在一起以为输出“ y”建立最适合提供的数据集的方程式的方法。 提供的数据集采用每个输入变量x0，x1等的采样点(或观察值)的形式，以及相应的“ y”。 由于我们不想对数据进行过度拟合，因此需要限制方程的允许复杂度，或者至少具有在复杂度约束下求解的能力。 与神经网络不同，学习一个仅包含几个短表达式的公式可以为您提供一个高度可解释的模型，并且可以带来您可能无法从具有数百万个权重和偏差的神经网络模型中获得的见解。

Why is this interesting? Well, science tends to generate lots of observations (data) that scientists want to generalize into underlying rules. These rules are equations that “fit” the observations. Unlike a “usual” machine learning model, equations of the form y=f(x) are very clear, and they can omit some of the variables in the data that are not needed. In the practicing machine learning engineer’s toolbox, regression trees would be the closest concept I can think of that implements this idea of learning an interpretable model that connects observations to a prediction. Having a new way to try and fit a regression model to data is a good addition to the toolbox of stuff you can try on your dataset.

为什么这很有趣？ 好吧，科学往往会产生大量观察结果(数据)，科学家希望将其概括为基本规则。 这些规则是“适合”观察结果的方程式。 与“通常的”机器学习模型不同，形式为y = f(x)的方程非常清晰，它们可以忽略数据中不需要的一些变量。 在实践中的机器学习工程师的工具箱中， 回归树将是我能想到的最接近的概念，它实现了这种学习可解释模型的思想，该模型将观察与预测联系起来。 有一种新的方法可以尝试将回归模型拟合到数据中，这是可以在数据集中尝试的工具箱的很好的补充。

In this article, I want to explore this new library as a user (how to use it), rather than a scientist (how does it work). AI-Feynman 2.0 reminds me of UMAP, in that it includes very fancy math on the inside of the solver, but does something useful to me in an abstracted way that I can treat as a black box. I understand that the code is going to be updated in stages over the next several months, and so the way the interface to the code looks today may not be the way it works when you are reading this. Hopefully, more documentation will also be added to give you a quick path to trying this on your data. For the moment, I’m including a notebook with this article so that you can dive in and get everything working from one place.

在本文中，我想以用户(如何使用)而不是科学家(如何工作)的方式来探索这个新库。 AI-Feynman 2.0使我想起UMAP ，因为它在求解器内部包含了非常漂亮的数学运算，但是以抽象的方式对我有用，可以将其视为黑匣子。 我知道该代码将在接下来的几个月中分阶段进行更新，因此，当您阅读本文时，今天的代码界面外观可能不是它的工作方式。 希望还会添加更多文档，以为您提供在数据上进行尝试的快速途径。 目前，我在本文中包括一个笔记本，以便您可以潜入并从一个地方进行所有工作。

The library uses machine learning to help with the equation discovery, breaking the problem into subproblems recursively, but let’s not get too far into the weeds. Let’s instead turn our attention to using the library. You are welcome to read the paper to learn more about how the library does what it does to solve the symbolic regression mystery on your data.

该库使用机器学习来帮助方程式发现，将问题递归地分解为子问题，但我们不要对杂草深入。 相反，让我们把注意力转向使用库。 欢迎您阅读该文章，以了解有关库如何解决数据上的符号回归之谜的更多信息。

2.代码 (2. Code)

A Google Collab notebook containing all of the code for this article is available here:

包含本文所有代码的Google Collab笔记本可在以下位置找到：

Some notes on the output are important. The solver prints many times the Complexity, RMSE, and Expression. Be aware that the RMSE number is not actually the Root Mean Squared Error. It is the Mean Error Description Length (MEDL) described in the paper, and that message will be changed soon. Also, the Expression printout is not the equation for the dataset, but rather for the sub-problem within the overall problem graph that the solver is currently working on. This is important because you will find that sometimes there is a printout that seems like it has a very low error, but it only applies to some subproblem and is not the equation you are trying to find. The final results are stored in the results folder using the name of the input file.

关于输出的一些注释很重要。 求解器会打印复杂度 ， RMSE和表达式多次。 请注意，RMSE编号实际上不是均方根误差。 这是本文中描述的平均错误描述长度(MEDL)，该消息将很快更改。 同样，表达式打印输出不是数据集的方程式，而是求解器当前正在处理的整体问题图中的子问题。 这很重要，因为您会发现有时打印输出看上去误差很小，但是它仅适用于某些子问题，而不是您要查找的方程式。 最终结果使用输入文件的名称存储在results文件夹中。

3.从AI-Feynman存储库尝试第一个示例 (3. Try the First Example from the AI-Feynman Repository)

Clone the repository and install the dependencies. Next, compile the Fortran code and run the first example dataset from the AI-Feynman repository (example1.txt from the repository).

克隆存储库并安装依赖项。 接下来，编译Fortran代码并运行AI-Feynman存储库中的第一个示例数据集(存储库中的example1.txt)。

The first few steps are listed here:

这里列出了前几个步骤：

Next, put this file into the Code directory and run it with python3:

接下来，将此文件放入Code目录并使用python3运行它：

The first line of the example1.txt file is:

example1.txt文件的第一行是：

1.6821347439986711 1.1786188905177983 4.749225735259924 1.3238356535004034 3.462199507094163

Example 1 contains data generated from an equation, where the last column is the regression target, and the rest of the columns are the input data. The following example shows the relationship between the first line of the file example1.txt and the formula used to make the data.

示例1包含从方程式生成的数据，其中最后一列为回归目标，其余列为输入数据。 下面的示例显示文件example1.txt的第一行与用于制作数据的公式之间的关系。

We can see from running the code snippet above that the target “y” data points in example1.txt are generated using the equation on line 3, where the inputs are all the columns except for the last one, and the equation generates the last column.

通过运行上面的代码片段，我们可以看到example1.txt中的目标“ y”数据点是使用第3行的公式生成的，其中输入是除最后一个列以外的所有列，而该方程式生成了最后一列。

Let’s now run the program. In the folder AI-Feynman/Code/ run the command python3 ai_feynman_magic.py to run the program we wrote above which in turn fits equations to the example1.txt dataset.

现在运行程序。 在文件夹AI-Feynman / Code /中，运行命令python3 ai_feynman_magic.py来运行我们上面编写的程序，该程序又将等式与example1.txt数据集拟合。

The solver runs for a long time, trying different kinds of equations at different levels of complexity, and assessing the best fit for each one. As it works through the solution, it prints intermediate results. If it hits a super low error you can stop the program and just use the equation. It’s really your call if you let it run to the end. For the input file example1.txt, the results show up in AI-Feynman/Code/results/solution_example1.txt. There are other spots where results are generated, but this is the place we care about right now. That file “solution_…txt” ranks the identified solutions. It’s funny that assuming y is a constant is a common strategy for the solver. Constants have no input variables, and so they have low complexity in terms of the number of bits. In the case of example 1, the equation ((x0-x1)**2 + (x2-x3)**2)**0.5 fit the best.

该求解器运行了很长时间，尝试以不同的复杂度尝试各种方程式，并评估每个方程式的最佳拟合。 在解决方案中工作时，它会打印中间结果。 如果遇到超低错误，则可以停止该程序并仅使用方程式。 如果让它运行到最后，那真的是您的电话。 对于输入文件example1.txt ，结果显示在AI-Feynman / Code / results / solution_example1.txt中 。 还有其他产生结果的地方，但这是我们现在关注的地方。 该文件“ solution_…txt ”对所标识的解决方案进行排名。 有趣的是，假设y为常数是求解器的常用策略。 常量没有输入变量，因此就位数而言，其复杂度较低。 在示例1的情况下，公式((x0-x1)** 2 +(x2-x3)** 2)** 0.5最合适。

4.尝试我们自己的简单示例 (4. Try Our Own Simple Example)

In the Collab notebook, I now moved the repository and data to Google Drive so that it will persist. The following code generates 10,000 examples from an equation. This example has 2 “x” variables and 2 duplicated “x” variables. Of course, y is still the output.

现在，在Collab笔记本中，我将存储库和数据移到了Google云端硬盘中，以便持久保存。 以下代码从一个方程式生成10,000个示例。 此示例具有2个“ x”变量和2个重复的“ x”变量。 当然，y仍然是输出。

Plotting the first variable against Y we get:

对Y绘制第一个变量，我们得到：

Now that we took a peek at what our data looks like, let’s ask the solver to find a simple equation that fits our data, using our dataset. The idea is that we want the solver to notice that you don’t need all of the supplied variables in order to fit the data.

现在我们来看看数据是什么样子，让我们让求解器使用我们的数据集找到一个适合我们数据的简单方程式。 这个想法是我们希望求解器注意到您并不需​​要所有提供的变量来拟合数据。

Here is an example of a permissions problem:

这是一个权限问题的示例：

If you have file permission issues when you try to run the code, open up the file permissions like this:

如果在尝试运行代码时遇到文件权限问题，请打开文件权限，如下所示：

chmod +777 AI-Feynman/Code/*

Below is the command to run the solver. Go get coffee, because this is not going to be fast…

下面是运行求解器的命令。 去喝咖啡，因为这不会很快…

python3 ai_feynman_duplicate_variables.py

If you have nothing better to do, watch the solver go. Notice the solver goes through a list of equation types before mixing it up. The initial models it tries out are quickly mapped to x0 and x2 as it “realized” x1 and x3 are duplicates and so not needed. Later on, the solver found the equation 3.000000000000+log(sqrt(exp((x2-x1)))) which is a bit crazy but looks like a plane.

如果您无事可做，请观察求解器。 请注意，求解器在混合之前先浏览了一系列方程类型。 它尝试的初始模型被快速映射到x0和x2，因为它“实现”了x1和x3是重复的，因此不需要。 后来，求解器发现方程3.000000000000 + log(sqrt(exp((x2-x1))))有点疯狂，但看起来像飞机。

We can see on WolframAlpha that an equivalent form of this equation is:

我们可以在WolframAlpha上看到此等式的等效形式为：

y=(x2 - x1)/2 + 3.000000000000

y =(x2-x1)/ 2 + 3.000000000000

which is what we used to generate the dataset!

这就是我们用来生成数据集的东西！

The solver settled on y=log(sqrt(exp(-x1 + x3))) + 3.0 which we know is a correct description of our plane, from the wolfram alpha thing above. The solver ended up using x1 and x3, dropping x0 because it is a copy of x1 and so not needed, and similarly dropping x2 because it is not needed when using x3.

求解器以y = log(sqrt(exp(-x1 + x3)))+ 3.0为基础，根据上面的Wolfram alpha知识，我们知道这是对飞机的正确描述。 求解器最终使用x1和x3，删除了x0，因为它是x1的副本，因此不需要；同样，删除了x2，因为使用x3时不需要它。

Now, that worked, but it was a bit of a softball problem. The data has an exact solution, and so it didn’t need to fit noisy data, which is not a realistic real-world situation. Real data is messy. Let’s now add noise to the dataset and see how the library holds up. We don’t need to go as far as introducing missing variables and imputation. Let’s just make the problem a tiny bit harder to mess with the solver.

现在，这可行了，但这是一个垒球问题。 数据具有精确的解决方案，因此它不需要适合嘈杂的数据，这不是现实的现实情况。 真实数据是混乱的。 现在让我们向数据集中添加噪声，看看库如何保持。 我们不需要深入介绍缺失的变量和插补。 让我们让问题更难解决求解器。

5.噪声数据的符号回归 (5. Symbolic Regression on Noisy Data)

The following code creates points on the same plane as the previous example, but this time noise is added.

以下代码在与上一个示例相同的平面上创建点，但是这次添加了噪声。

The following figure shows how the duplicate columns are now no longer exact duplicates. Will the solver average the points with noise on them to get a better signal? I would average x0 and x1 into a cleaner point, and then average x2 and x3 into a cleaner point. Let’s see what the solver decides to do.

下图显示了重复列现在不再是完全重复的情况。 求解器会平均点上带有噪声的点以获得更好的信号吗？ 我将x0和x1平均为一个清洁点，然后将x2和x3平均为一个清洁点。 让我们看看求解器决定做什么。

We now make yet another runner file as follows:

现在，我们再制作一个运行程序文件，如下所示：

If you have permissions issues, do the chmod 777 thing, or 775 or whatever. To run the program do this:

如果您有权限问题，请执行chmod 777或775之类的操作。 要运行程序，请执行以下操作：

python3 ai_feynman_duplicateVarsWithNoise.py

As the solver works through ideas, it comes up with some wild stuff. You can sort of see in the figure below the plane-like shape in one solution the solver tried: 1.885417681639+log((((x1+1)/cos((x0–1)))+1)). Unfortunately the 2 variables it tried there are x0 and x1, which are duplicates of each other with a small amount of noise added.

当求解器解决想法时，会带来一些荒唐的东西。 您可以在一个求解器尝试过的解决方案中的平面形状下面的图中看到： 1.885417681639 + log((((x1 + 1)/ cos((x0–1)))+ 1)) 。 不幸的是，它尝试过的2个变量分别是x0和x1，它们相互重复，并添加了少量噪声。

Nice try solver. Let’s keep it running and see what happens next.

不错的尝试求解器。 让我们继续运行，看看接下来会发生什么。

The solver found the equation:

求解器发现方程：

y = 3.0–0.25*((x0+x1)-(x2+x3))

y = 3.0–0.25 *((x0 + x1)-(x2 + x3))

As I had hoped, the solver figured out that averaging x0 and x1 gets you a cleaner (less noisy) x01, and averaging x2 and x3 similarly results in a less noisy x23. Recall that the original formula used to make “y” was operating on the input data before we added noise to the inputs:

正如我希望的那样，求解器计算出，对x0和x1进行平均可以使您获得一个更干净(噪声较小)的x01，而对x2和x3进行平均同样可以减少x23的噪声。 回想一下，在我们向输入添加噪声之前，用于使“ y”成为原始公式用于输入数据：

y = -0.5*x01+0.5*x23+3

y = -0.5 * x01 + 0.5 * x23 + 3

Interestingly, the solver also found

有趣的是，求解器还发现

y=3.000000000000+log(sqrt(exp((x2-x0))))

y = 3.000000000000 + log(sqrt(exp((x2-x0))))

This is another version of the formula that uses fewer variables in exchange for a slightly less perfect fit to the data (because of the added noise). And so the solver gives you, the user, the option to see the formula that fits the data at different levels of complexity.

这是公式的另一个版本，该公式使用较少的变量来换取对数据的不太理想的拟合(因为增加了噪声)。 因此，求解器为用户提供了查看适合不同复杂度数据的公式的选项。

六，结论 (6. Conclusion)

A symbolic regression solver called AI-Feynman 2.0 was tested out in this article, starting with the example that comes with the repo, moving to an example we make ourselves from scratch, and finally challenging the solver by adding some noise. A notebook for reproducing this article can be found HERE.

本文测试了一个名为AI-Feynman 2.0的符号回归求解器，从回购随附的示例开始，转到我们从头开始制作的示例，最后通过添加一些噪声来挑战求解器。 可在此处找到用于复制本文的笔记本。

Special thanks to Silviu Marian Udrescu for helping me to better understand the code, and for reviewing an earlier draft of this work to make sure I don’t say silly things. This is going to be fun to try on real-world problems. I have been containerizing this library for Gravity-ai.com to apply to real-world datasets. Hopefully you will find it useful and use it for your own work.

特别感谢Silviu Marian Udrescu帮助我更好地理解了代码，并回顾了此工作的早期草稿以确保我不会说些傻话。 尝试解决实际问题将很有趣。 我一直在为Gravity-ai.com对该库进行容器化，以应用于实际数据集。 希望您会发现它有用并可以用于您自己的工作。

If you liked this article, then have a look at some of my most read past articles, like “How to Price an AI Project” and “How to Hire an AI Consultant.” And hey, join the newsletter!

如果您喜欢这篇文章，那么请看一下我过去阅读最多的一些文章，例如“ 如何为AI项目定价 ”和“ 如何雇用AI顾问” 。 嗨，快来加入新闻通讯 ！

Until next time!

直到下一次！

-DanielLemay.aidaniel@lemay.ai

-Daniel Lemay.ai daniel@lemay.ai