Tech giants like Google and Microsoft have taken notice of exciting new AI research such as GPT-3, which can write articles, website markup, and even software code. But will their bottom lines stifle any real progress?

像Google和Microsoft这样的科技巨头已经注意到了激动人心的AI新研究，例如GPT-3，它可以编写文章，网站标记甚至软件代码。 但是，他们的底线会扼杀任何实际进展吗？

By Ben Dickson

通过本迪克森

A recent article in The Guardian stirred up a lot of excitement-and a little fear-on social media. The reason: The initial draft was reportedly written by GPT-3, OpenAI’s new text generator.

《卫报》上最近的一篇文章激起了社会媒体的兴奋和恐惧。 原因：据报道，最初的草案是由OpenAI的新文本生成器GPT-3编写的。

Since its beta release, GPT-3, an artificial intelligence system that takes a cue and generates text, has captivated the tech community and the media. Developers and computer scientists have been using it to write articles, website markup, and even software code. Some entrepreneurs are contemplating creating new products on GPT-3.

自测试版发布以来，具有提示功能并生成文本的人工智能系统GPT-3吸引了技术界和媒体。 开发人员和计算机科学家一直在使用它编写文章，网站标记甚至软件代码。 一些企业家正在考虑在GPT-3上开发新产品。

While flawed in fundamental ways, GPT-3 still shows how far advances in natural language processing have come. This is by far the largest and most coherent text-generation algorithm ever created.

尽管GPT-3在根本上存在缺陷，但它仍然显示出自然语言处理技术已经取得了多大进步。 这是迄今为止创建的最大，最一致的文本生成算法。

But it also highlights some of the problems the AI research community faces, including its growing dependence on the wealth of large tech companies. This is a problem that could endanger the scientific mission for which OpenAI and other AI research labs were founded.

但这也凸显了AI研究界面临的一些问题，包括其对大型科技公司财富的日益依赖。 这个问题可能会危及OpenAI和其他AI研究实验室建立的科学使命。

GPT-3的费用 (The Cost of GPT-3)

GPT-3 is a massive deep-learning model. Deep learning is a type of AI system that develops its behavior through experience. Every deep learning model is composed of many layers of parameters that start at random values and gradually tune themselves as the model is trained on examples.

GPT-3是一个大规模的深度学习模型。 深度学习是一种通过经验来发展其行为的AI系统。 每个深度学习模型均由多层参数组成，这些参数层从随机值开始，并随着在实例上进行训练而逐渐进行自我调整。

Before deep learning, programmers and domain experts had to manually write the commands that defined the logic and rules to parse and make sense of text. With deep learning, you provide a model with a large corpus of text-say, Wikipedia articles-and it adjusts its parameters to capture the relations between the different words. You can then use the model for a variety of language tasks such as answering questions, automatic email-reply suggestions, and advanced search.

在进行深度学习之前，程序员和领域专家必须手动编写定义逻辑和规则的命令，以解析和理解文本。 通过深度学习，您可以为模型提供大量的语料库，例如Wikipedia文章，并且可以调整其参数以捕获不同单词之间的关系。 然后，您可以将该模型用于各种语言任务，例如回答问题，自动电子邮件回复建议和高级搜索。

Research and development in the past few years has shown that in general, the performance of deep-learning models improves as they are given larger numbers of parameters and trained on bigger data sets.

过去几年的研究和开发表明，一般而言，深度学习模型的性能会得到提高，因为它们会被赋予更多的参数并在更大的数据集上进行训练。

In this respect, GPT-3 has broken all records: It is composed of 175 billion parameters, which makes it more than a hundred times larger than its predecessor, GPT-2. And the data set used to train the AI is at least 10 times larger than GPT-2’s 40-gigabyte training corpus. Although there’s much debate about whether larger neural networks will solve the fundamental problem of understanding the context of language, GPT-3 has outperformed all of its predecessors in language-related tasks.

在这方面，GPT-3打破了所有记录：它由1,750亿个参数组成，这使其比其前身GPT-2大100倍以上。 用于训练AI的数据集至少比GPT-2的40 GB训练语料库大10倍。 尽管关于大型神经网络是否能够解决理解语言上下文的根本问题仍有很多争议，但GPT-3在与语言有关的任务中胜过了所有其前任。

But the benefits of larger neural networks come with trade-offs. The more parameters and layers you add to a neural network, the more expensive its training becomes. According to an estimate by Chuan Li, the Chief Science Officer of Lambda, a provider of hardware and cloud resources for deep learning, it could take up to 355 years and $4.6 million to train GPT-3 on a server with a V100 graphics card.

但是，更大的神经网络的好处在于权衡取舍。 添加到神经网络的参数和层越多，其训练费用就越高。 根据用于深度学习的硬件和云资源提供商Lambda的首席科学官Chuan Li的估计，在配备V100显卡的服务器上训练GPT-3可能需要长达355年的时间和460万美元。

“Our calculation with a V100 GPU is extremely simplified. In practice, you can’t train GPT-3 on a single GPU, but with a distributed system with many GPUs like the one OpenAI used,” Li says. “One will never get perfect scaling in a large distributed system due to the overhead of device-to-device communication. So in practice, it will take more than $4.6 million to finish the training cycle.”

“我们使用V100 GPU的计算得到了极大简化。 在实践中，您不能在单个GPU上训练GPT-3，但要在具有许多GPU的分布式系统(例如使用的一个OpenAI)上进行训练，” Li说。 “由于设备到设备通信的开销，在大型分布式系统中永远无法获得完美的扩展。 因此，在实践中，完成培训周期将需要超过460万美元。”

This estimate is still simplified. Training a neural network is hardly a one-shot process. It involves a lot of trial and error, and engineers must often change the settings and retrain the network to obtain optimal performance.

该估计仍被简化。 训练神经网络几乎不是一shot而就的过程。 它涉及很多试验和错误，工程师必须经常更改设置并重新培训网络以获得最佳性能。

“There are certainly behind-the-scenes costs as well: parameter tuning, the prototyping that it takes to get a finished model, the cost of researchers, so it certainly was expensive to create GPT-3,” says Nick Walton, the co-founder of Latitude and the creator of AI dungeon, a text-based game created on GPT-2.

“当然还有幕后的成本：参数调整，获得完整模型所需的原型，研究人员的成本，因此创建GPT-3肯定是昂贵的，”合作伙伴Nick Walton说道。 -Latitude的创始人和AI地下城的创造者，这是一款基于GPT-2创建的基于文本的游戏。

Walton said that the real cost of the research behind GPT-3 could be anywhere between 1.5 to 5 times the cost of training the final model, but he added, “It’s really hard to say without knowing what their process looks like internally.”

沃尔顿说，GPT-3背后的研究的实际成本可能是训练最终模型的成本的1.5到5倍之间，但他补充说：“在不知道内部流程是什么样的情况下，很难说。”

追求盈利模式 (Going to a For-Profit Model)

OpenAI was founded in late 2015 as a nonprofit research lab with the mission to develop human-level AI for the benefit of all humanity. Among its founders were Tesla CEO Elon Musk and Sam Altman, former Y Combinator president, who collectively donated $1 billion to the lab’s research. Altman later became the CEO of OpenAI.

OpenAI成立于2015年末，是一个非营利性研究实验室，其使命是开发造福全人类的人类级AI 。 其创始人包括特斯拉首席执行官埃隆·马斯克(Elon Musk)和前Y Combinator总裁萨姆·奥特曼(Sam Altman)，他们共同为该实验室的研究捐款10亿美元。 奥特曼后来成为OpenAI的首席执行官。

But given the huge costs of training deep-learning models and hiring AI talent, $1 billion would cover only a few years’ worth of OpenAI’s research. It was clear from the beginning that the lab would run into cash problems long before it reached its goal.

但是考虑到培训深度学习模型和雇用AI人才的巨额成本，10亿美元将仅涵盖OpenAI研究的几年价值。 从一开始就很明显，实验室在达到目标之前就陷入了现金问题。

“We’ll need to invest billions of dollars in upcoming years into large-scale cloud compute, attracting and retaining talented people, and building AI supercomputers,” the lab declared in 2019, when it renamed itself OpenAI LP and restructured to a “capped-profit” company. The change allowed venture capital firms and large tech companies to invest in OpenAI for returns “capped” at a hundred times their initial investment.

该实验室在2019年宣布时将其命名为OpenAI LP，并改组为“封闭式”实验室，该实验室于2019年宣布：“未来几年，我们需要投资数十亿美元用于大规模云计算，吸引和留住人才，并建造AI超级计算机。” -利润”公司。 这一变化使风险投资公司和大型科技公司可以在OpenAI上进行投资，以将其“最高”回报限制为初始投资的100倍。

Shortly after the announcement, Microsoft invested $1 billion in OpenAI. The infusion of cash allowed the company to continue to work on GPT-3 and other expensive deep-learning projects. But investor money always comes with strings attached.

宣布后不久，微软向OpenAI投资了10亿美元。 注入的现金使公司得以继续从事GPT-3和其他昂贵的深度学习项目。 但是投资者的钱总是附带条件。

向模糊过渡 (Shifting Toward Obscurity)

In June, when it announced GPT-3, the company did not release its AI model to the public, as is the norm in scientific research. Instead, it released an application programming interface (API) that allows developers to give GPT-3 input and obtain the results. In the future, the company will commercialize GPT-3 by renting out access to the API.

6月，当它宣布GPT-3时，该公司并未像科学研究的规范那样向公众发布其AI模型。 相反，它发布了一个应用程序编程接口(API)，允许开发人员提供GPT-3输入并获取结果。 将来，该公司将通过出租API的访问权来实现GPT-3的商业化。

“Commercializing the technology helps us pay for our ongoing AI research, safety, and policy efforts,” OpenAI wrote in a blog post announcing the GPT-3 API.

OpenAI在宣布GPT-3 API的博客文章中写道：“将该技术商业化有助于我们为正在进行的AI研究，安全和政策工作付费。”

But to make GPT-3 profitable, OpenAI will have to make sure other companies can’t replicate it, which is why it is not making the source code and trained model public. Organizations and individuals can request access to the GPT-3 API-but not every request is approved.

但是，要使GPT-3盈利，OpenAI必须确保其他公司不能复制它，这就是为什么它不公开源代码和经过训练的模型。 组织和个人可以请求访问GPT-3 API，但并非所有请求都得到批准。

Among those who weren’t given access to GPT-3 API are Gary Marcus, cognitive scientist and AI researcher, and Ernest Davis, computer science professor at New York University, who were interested in testing the capabilities and limits of GPT-3.

未被授予使用GPT-3 API权限的人包括认知科学家和AI研究人员Gary Marcus和纽约大学计算机科学教授Ernest Davis，他们对测试GPT-3的功能和限制感兴趣。

“OpenAI has thus far not allowed us research access to GPT-3, despite both the company’s name and the nonprofit status of its oversight organization. Instead, OpenAI put us off indefinitely despite repeated requests-even as it made access widely available to the media,” Marcus and Davis wrote in an article published in MIT Technology Review. “OpenAI’s striking lack of openness seems to us to be a serious breach of scientific ethics, and a distortion of the goals of the associated nonprofit.”

“尽管公司的名称和其监督组织的非营利组织身份，OpenAI迄今仍不允许我们对GPT-3进行研究访问。 相反，尽管反复提出要求，OpenAI仍然无限期地推迟了我们的工作，即使它使媒体可以广泛使用它。” Marcus和Davis在《麻省理工科技评论》上发表的一篇文章中写道。 “ OpenAI明显缺乏开放性，在我们看来，这严重违反了科学道德规范，并扭曲了相关非营利组织的目标。”

The two scientists managed to run the experiments through a colleague who had access to the API, but their research was limited to a small number of tests. Marcus had been a vocal critic of the hype surrounding GPT-3’s predecessor.

两位科学家设法通过可以访问API的同事进行了实验，但是他们的研究仅限于少量测试。 马库斯曾大声批评围绕GPT-3的前身大肆宣传。

可以保存AI研究吗？ (Can AI Research Be Saved?)

GPT-3 shows the growing challenges of scientific AI research. The focus on creating larger and larger neural networks is increasing the costs of research. And, for the moment, the only organizations that can dispense that kind of money are large tech companies such as Google, Microsoft, and SoftBank.

GPT-3显示了AI科学研究日益严峻的挑战。 对创建越来越大的神经网络的关注增加了研究成本。 而且，目前，唯一可以分配这类资金的组织是大型科技公司，例如Google，Microsoft和SoftBank。

But those companies are interested in short-term returns on investment, not long-term goals that benefit humanity in its entirety.

但是那些公司对短期投资回报感兴趣，而不是对整个人类有利的长期目标。

OpenAI now has a commitment to Microsoft and other potential investors, and it must show proof that it is a profitable company to ensure future funding. At the same time, it wants to pursue its scientific mission of creating beneficial AGI (artificial general intelligence, essentially human-level AI), which does not have short-term returns and is at least decades away.

OpenAI现在对微软和其他潜在投资者有承诺，并且必须证明它是确保未来资金投入的盈利公司。 同时，它希望履行其科学使命，即创建有益的AGI(人工通用情报，本质上是人类水平的AI)，该投资没有短期回报，而且至少需要几十年的时间。

Those two goals conflict in other ways. Scientific research is predicated on transparency and information sharing among different communities of scientists. In contrast, creating profitable products requires hiding research and hoarding company secrets to keep the edge over competitors.

这两个目标在其他方面存在冲突。 科学研究基于不同科学家社区之间的透明度和信息共享。 相反，创造有利润的产品需要隐藏研究和ho积公司机密，以保持领先于竞争对手的优势。

Finding the right balance between the nonprofit mission and the for-profit commitment will be extremely difficult. And OpenAI’s situation is not an isolated example. DeepMind, the UK-based research lab that is considered one of OpenAI’s peers, faced similar problems after it was acquired by Google in 2014.

在非营利组织的使命与营利性承诺之间找到适当的平衡将非常困难。 而且OpenAI的情况并非孤立的例子。 DeepMind是位于英国的研究实验室，被认为是OpenAI的同行之一，它在2014年被Google收购后也面临类似的问题。

Many scientists believe that AGI-if ever achieved-will be one of the most impactful inventions of humanity. If this is true, then achieving AGI will require the concerted efforts and contributions of the international community, not merely the deep pockets of companies whose main focus is their bottom line.

许多科学家认为，如果能够实现AGI，它将是人类最有影响力的发明之一。 如果这是真的，那么实现AGI将需要国际社会的共同努力和贡献，而不仅仅是主要关注其底线的公司的财大气粗。

A good model might be the Large Hadron Collider project, which obtained a $9 billion budget from funding agencies in CERN’s member and non-member states. While member states will eventually benefit from the results of CERN’s work, they don’t expect the organization to turn in profits in the short term.

一个很好的模型可能是大型强子对撞机项目，该项目从CERN成员国和非成员国的供资机构获得了90亿美元的预算。 尽管成员国最终将从CERN的工作成果中受益，但他们并不希望该组织在短期内实现盈利。

A similar initiative might help OpenAI and other research labs to continue chasing the dream of human-level AI without having to worry about returning investor money.

类似的举措可能会帮助OpenAI和其他研究实验室继续追逐人类级AI的梦想，而不必担心会退还投资者的钱。

Originally published at https://www.pcmag.com.

最初发布在https://www.pcmag.com 。