How energy-intensive is the AI infrastructure today? And what does that mean for the future of discipline?

如今的AI基础架构有多耗能？ 这对学科的未来意味着什么？

Today’s AI algorithms, software and hardware combined are 10X to 100X more energy-intensive than they should be. In light of Microsoft’s recent announcement of its carbon negative commitment, my challenge to the industry is clear: let’s improve AI hardware and software so that we don’t overheat our planet.

如今的AI算法，软件和硬件相结合，比它们应消耗的能源消耗高出10到100倍。 鉴于微软最近宣布了其对碳减排的承诺 ，我对行业的挑战显而易见：让我们改进AI硬件和软件，以免我们的星球过热。

The computing industry is always optimizing for speed and innovation, but not necessarily considering the lifetime energy cost of that speed. I saw an inflection point around 2012 when the progression of AI hardware and algorithmic capabilities began to deviate from Moore’s law. Prior to that, most AI solutions were running on one, maybe two processors with workloads tracking to Moore’s law. A steady progression of workloads from the Perceptron in 1958 to systems like Bidirectional LSTM neural networks for speech recognition in the mid-2000s.

计算行业始终在针对速度和创新进行优化，但不一定要考虑该速度的生命周期能源成本。 我在2012年左右看到了一个转折点，当时AI硬件和算法功能的发展开始偏离摩尔定律。 在此之前，大多数AI解决方案都在一个(可能是两个)处理器上运行，其工作负载可跟踪摩尔定律。 从1958年的感知器到2000年代中期用于语音识别的双向LSTM神经网络等系统，工作量一直在稳步增长。

Training AI models with multiple GPUs changed everything. After Alex Krizhevsky and team designed the AlexNet model with two GPUs in 2012, the computing power and electrical energy involved in training AI models took off at an entirely different pace: over 100X compounding every two years. Theirs was certainly not the first Convolutional Neural Network (CNN), but their “SuperVision” entry swept the field, winning the 2012 ImageNet competition by a huge margin. The next year nearly all competitors used CNNs and trained with multiple processors!

用多个GPU训练AI模型改变了一切。 Alex Krizhevsky及其团队在2012年设计了带有两个GPU的AlexNet模型之后，用于训练AI模型的计算能力和电能以完全不同的速度发展：每两年复合增长100倍以上。 他们的肯定不是第一个卷积神经网络(CNN)，但是他们的“ SuperVision”入门席卷了整个领域，在2012年ImageNet竞赛中大获全胜。 第二年，几乎所有竞争对手都使用了CNN，并接受了多处理器的培训！

Fast forward to 2019, and quickly developing innovative neural networks for Natural Language Processing may require hundreds or thousands of distributed GPUs — like self-attention encoder-decoder models that employ Neural Architecture Search (NAS) methods. According to a recent University of Massachusetts Amherst study, the amount of CO2 emitted from energy generation plants to power the computation involved in creating a new state-of-the-art AI model, was the equivalent of five automobile lifetime’s worth of CO2 emissions. If that’s what it takes to train only one new AI model, you can see that it is just not compatible with prioritization of sustainability.

快进到2019年，快速开发用于自然语言处理的创新神经网络可能需要成百上千个分布式GPU，例如采用神经体系结构搜索(NAS)方法的自注意编码器-解码器模型。 根据马萨诸塞州大学阿默斯特分校最近的一项研究，能源工厂排放的二氧化碳量可为创建新的先进AI模型所需的计算提供动力，相当于汽车生命周期内的5吨二氧化碳排放量。 如果仅是训练一种新的AI模型而已，那么您会发现它与可持续性优先级不兼容。

I believe we can incentivize the AI industry to make a change in the overall lifetime energy budget for AI workloads, and identify startups that are already committed to this cause.

我相信我们可以激励AI行业改变AI工作负载的整个生命周期能源预算，并确定已经致力于这一事业的初创公司。

Where do you see the biggest opportunities for the highest impact energy savings?

您在哪里看到最大的节能影响最大的机会？

My colleagues and I think it’s joint optimization of three things: energy-efficient AI hardware, co-designed efficient AI algorithms and AI-aware computer networks.

我和我的同事认为这是对三件事的共同优化：高能效AI硬件，共同设计的高效AI算法和支持AI的计算机网络。

The challenge is that the energy consumption of AI models is likely the last thing an AI algorithm developer is thinking about (unless they’re focused on mobile phones). Usually the early optimizations are foremost around performance. AI engineers often think: “what’s my peak accuracy” and “how fast can I train the model” — both of which need faster computing and more energy.

挑战在于，AI模型的能耗可能是AI算法开发人员正在考虑的最后一件事(除非它们专注于手机)。 通常，早期的优化主要围绕性能。 AI工程师经常认为：“我的峰值精度是多少”和“我能训练模型的速度有多快”，这两者都需要更快的计算和更多的精力。

I support a new success metric to help incentivize the AI industry and startups to reduce energy and CO2 emissions at data center scale. We need to shift the focus to higher throughput and lower lifetime total cost of ownership of a system for given computing workloads. I stress “system” because often hardware marketing metrics forget to mention the energy cost of extra processors, memory, and networks required for an AI training system.

我支持一项新的成功指标，以帮助激励AI行业和初创企业减少数据中心规模的能源和二氧化碳排放量。 对于给定的计算工作负载，我们需要将重点转移到更高的吞吐量和更低的系统生命周期总拥有成本上。 我强调“系统”，因为通常硬件营销指标会忘记提及AI培训系统所需的额外处理器，内存和网络的能源成本。

Success Metric = Workload Throughput ÷ [ ($ Cost of System) + ($ Cost of Lifetime Energy of System) ]

成功指标=工作量吞吐量÷[((系统成本$)+(系统生命周期成本)]

Throughput measures how fast we can compute the required AI algorithms. In the phraseology of the late Harvard Business School Professor Clayton Christensen, workload throughput is the “job” that matters at the end of the day. Not peak Floating Point Operations Per Second (FLOPS) which are magical, mystical marketing numbers only loosely related to getting the computational “job” done.

吞吐量衡量我们可以多快地计算所需的AI算法。 用已故的哈佛商学院教授克莱顿·克里斯滕森(Clayton Christensen)的话说，工作量吞吐是一天中最重要的“工作”。 并非每秒浮点运算(FLOPS)的峰值，这是神奇的，神秘的营销数字，仅与完成计算“工作”紧密相关。

The denominator of this ratio is the computing hardware cost plus the lifetime energy cost of operating that hardware including cooling and any extra network and processors required.

该比率的分母是计算硬件成本加上运行该硬件的终生能源成本，包括冷却以及所需的任何额外网络和处理器。

With this new ratio, AI designers have far more degrees of freedom to optimize software, hardware and algorithms. For example, the power consumption of an AI chip itself — whether it is 50 watts or 450 watts — doesn’t matter as much. The lifetime energy consumption of many chips to deliver a certain workload throughput is what matters most. If we can maximize this success ratio, then by definition energy and CO2 emissions are reduced as well.

有了这个新的比率，AI设计师可以在更大程度上自由地优化软件，硬件和算法。 例如，AI芯片本身的功耗(无论是50瓦还是450瓦)都无关紧要。 最重要的是要交付一定的工作负载吞吐量的许多芯片的终生能耗。 如果我们可以最大程度地提高成功率，那么按照定义，能源和二氧化碳排放量也将减少。

Why change the “performance” mindset that has been the status quo for so long?

为什么要改变长期以来保持现状的“绩效”心态？

AI has an existential problem. As its models continue to get larger, more computationally complex, and more accuracy is desired to reach human performance levels, the energy required to train those models increases exponentially. At some point if things continue as they have, researchers won’t be able to get enough computers or energy to create the new AI algorithms we want.

人工智能存在一个问题。 随着其模型不断变得越来越大，计算更加复杂，并且需要更高的精度来达到人类的绩效水平，训练这些模型所需的能量成倍增加。 如果事情按原样继续进行，研究人员将无法获得足够的计算机或精力来创建我们想要的新AI算法。

I’m really worried about that potentially stalling AI innovation. Not many research labs can string together 4,000 leading-edge processors and run them for weeks. They just don’t have the resources to deploy exascale computers. So at some point — without change — we have the potential to reach a ceiling of innovation. I’d hate to see another AI winter.

我真的很担心AI创新可能会停滞不前。 没有多少研究实验室可以将4,000个领先的处理器组合在一起并运行数周。 他们只是没有资源来部署百亿亿次计算机。 因此，在某个时刻(无需更改)，我们有可能达到创新的上限。 我不想看到另一个AI冬季。

If our AI industry innovates around the success metric, then we will benefit from AI that is more compatible with sustainability, yet meets performance goals with lower lifetime energy hardware, more efficient AI algorithms and lower energy infrastructure. +

如果我们的AI行业围绕成功指标进行创新，那么我们将受益于与可持续性更兼容的AI，同时通过降低使用寿命的能源硬件，更高效的AI算法和更低的能源基础架构来满足性能目标。 +