决策报表服务器安装

by Chamath Kirinde

由查马斯·基林德(Chamath Kirinde)

无服务器流处理如何使决策更加容易 (How serverless stream processing will make decision-making easier)

About a year ago, we started being a part of the digital transformation with the first ever cloud-based IDE for serverless development. It was no cakewalk — we’ve been burning the candle at both ends trying to cover the majority from AWS’s serverless stack. Working with AWS Kinesis made me realise the beauty of serverless — of course, the exposure to streaming data with Kafka spared me some time going through the rudiments.

大约一年前 ，我们开始通过第一个基于云的IDE进行无服务器开发的数字化转型的一部分。 这不是小菜一碟，我们一直在尽力解决AWS的无服务器堆栈中的大部分问题。 与AWS Kinesis的合作使我体会到了无服务器的美妙之处 -当然，使用Kafka暴露流数据为我节省了一些时间。

TL; DR (TL;DR)

Did you ever wonder…

你有没有想过...

• How “Google Search” suggests things to you when you’re half-typing your query?

  当您对查询输入一半时， “ Google搜索”如何为您提供建议？

• How “Cheapest Airlines” start to appear everywhere after you searched for a country?

  搜索国家/地区后， “最便宜的航空公司”如何开始出现在任何地方？

• How online role-playing games adjust according to your decisions?在线角色扮演游戏如何根据您的决定进行调整？
• How gambling sites predict the odds of a live game?赌博网站如何预测直播游戏的几率？

• Why were Curry and Thompson benched while Portland was handing the Warriors their worst loss in a 73-win NBA season?

  为什么在波特兰让勇士队在73胜的NBA赛季中输给勇士队时， 库里和汤普森却坐在板凳上 ？

The power of real-time streaming data analytics is astonishing indeed. Now, since serverless technology is gaining some momentum, maybe you won’t have to worry about taking risky decisions on your own at all. This post covers the basics of “Serverless Streaming Data Processing” and how it will be an influential component of our decision making in the future.

实时流数据分析的功能确实令人惊讶。 现在，由于无服务器技术正在蓬勃发展，也许您完全不必担心自己做出冒险的决定。 这篇文章涵盖了“无服务器流数据处理”的基础知识，以及它将如何成为我们未来决策的有影响力的组成部分。

数据，无处不在 (Data, Data Everywhere)

Life is an endless series of events. The technology around us has made it a stream of digital actions emitting streams of data. If you turn back and investigate your life very carefully, you’ll see the never-ending string of data you have generated with your every digital action. It could be a lot to digest at first, but let’s explore some scenarios and try to find what applies to you and me.

生活是无尽的事件。 我们周围的技术使它成为发出数据流的数字动作流。 如果您回头并非常仔细地研究生活，您将看到在每次数字操作中生成的无休止的数据串。 一开始可能需要大量消化，但是让我们探索一些场景并尝试找到适合您和我的情况。

• Online banking and convenient e-commerce purchasing capabilities网上银行和便捷的电子商务购买功能
• Ride-sharing, modern-day traveling and transportation拼车，现代旅行和运输
• Industrial equipment and agricultural use cases like monitored machinery, autonomous tractors, and precision farming工业设备和农业用例，例如受监控的机械，自动拖拉机和精确耕作
• Automated power generation and smart grids, Zero-net Buildings, Smart metering自动化发电和智能电网，零网络建筑，智能计量
• Real-estate property recommendations based on geo-location, predictive maintenance基于地理位置和预测性维护的房地产建议
• Online dating and matchmaking relying on complex personality patterns and attribute distribution依靠复杂的个性模式和属性分布进行在线约会和配对

• Financial trading according to the real-time changes in the stock market, analytical risk management根据股票市场的实时变化进行金融交易，分析风险管理
• Movies, songs and other digital media with a better experience depending on the demographics, preference, and emotions根据人口统计，喜好和情感，具有更好体验的电影，歌曲和其他数字媒体
• Improved web and mobile application experience based on usage基于使用情况的改进的Web和移动应用程序体验
• Dynamic and personalised experiences in online gaming在线游戏的动态和个性化体验
• Enhanced social media experiences with hyper-personalisation and predictive analytics通过超个性化和预测分析增强社交媒体体验
• Telemetry from connected devices, or remote data centres from geospatial or spatial services like weather, resource assessment来自连接设备的遥测，或者来自地理空间或空间服务(例如天气，资源评估)的远程数据中心
• Sports analytics to enhance the players’ performance reducing health risks运动分析可增强运动员的表现，减少健康风险

All these events produce data — lots of it. Due to the frequency of this data emission, it has become an increasing burden to the digital space.

所有这些事件都会产生数据-很多。 由于这种数据发射的频率，它已成为数字空间越来越重的负担。

什么是流数据？ (What is Streaming Data?)

In a survey conducted last year about data, it’s estimated that with the current pace of data generation,

在去年进行的有关数据的调查中，据估计，按照当前数据生成的速度，

1.7 MB of data will be created every second for every person on earth by 2020

到2020年，全球每个人每秒将创建1.7 MB数据

Data that is poured out continuously by a gazillion sources every second has become a fact we can’t just ignore. Big Data discipline was an eye-opener for the tech world to apply this once irritating data to do something useful. This same irksome data is collected and analysed by a new species, namely data scientists ?. Due to the nature of continuity and often being in small sizes (order of Kilobytes) these data flows — usually referred by the moniker streaming data — are collected simultaneously as records and sent in for further processing.

数百万个源每秒不断涌出的数据已经成为我们不能忽略的事实。 大数据学科让科技界大开眼界，可以将这种曾经令人烦恼的数据用于有用的事情。 相同的令人讨厌的数据是由一个新物种(即数据科学家)收集和分析的。 由于连续性的性质，并且通常是小尺寸(千字节)，这些数据流(通常被称为绰号流数据)被同时收集为记录，并发送进去进行进一步处理。

从流处理到明智的决策 (From stream processing to smart decisions)

A streaming data processing structure is usually comprised of two layers — a storage layer and a processing layer. The former is responsible for ordering large streams of records and facilitating persistence and accessibility at high speeds. The processing layer takes care of data consumption, executing computations, and notifying the storage layer to get rid of already processed records. Data processing is done for each record incrementally or by matching over sliding time windows. Processed data is then subjected to streaming analytics operations and the derived information is used to make context-based decisions.

流数据处理结构通常由两层组成-存储层和处理层。 前者负责订购大量记录，并促进高速的持久性和可访问性。 处理层负责数据消耗，执行计算并通知存储层摆脱已处理的记录。 对每个记录进行增量处理或通过在滑动时间窗口内进行匹配来进行数据处理。 然后，对处理的数据进行流分析操作，并将派生的信息用于做出基于上下文的决策。

For instance, companies can track public sentiment changes on their products by analysing social media streams continuously. The world’s most influential nations can intervene in decisive events like presidential elections in other powerful countries. And mobile apps can offer personalised recommendations for products based on geo-location of devices and user emotions.

例如，公司可以通过持续分析社交媒体流来跟踪其产品上公众情绪的变化。 世界上最有影响力的国家可以干预诸如其他大国的总统选举之类的决定性事件 。 移动应用程序可以根据设备的地理位置和用户情绪为产品提供个性化推荐。

Most applications collect a portion of their data at the outset to produce simple summary reports and take simple decisions such as triggering alarms or calculating a moving average value. As time flies by, these become more and more sophisticated, and companies might want to access profound insights to perform intricate activities in turn with the aid of Machine Learning algorithms and data analysis techniques.

大多数应用程序从一开始就收集其一部分数据，以生成简单的摘要报告并做出简单的决定，例如触发警报或计算移动平均值。 随着时间的流逝，这些变得越来越复杂，并且公司可能希望借助机器学习算法和数据分析技术来获取深刻的见解，从而依次执行复杂的活动。

The continual growth of data has made data scientists work around the clock to come up with trailblazing solutions to utilise as much data as possible to fabricate alternate futures with better decisions.

数据的不断增长使数据科学家日以继夜地研究出开拓性的解决方案，以利用尽可能多的数据来构建具有更好决策的备用期货。

服务促进者 (Service Facilitators)

Adoption of the ideal cloud provider to fit organisational requirements can be overwhelming. However, all the major cloud service providers are equipped with competitive options to accommodate stream processing due to its ubiquitous impact. Here’s a list of commonly used serverless services to bolster enterprise-grade applications, highly relying on streaming data.

选择理想的云提供商来满足组织要求可能是压倒性的。 但是，由于其无处不在的影响，所有主要的云服务提供商都具备竞争性的选择来适应流处理。 这是高度依赖流数据的用于增强企业级应用程序的常用无服务器服务的列表。

现场例子 (Live Examples)

Many companies use insights from stream analytics to enhance the visibility of their businesses. This allows them to deliver customers a personalised experience. Additionally, near real-time transparency gives these firms the flexibility to promptly address emergencies.

许多公司使用来自流分析的见解来增强其业务的可见性。 这使他们能够为客户提供个性化的体验。 此外，近乎实时的透明度使这些公司可以灵活地及时处理紧急情况。

The emerging serverless architecture has driven all the leading cloud service platforms to present complementary solutions. Stream processing was made available for serverless application development with fully-managed, cloud-based services for real-time data processing over large Distributed Data Streams.

新兴的无服务器架构已驱动所有领先的云服务平台提供互补的解决方案。 流处理可用于无服务器应用程序开发，它具有完全托管的基于云的服务，可通过大型分布式数据流进行实时数据处理。

1.超个性化电视 (1. Hyper-personalised Television)

Netflix, the leading online television network in the world, developed a solution which centralises their flow logs using Amazon Kinesis Streams. As a system processing billions of traffic flows every day, this eliminates plenty of complexity for them because of the absence of a database in the architecture. Due to the high scalability and lightning speed, they can discover and address issues as they arise, and monitor the application on a massive scale.

Netflix是全球领先的在线电视网络，它开发了一种解决方案，可使用Amazon Kinesis Streams集中其流日志。 由于系统每天要处理数十亿流量，因此由于体系结构中没有数据库，因此为他们消除了很多复杂性。 由于具有很高的可扩展性和迅捷的速度，他们可以发现并解决出现的问题，并可以大规模监控应用程序。

With the upgraded recommendation algorithm, video transcoding, and licensing popular media, this subsequently grants a seamless experience to subscribers. With the exponential growth of subscribers, the company’s responsibilities increase by the day. However, nothing seems to be a problem for Netflix since they are considered to have a sound decision-making model.

借助升级的推荐算法，视频转码和流行媒体许可，这随后将为订户提供无缝体验。 随着订户的指数级增长，公司的职责每天都在增加。 但是，对于Netflix来说，似乎没有什么问题，因为它们被认为具有完善的决策模型 。

2.改善决策者的决策 (2. Improving the decisions of the decision makers)

As a leading source of integrated and intelligent information for businesses and professionals, Thomson Reuters provide their services to decision makers in a wide range of domains like financing and risk, science, legal, technology. This company built an in-house analytics engine to take full control of data and moved to AWS because they were familiar with its capabilities and scale.

作为为企业和专业人士提供集成和智能信息的领先来源，汤森路透(Thomson Reuters)为广泛领域的决策者提供服务，例如融资和风险，科学，法律，技术。 该公司构建了一个内部分析引擎来完全控制数据，并迁移到AWS，因为他们熟悉其功能和规模。

The new real-time pipeline attached to Amazon Kinesis stream produces better results in perceptive customer experience with accurate economic forecasts, financial trends for beneficiaries including a range of government activities.

附加到Amazon Kinesis流的新实时管道通过准确的经济预测，受益人的财务趋势(包括一系列政府活动)，在可感知的客户体验中产生更好的结果。

3.独角兽：交通拥堵的解决方案 (3. Unicorn: a solution to traffic congestion)

Jakarta has become a heavily congested city where the motorcycle has been deemed the most efficient mode of transport. To exploit this business opportunity, GO-JEK — one of the few unicorn businesses in Southeast Asia — started as a call centre for motorcycle taxi bookings. However, to meet the demand in exceeding expectations, the company had to consider expansion. Now with the support of Google Cloud Professional Services, the business architecture built on Cloud Dataflow for stream inference enables them to predict changes in demand effectively.

雅加达已成为一个拥挤不堪的城市，摩托车被认为是最有效的交通方式。 为了利用这一商机，GO-JEK(东南亚为数不多的独角兽企业之一)开始作为摩托车出租车预订的呼叫中心。 但是，为了满足超出预期的需求，该公司不得不考虑扩张。 现在，在Google Cloud Professional Services的支持下，基于Cloud Dataflow进行流推断的业务架构使他们能够有效地预测需求变化。

There are more stories about how cloud platforms like AWS, Google, Microsoft Azure, and IBM Cloud are exploited by companies to make their clients’ lives better and secure.

还有更多关于公司如何利用AWS ， Google ， Microsoft Azure和IBM Cloud之类的云平台来改善其客户的生活和安全性的故事。

无服务器流处理的局限性 (Limitations of Serverless Stream Processing)

Serverless stream processing is increasingly becoming a vital part of decision-making engines. However, with the current set of features, it’s not the ideal solution for some scenarios. Implementing real-time analytics for sliding windows and temporal event patterns is not a course for the faint-hearted.

无服务器流处理正日益成为决策引擎的重要组成部分。 但是，使用当前的功能集，对于某些情况来说，它并不是理想的解决方案。 对滑动窗口和时间事件模式实施实时分析不是胆小者的必修课。

The best way to assimilate never-ending data of this magnitude is through real-time dashboards which requires additional data organisation and persisting. These manoeuvres introduce undesirable latency and data management issues into the context. However, technology is evolving and trying to catch up to the speeds with integration using advanced cloud data management techniques to produce materialised views.

吸收如此巨大的无休止数据的最佳方法是通过实时仪表板，这需要额外的数据组织和持久性。 这些操作将不希望的等待时间和数据管理问题引入到上下文中。 但是，技术在不断发展，并试图通过使用高级云数据管理技术进行集成以生成实体化视图来跟上速度。

Stream Processing often uses a time-based or record-based window to be processed in contrast to the batch-based processing, which can lead to challenges in use cases that require query re-execution.

与基于批处理相比，流处理通常使用基于时间或基于记录的窗口来进行处理，这在需要重新执行查询的用例中会带来挑战。

Nowadays, application requirements grow beyond aggregated analytics. Increasing the window size seems to be an appropriate temporary solution, but it develops another intractable problem — Memory Management. Modern-day solutions usually provide advanced memory management and scheduling techniques to overcome this, but the world will see further improvements.

如今，应用程序需求已超出聚合分析的范围。 增大窗口大小似乎是一个适当的临时解决方案，但是这又带来了另一个棘手的问题-内存管理。 当今的解决方案通常提供先进的内存管理和调度技术来克服这一问题，但是世界将会看到进一步的改进。

结论 (Conclusion)

All in all, it’s apparent that serverless stream processing has been playing a prominent role around us without us even knowing. With the power of serverless data stream processing, applications can evolve from traditional batch processing to real-time analytics. The revelation of profound insights will result in effective decision making without having to manage infrastructure.

总而言之，很明显，无服务器流处理一直在我们周围发挥着重要作用，甚至我们都不知道。 借助无服务器数据流处理的强大功能，应用程序可以从传统的批处理演变为实时分析。 深刻见解的启示将导致无需管理基础架构即可做出有效的决策。

Even today, many organizations practise orthodox decision-making strategies based on the analytics derived using the big data clusters that belonged to THE PAST. New horizons of serverless and real-time data processing are now equipped with the power to make effective decisions and create a more productive, relevant, and most importantly secure world around you.

直到今天，许多组织仍在根据使用属于THE PAST的大数据集群得出的分析结果来实践正统的决策策略。 无服务器和实时数据处理的新视野现在具备了做出有效决策并创建您周围更高效，相关且最重要的安全世界的能力。

Will serverless stream processing make emotional decision making obsolete and computerized rational judgement the norm?

无服务器流处理将使情感决策过时和计算机化的理性判断成为规范吗？

What do you think?

你怎么看？

您现在应该怎么办？ (What should you do now?)

• Clap. Appreciate and let others find this article.

  拍。 感谢并让其他人找到这篇文章。

• Comment. Share your thoughts.

  评论。 分享你的意见。

• Follow me. Chamath Kirinde to receive updates on articles like this.

  跟着我。 Chamath Kirinde将收到有关此类文章的更新。

• Keep in touch. LinkedIn, Twitter, Chummy Charms

  保持联系。 LinkedIn ， Twitter ， 魅力十足

• Think Serverless. SLAppForge

  想想无服务器。 SLAppForge

Originally published at chummycharms.blogspot.com.

最初发布于chummycharms.blogspot.com 。

翻译自: https://www.freecodecamp.org/news/how-serverless-stream-processing-will-make-decision-making-easier-d929502b43c8/

决策报表服务器安装