面试系统设计_系统设计面试问题

面试系统设计

You may have heard the terms "Architecture" or "System Design." These come up a lot during developer job interviews – especially at big tech companies.

您可能已经听说过“架构”或“系统设计”这两个术语。在开发人员工作面试中，尤其是在大型科技公司中，这些问题很多。

This in-depth guide will help prepare you for the System Design interview, by teaching you basic software architecture concepts.

通过指导您基本的软件体系结构概念，本深入指南将帮助您为系统设计面试做准备。

This is not an exhaustive treatment, since System Design is a vast topic. But if you're a junior or mid-level developer, this should give you a strong foundation.

这不是详尽的处理方法，因为系统设计是一个广泛的主题。但是，如果您是初级或中级开发人员，这应该为您奠定坚实的基础。

From there, you can dig deeper with other resources. I've listed some of my favourite resources at the very bottom of this article.

从那里，您可以深入挖掘其他资源。我在本文的底部列出了一些我最喜欢的资源。

I've broken this guide into bite-sized chunks by topic and so I recommend you bookmark it. I've found spaced learning and repetition to be incredibly valuable tools to learn and retain information. And I've designed this guide to be chunked down into pieces that are easy to do spaced repetition with.

我已按主题将本指南分为几小部分，因此建议您将其添加为书签。我发现间隔学习和重复学习是学习和保留信息的极有价值的工具。而且我已将本指南设计为易于分解的小片段。

Section 1: Networks & Protocols (IP, DNS, HTTP, TCP etc)

第1部分：网络和协议(IP，DNS，HTTP，TCP等)
Section 2: Storage, Latency & Throughput

第2节：存储，延迟和吞吐量
Section 3: Availability

第3节：可用性
Section 4: Caching

第4节：缓存
Section 5: Proxies

第5节：代理
Section 6: Load Balancing

第6节：负载平衡
Section 7: Consistent Hashing

第7节：一致的哈希
Section 8: Databases

第8节：数据库
Section 9: Leader Election

第9节：领导人选举
Section 10: Polling, Streaming, Sockets

第10节：轮询，流式传输，套接字
Section 11: Endpoint Protection

第11节：端点保护
Section 12: Messages & Pub-Sub

第12节：消息和发布订阅
Section 13: Smaller Essentials

第13节：较小的必需品

Let's get started!

让我们开始吧！

第1节：网络和协议 (Section 1: Networks and Protocols)

"Protocols" is a fancy word that has a meaning in English totally independent of computer science. It means a system of rules and regulations that govern something. A kind of "official procedure" or "official way something must be done".

“协议”是一个花哨的单词，在英语中的含义完全独立于计算机科学。这意味着管理某些事物的规则和法规系统。一种“官方程序”或“必须完成某些事情的官方方式”。

For people to connect to machines and code that communicate with each other, they need a network over which such communication can take place. But the communication also needs some rules, structure, and agreed-upon procedures.

为了使人们连接到彼此通信的机器和代码，他们需要一个可以在其上进行通信的网络。但是沟通也需要一些规则，结构和商定的程序。

Thus, network protocols are protocols that govern how machines and software communicate over a given network. An example of a network is our beloved world wide web.

因此，网络协议是控制机器和软件如何通过给定网络进行通信的协议。网络的一个例子是我们钟爱的万维网。

You may have heard of the most common network protocols of the internet era - things like HTTP, TCP/IP etc. Let's break them down into basics.

您可能听说过Internet时代最常见的网络协议-例如HTTP，TCP / IP等。让我们将它们分解为基础。

IP- 互联网协议 (IP - Internet Protocol)

Think of this as the fundamental layer of protocols. It is the basic protocol that instructs us on how almost all communication across internet networks must be implemented.

将此视为协议的基本层。这是基本协议，可指导我们如何实现几乎所有跨Internet网络的通信。

Messages over IP are often communicated in "packets", which are small bundles of information (2^16 bytes). Each packet has an essential structure made up of two components: the Header and the Data.

IP上的消息通常以“数据包”形式进行通信，这些信息包是一小束信息(2 ^ 16字节)。每个数据包都有一个由两个部分组成的基本结构：标头和数据。

The header contains "meta" data about the packet and its data. This metadata includes information such as the IP address of the source (where the packet comes from) and the destination IP address (destination of the packet). Clearly, this is fundamental to being able to send information from one point to another - you need the "from" and "to" addresses.

标头包含有关数据包及其数据的“元”数据。该元数据包括诸如源(数据包来自何处)的IP地址和目标IP地址(数据包的目的地)之类的信息。显然，这是能够将信息从一个点发送到另一点的基础-您需要“从”和“到”地址。

And an IP Address is a numeric label assigned to each device connected to a computer network that uses the Internet Protocol for communication. There are public and private IP addresses, and there are currently two versions. The new version is called IPv6 and is increasingly being adopted because IPv4 is running out of numerical addresses.

IP地址是分配给连接到使用Internet协议进行通信的计算机网络的每个设备的数字标签。有公用和专用IP地址，当前有两个版本。新版本称为IPv6，由于IPv4的数字地址用完了，因此越来越多地被采用。

The other protocols we will consider in this post are built on top of IP, just like your favorite software language has libraries and frameworks built on top of it.

我们将在本文中考虑的其他协议是基于IP构建的，就像您最喜欢的软件语言具有基于IP构建的库和框架一样。

TCP- 传输控制协议 (TCP - Transmission Control Protocol)

TCP is a utility built on top of IP. As you may know from reading my posts, I firmly believe you need to understand why something was invented in order to truly understand what it does.

TCP是建立在IP之上的实用程序。正如您可以从阅读我的文章知道，我坚信你需要了解为什么有些为了被发明真正了解它做什么。

TCP was created to solve a problem with IP. Data over IP is typically sent in multiple packets because each packet is fairly small (2^16 bytes). Multiple packets can result in (A) lost or dropped packets and (B) disordered packets, thus corrupting the transmitted data. TCP solves both of these by guaranteeing transmission of packets in an ordered way.

创建TCP是为了解决IP问题。 IP上的数据通常以多个数据包发送，因为每个数据包都非常小(2 ^ 16字节)。多个数据包可能导致(A)数据包丢失或丢失，以及(B)无序的数据包，从而破坏了传输的数据。 TCP通过保证以有序方式传输数据包来解决这两种情况。

Being built on top of IP, the packet has a header called the TCP header in addition to the IP header. This TCP header contains information about the ordering of packets, and the number of packets and so on. This ensures that the data is reliably received at the other end. It is generally referred to as TCP/IP because it is built on top of IP.

数据包建立在IP之上，除IP报头外，还具有称为TCP报头的报头。此TCP头包含有关数据包顺序和数据包数量等信息。这样可以确保在另一端可靠地接收数据。由于它建立在IP之上，因此通常称为TCP / IP。

TCP needs to establish a connection between source and destination before it transmits the packets, and it does this via a "handshake". This connection itself is established using packets where the source informs the destination that it wants to open a connection, and the destination says OK, and then a connection is opened.

TCP需要先在源和目的地之间建立连接，然后再通过“握手”进行连接。此连接本身是使用数据包建立的，其中源将其通知目的地它想打开连接，并且目的地说“确定”，然后打开连接。

This, in effect, is what happens when a server "listens" at a port - just before it starts to listen there is a handshake, and then the connection is opened (listening starts). Similarly, one sends the other a message that it is about to close the connection, and that ends the connection.

实际上，这就是服务器在端口“侦听”时发生的情况-在它开始侦听之前，先握手，然后打开连接(侦听开始)。类似地，一个向另一个发送一条消息，即将关闭连接，并结束连接。

HTTP- 超文本传输协议 (HTTP - Hyper Text Transfer Protocol)

HTTP is a protocol that is an abstraction built on top of TCP/IP. It introduces a very important pattern called the request-response pattern, specifically for client-server interactions.

HTTP是一种基于TCP / IP的抽象协议。它引入了一个非常重要的模式，称为请求-响应模式，专门用于客户端-服务器交互。

A client is simply a machine or system that requests information, and a server is the machine or system that responds with information. A browser is a client, and a web-server is a server. When a server requests data from another server then the first server is also a client, and the second server is the server (I know, tautologies).

客户端只是请求信息的机器或系统，而服务器是响应信息的机器或系统。浏览器是客户端，而网络服务器是服务器。当服务器从另一台服务器请求数据时，第一台服务器也是客户端，第二台服务器是服务器(我知道是重言式)。

So this request-response cycle has its own rules under HTTP and this standardizes how information is transmitted across the internet.

因此，此请求-响应周期在HTTP下具有其自己的规则，这标准化了如何通过Internet传输信息。

At this level of abstraction we typically don't need to worry too much about IP and TCP. However, in HTTP, requests and responses have headers and bodies too, and these contain data that can be set by the developer.

在这种抽象级别上，我们通常不需要太担心IP和TCP。但是，在HTTP中，请求和响应也具有标头和正文，并且标头和正文包含开发人员可以设置的数据。

HTTP requests and responses can be thought of as messages with key-value pairs, very similar to objects in JavaScript and dictionaries in Python, but not the same.

HTTP请求和响应可以被视为具有键-值对的消息，与JavaScript中的对象和Python中的字典非常相似，但并不相同。

Below is an illustration of the content, and key-value pairs in HTTP request and response messages.

下图说明了HTTP请求和响应消息中的内容和键值对。

HTTP also comes with some "verbs" or "methods" which are commands that give you an idea of what sort of operation is intended to be performed. For example, the common HTTP methods are "GET", "POST", "PUT", "DELETE" and "PATCH", but there are more. In the above picture, look for the HTTP verb in the start line.

HTTP还带有一些“动词”或“方法”，这些命令是使您了解打算执行哪种操作的命令。例如，常见的HTTP方法是“ GET”，“ POST”，“ PUT”，“ DELETE”和“ PATCH”，但还有更多方法。在上图中，在起始行中查找HTTP动词。

第2节：存储，延迟和吞吐量 (Section 2: Storage, Latency & Throughput)

存储 (Storage)

Storage is about holding information. Any app, system, or service that you program will need to store and retrieve data, and those are the two fundamental purposes of storage.

存储是关于保存信息的。您编程的任何应用程序，系统或服务都需要存储和检索数据，而这是存储的两个基本目的。

But it's not just about storing data – it's also about fetching it. We use a database to achieve this. A database is a software layer that helps us store and retrieve data.

但这不只是存储数据，还在于获取数据。我们使用数据库来实现这一目标。数据库是帮助我们存储和检索数据的软件层。

These two primary types of operations, storing and retrieving, are also variously called 'set, get', 'store, fetch', 'write, read' and so on. To interact with storage, you will need to go through the database, which acts as an intermediary for you to conduct these fundamental operations.

这两种主要的操作类型(存储和检索)也分别称为“设置，获取”，“存储，获取”，“写入，读取”等。要与存储进行交互，您将需要遍历数据库，该数据库充当您执行这些基本操作的中介。

The word "storage" can sometimes fool us into thinking about it in physical terms. If I "store" my bike in the shed, I can expect it to be there when I next open the shed.

“存储”一词有时会使我们从物理角度上去思考它。如果我将自行车“存放”在棚子里，那么我下次打开棚子时可以期望它在那里。

But that doesn't always happen in the computing world. Storage can broadly be of two types: "Memory" storage and "Disk" storage.

但这并不总是在计算世界中发生。存储大致可以分为两种类型：“内存”存储和“磁盘”存储。

Of these two, the disk storage tends to be the more robust and "permanent" (not truly permanent, so we often use the word "persistent" storage instead). Disk storage is persistent storage. This means that when you save something to Disk, and turn the power off, or restart your server, that data will "persist". It won't be lost.

在这两个磁盘中，磁盘存储趋向于更健壮和“永久”(不是真正的永久，因此我们经常使用“永久”存储一词)。磁盘存储是持久性存储。这意味着，当您将某些内容保存到磁盘，然后关闭电源或重新启动服务器时，该数据将“持续存在”。它不会丢失。

However, if you leave data in "Memory" then that usually gets wiped away when you shut down or restart, or otherwise lose power. The computer you use everyday has both these storage types. Your hard disk is "persistent" Disk storage, and your RAM is transient Memory storage.

但是，如果将数据保留在“内存”中，那么通常在关闭或重新启动时这些数据会消失，否则会断电。您每天使用的计算机都具有这两种存储类型。您的硬盘是“永久性”磁盘存储，而您的RAM是暂时性的内存存储。

On servers, if the data you're keeping track of is only useful during a session of that server, then it makes sense to keep it in Memory. This is much faster and less expensive than writing things to a persistent database.

在服务器上，如果您跟踪的数据仅在该服务器的会话期间有用，则将其保留在内存中是有意义的。与将内容写入持久数据库相比，这要快得多且成本更低。

For example, a single session may mean when a user is logged in and using your site. After they log out, you may not need to hold on to bits of data that you collected during the session.

例如，一个会话可能意味着用户登录并使用您的网站。他们注销后，您可能不需要保留在会话期间收集的部分数据。

But whatever you do want to hold on to (like shopping cart history) you will put in persistent Disk storage. That way you can access that data the next time the user logs in, and they will have a seamless experience.

但是，无论您想保留什么(例如购物车历史记录)，您都将放入永久性磁盘存储中。这样，您下次用户登录时就可以访问该数据，他们将获得无缝的体验。

Ok, so this seems quite simple and basic, and it's meant to be. This is a primer. Storage can get very complex. If you take a look at the range of storage products and solutions your head will spin.

好的，这看起来非常简单和基本，而且确实如此。这是入门。存储会变得非常复杂。如果您看一下存储产品和解决方案的范围，您的头会旋转。

This is because different use-cases require different types of storage. They key to choosing the right storage types for your system depends on a lot of factors and the needs of your application, and how users interact with it. Other factors include:

这是因为不同的用例需要不同的存储类型。它们为系统选择正确的存储类型的关键取决于许多因素，应用程序的需求以及用户与之交互的方式。其他因素包括：

the shape (structure) of your data, or
数据的形状(结构)，或
what sort of availability it needs (what level of downtime is OK for your storage), or
它需要什么样的可用性(哪种水平的停机时间适合您的存储)，或者
scalability (how fast do you need to read and write data, and will these reads and writes happen concurrently (simultaneously) or sequentially) etc, or
可伸缩性(您需要多快读写数据，这些读写将同时(同时)或顺序进行)等，或
consistency - if you protect against downtime using distributed storage, then how consistent is the data across your stores?
一致性-如果您使用分布式存储来防止停机，那么整个存储中的数据的一致性如何？

These questions and the conclusions require you to consider your trade-offs carefully. Is consistency more important than speed? Do you need the database to service millions of operations per minute or only for nightly updates? I will be dealing with these concepts in sections later, so don't worry if you've no idea what they are.

这些问题和结论要求您仔细考虑您的取舍。一致性比速度重要吗？您需要该数据库每分钟执行数百万次操作还是仅用于每晚更新？我将在后面的部分中处理这些概念，所以如果您不知道它们是什么，请不要担心。

潜伏 (Latency)

"Latency" and "Throughput" are terms you're going to hear a lot of as you start to get more experienced with designing systems to support the front end of your application. They are very fundamental to the experience and performance of your application and the system as a whole. There is often a tendency to use these terms in a broader sense than intended, or out of context, but let's fix that.

当您开始在设计系统以支持应用程序的前端方面获得更多经验时，您会经常听到“延迟”和“吞吐量”这两个术语。它们对于您的应用程序和整个系统的体验和性能至关重要。通常倾向于以比预期更广泛的含义使用这些术语，或者出于上下文考虑，但让我们对其进行修复。

Latency is simply the measure of a duration. What duration? The duration for an action to complete something or produce a result. For example: for data to move from one place in the system to another. You may think of it as a lag, or just simply the time taken to complete an operation.

延迟只是持续时间的度量。什么时间？完成某件事或产生结果的动作的持续时间。例如：数据从系统中的一个位置移动到另一个位置。您可能会认为这是一个滞后，或者仅仅是完成操作所花费的时间。

The most commonly understood latency is the "round trip" network request - how long does it take for your front end website (client) to send a query to your server, and get a response back from the server.

最常见的延迟是“往返”网络请求-前端网站(客户端)向服务器发送查询并从服务器返回响应需要多长时间。

When you're loading a site, you want this to be as fast and as smooth as possible. In other words you want low latency. Fast lookups means low latency. So finding a value in an array of elements is slower (higher latency, because you need to iterate over each element in the array to find the one you want) than finding a value in a hash-table (lower latency, because you simply look up the data in "constant" time , by using the key. No iteration needed.).

加载网站时，您希望它尽可能快和流畅。换句话说，你想低延迟 。快速查找意味着低延迟。因此，在元素数组中查找值比在哈希表中查找值要慢(延迟较高，因为您需要遍历数组中的每个元素以查找所需的元素)(延迟较低，因为您只是看一下通过使用密钥在“固定”时间内更新数据(无需迭代)。

Similarly, reading from memory is much faster than reading from a disk (read more here). But both have latency, and your needs will determine which type of storage you pick for which data.

同样，从内存中读取要比从磁盘中读取快得多( 在此处了解更多)。但是两者都有延迟，您的需求将决定您选择哪种数据的存储类型。

In that sense, latency is the inverse of speed. You want higher speeds, and you want lower latency. Speed (especially on network calls like via HTTP) is determined also by the distance. So, latency from London to another city, will be impacted by the distance from London.

从这个意义上讲，延迟是速度的倒数。您想要更高的速度，并且想要更低的延迟。速度(尤其是通过HTTP进行的网络呼叫)的速度也取决于距离。因此，从伦敦到另一个城市的延迟时间将受到与伦敦的距离的影响。

As you can imagine, you want to design a system to avoid pinging distant servers, but then storing things in memory may not be feasible for your system. These are the tradeoffs that make system design complex, challenging and extremely interesting!

可以想像，您希望设计一个系统来避免对远程服务器执行ping操作，但是对于系统而言，将内容存储在内存中可能并不可行。这些折衷使系统设计变得复杂，具有挑战性并且非常有趣！

For example, websites that show news articles may prefer uptime and availability over loading speed, whereas online multiplayer games may require availability and super low latency. These requirements will determine the design and investment in infrastructure to support the system's special requirements.

例如，显示新闻报道的网站可能更喜欢正常运行时间和可用性，而不是加载速度，而在线多人游戏可能需要可用性和超低延迟。这些要求将确定支持系统特殊要求的基础结构的设计和投资。

通量 (Throughput)

This can be understood as the maximum capacity of a machine or system. It's often used in factories to calculate how much work an assembly line can do in an hour or a day, or some other unit of time measurement.

这可以理解为机器或系统的最大容量。它通常在工厂中用于计算装配线在一小时或一天之内可以完成的工作量，或其他时间单位。

For example an assembly line can assemble 20 cars per hour, which is its throughput. In computing it would be the amount of data that can be passed around in a unit of time. So a 512 Mbps internet connection is a measure of throughput - 512 Mb (megabits) per second.

例如，一条装配线每小时可以装配20辆汽车，这就是它的吞吐量。在计算中，它是单位时间内可以传递的数据量。因此512 Mbps互联网连接是吞吐量的度量-每秒512 Mb(兆位)。

Now imagine freeCodeCamp's web-server. If it receives 1 million requests per second, and can serve only 800,000 requests, then its throughput is 800,000 per second. You may end up measuring the throughput in terms of bits instead of requests, so it would be N bits per second.

现在想象一下freeCodeCamp的Web服务器。如果它每秒接收一百万个请求，并且只能处理800,000个请求，则其吞吐量为每秒80万。您可能最终以比特而不是请求来衡量吞吐量，因此它将是每秒N位。

In this example, there is a bottleneck because the server cannot handle more than N bits a second, but the requests are more than that. A bottleneck is therefore the constraint on a system. A system is only as fast as its slowest bottleneck.

在此示例中，存在瓶颈，因为服务器每秒处理的位数不能超过N个，但是请求的位数不止于此。因此，瓶颈是对系统的约束。一个系统的速度只有最慢的瓶颈。

If one server can handle 100 bits per second, and another can handle 120 bits per second and a third can handle only 50, then the overall system will be operating at 50bps because that is the constraint - it holds up the speed of the other servers in a given system.

如果一台服务器每秒可以处理100位，另一台服务器每秒可以处理120位，而第三台服务器只能处理50位，那么整个系统将以50bps的速度运行，因为这是一个限制-它可以保持其他服务器的速度在给定的系统中。

So increasing throughput anywhere other than the bottleneck may be a waste - you may want to just increase throughput at the lowest bottleneck first.

因此，增加瓶颈以外的任何地方的吞吐量可能都是浪费–您可能只想首先在最低瓶颈处增加吞吐量 。

You can increase throughput by buying more hardware (horizontal scaling) or increasing the capacity and performance of your existing hardware (vertical scaling) or a few other ways.

您可以通过购买更多硬件(水平缩放)或增加现有硬件的容量和性能(垂直缩放)或其他几种方法来提高吞吐量。

Increasing throughput may sometimes be a short term solution, and so a good systems designer will think through the best ways to scale the throughput of a given system including by splitting up requests (or any other form of "load"), and distributing them across other resources etc. The key point to remember is what throughput is, what a constraint or bottleneck is, and how it impacts a system.

有时，增加吞吐量可能是一个短期解决方案，因此，优秀的系统设计人员将考虑通过最佳方法来扩展给定系统的吞吐量，包括拆分请求(或任何其他形式的“负载”)，并将其分布在各个请求上。需要记住的关键点是什么是吞吐量，什么是约束或瓶颈以及它如何影响系统。

Fixing latency and throughput are not isolated, universal solutions by themselves, nor are they correlated to each other. They have impacts and considerations across the system, so it's important to understand the system as a whole, and the nature of the demands that will be placed on the system over time.

固定延迟和吞吐量不是孤立的，通用的解决方案，也不是相互关联的。它们对整个系统都有影响和考虑因素，因此了解整个系统以及随时间推移对系统提出的需求的性质非常重要。

第3节：系统可用性 (Section 3: System Availability)

Software engineers aim to build systems that are reliable. A reliable system is one that consistently satisfies a user's needs, whenever that user seeks to have that need satisfied. A key component of that reliability is Availability.

软件工程师旨在构建可靠的系统。可靠的系统是一个始终满足用户需求的系统，只要该用户想要满足该需求。这种可靠性的关键组成部分是可用性。

It's helpful to think of availability as the resiliency of a system. If a system is robust enough to handle failures in the network, database, servers etc, then it can generally be considered to be a fault-tolerant system - which makes it an available system.

将可用性视为系统的弹性很有帮助。如果系统足够健壮，可以处理网络，数据库，服务器等中的故障，则通常可以将其视为容错系统-这使其成为可用系统。

Of course, a system is a sum of its parts in many senses, and each part needs to be highly available if availability is relevant to the end user experience of the site or app.

当然，从多种意义上讲，系统是其各个部分的总和，如果可用性与站点或应用程序的最终用户体验有关，则每个部分都必须具有高可用性。

量化可用性 (Quantifying Availability)

To quantify the availability of a system, we calculate the percentage of time that the system's primary functionality and operations are available (the uptime) in a given window of time.

为了量化系统的可用性，我们在给定的时间范围内计算系统的主要功能和操作可用的时间百分比(正常运行时间)。

The most business-critical systems would need to have a near-perfect availability. Systems that support highly variable demands and loads with sharp peaks and troughs may be able to get away with slightly lower availability during off-peak times.

最关键的业务系统将需要具有接近完美的可用性。在高峰时段，支持高变化需求和尖峰和低谷负载的系统可能会以较低的可用性摆脱困境。

It all depends on the use and nature of the system. But in general, even things that have low, but consistent demands or an implied guarantee that the system is "on-demand" would need to have high availability.

这完全取决于系统的用途和性质。但是总的来说，即使是那些需求较低但始终如一或隐含保证系统“按需”的事物，也需要具有高可用性。

Think of a site where you backup your pictures. You don't always need to access and retrieve data from this - it's mainly for you to store things in. You would still expect it to always be available any time you login to download even just a single picture.

想想一个备份图片的网站。您并非总是需要从中访问和检索数据-主要是用于存储内容。您仍然希望它在每次登录下载任何图片时始终可用。

A different kind of availability can be understood in the context of the massive e-commerce shopping days like Black Friday or Cyber Monday sales. On these particular days demand will skyrocket and millions will try to access the deals simultaneously. That would require an extremely reliable and high-availability system design to support those loads.

在黑色星期五或网络星期一销售这样的大型电子商务购物日中，可以理解另一种可用性。在这些特定日子里，需求将激增，数以百万计的人将尝试同时访问交易。这将需要极其可靠和高可用性的系统设计来支持这些负载。

A commercial reason for high availability is simply that any downtime on the site will result in the site losing money. Also, it could be really bad for reputation, for example, where the service is a service used by other businesses to offer services. If AWS S3 goes down, a lot of companies will suffer, including Netflix, and that is not good.

高可用性的商业原因仅仅是因为该站点上的任何停机都将导致该站点亏本。此外，这可能对声誉确实不利，例如，该服务是其他企业用来提供服务的服务。如果AWS S3出现故障，包括Netflix在内的许多公司将遭受损失，这不是很好 。

So uptimes are extremely important for success. It is worth remembering that commercial availability numbers are calculated based on annual availability, so a downtime of 0.1% (i.e. availability of 99.9%) is 8.77 hours a year!

因此，正常运行时间对于成功至关重要。值得记住的是，商业可用性数字是根据年度可用性计算得出的，因此0.1％(即99.9％的可用性)的停机时间是每年8.77个小时！

Hence, uptimes are extremely high sounding. It is common to see things like 99.99% uptime (52.6 minutes of downtime per year). Which is why it is now common to refer to uptimes in terms of "nines" - the number of nines in the uptime assurance.

因此，正常运行时间非常高。通常会看到99.99％的正常运行时间(每年52.6分钟的停机时间)。这就是为什么现在通常用“ nines”(正常运行时间保证中的九位数)来指称正常运行时间。

In today's world that is unacceptable for large-scale or mission critical services. Which is why these days "five nines" is considered the ideal availability standard because that translates to a little over 5 minutes of downtime per year.

在当今世界上，大规模或关键任务服务是不可接受的。这就是为什么现在“五个九”被认为是理想的可用性标准的原因，因为这相当于每年停机时间超过5分钟。

服务水平协议 (SLAs)

In order to make online services competitive and meet the market's expectations, online service providers typically offer Service Level Agreements/Assurances. These are a set of guaranteed service level metrics. 99.999% uptime is one such metric and is often offered as part of premium subscriptions.

为了使在线服务具有竞争力并满足市场的期望，在线服务提供商通常会提供服务水平协议/保证。这些是一组保证的服务级别指标。 99.999％的正常运行时间就是这样一种指标，通常作为高级订阅的一部分提供。

In the case of database and cloud service providers this can be offered even on the trial or free tiers if a customer's core use for that product justifies the expectation of such a metric.

对于数据库和云服务提供商，如果客户对该产品的核心使用证明了该指标的合理性，则即使在试用或免费套餐上也可以提供此服务。

In many cases failing to meet the SLA will give the customer a right to credits or some other form of compensation for the provider's failure to meet that assurance. Here, by way of example, is Google's SLA for the Maps API.

在许多情况下，如果未能满足SLA，则客户将获得信用证或其他形式的补偿，以弥补提供商未达到该保证的权利。举例来说，这里是Google针对Maps API的SLA。

SLAs are therefore a critical part of the overall commercial and technical consideration when designing a system. It is especially important to consider whether availability is in fact a key requirement for a part of a system, and which parts require high availability.

因此，在设计系统时，SLA是整个商业和技术考虑因素的关键部分。特别重要的是考虑可用性是否实际上是系统某个部分的关键要求，以及哪些部分需要高可用性。

设计HA (Designing HA)

When designing a high availability (HA) system, then, you need to reduce or eliminate "single points of failure". A single point of failure is an element in the system that is the sole element that can produce that undesirable loss of availability.

因此，在设计高可用性(HA)系统时，您需要减少或消除“单点故障”。单点故障是系统中的一个元素，它是唯一可能导致不希望的可用性损失的元素。

You eliminate single points of failure by designing 'redundancy' into the system. Redundancy is basically making 1 or more alternatives (i.e. backups) to the element that is critical for high availability.

您可以通过在系统中设计“冗余”来消除单点故障。冗余基本上是对对高可用性至关重要的元素进行1个或多个选择(即备份)。

So if your app needs users to be authenticated to use it, and there is only one authentication service and back end, and that fails, then, because that is the single point of failure, your system is no longer usable. By having two or more services that can handle authentication, you have added redundancy and eliminated (or reduced) single points of failure.

因此，如果您的应用需要用户进行身份验证才能使用它，并且只有一个身份验证服务和后端，并且该服务失败，则因为这是单点故障，所以您的系统不再可用。通过拥有两个或多个可以处理身份验证的服务，您增加了冗余并消除了(或减少了)单点故障。

Therefore, you need to understand and de-compose your system into all its parts. Map out which ones are likely to cause single points of failure, which ones are not tolerant of such failure, and which parts can tolerate them. Because engineering HA requires tradeoffs and some of these tradeoffs may be expensive in terms of time, money and resources.

因此，您需要了解系统并将其分解为所有部分。找出哪些可能导致单点故障，哪些不能容忍这种故障，哪些部分可以容忍它们。由于工程HA需要权衡，并且其中一些权衡在时间，金钱和资源方面可能是昂贵的。

第4节：缓存 (Section 4: Caching)

Caching! This is a very fundamental and easy-to-understand technique to speed up performance in a system. Thus caching helps to reduce "latency" in a system.

正在缓存！这是提高系统性能的非常基础且易于理解的技术。因此，缓存有助于减少系统中的“等待时间” 。

In our daily lives, we use caching as a matter of common-sense (most of the time...). If we live next door to a supermarket, we still want to buy and store some basics in our fridge and our food cupboard. This is caching. We could always step out, go next door, and buy these things every time we want food – but if its in the pantry or fridge, we reduce the time it takes to make our food. That's caching.

在我们的日常生活中，我们将缓存作为常识(在大多数情况下...)。如果我们住在超市的隔壁，我们仍然想在冰箱和食物柜中购买和存储一些基本物品。这是缓存。每当我们想要食物时，我们总是可以出门，去隔壁并购买这些东西-但是，如果它在食品室或冰箱中，我们会减少制作食物的时间。那是缓存。

常见的缓存方案 (Common Scenarios for Caching)

Similarly, in software terms, if we end up relying on certain pieces of data often, we may want to cache that data so that our app performs faster.

同样，就软件而言，如果我们最终经常依赖某些数据，则可能需要缓存该数据，以便我们的应用程序执行得更快。

This is often true when it's faster to retrieve data from Memory rather than disk because of the latency in making network requests. In fact many websites are cached (especially if content doesn't change frequently) in CDNs so that it can be served to the end user much faster, and it reduces load on the backend servers.

当由于发出网络请求的延迟而从内存而不是从磁盘检索数据更快时，通常会如此。实际上，许多网站都在CDN中缓存(尤其是内容不经常更改的情况)，以便可以更快地将其提供给最终用户，并减少了后端服务器的负载。

Another context in which caching helps could be where your backend has to do some computationally intensive and time consuming work. Caching previous results that converts your lookup time from a linear O(N) time to constant O(1) time could be very advantageous.

缓存可以帮助您解决的另一个问题是后端必须执行一些计算量大且耗时的工作。缓存将您的查找时间从线性O(N)时间转换为恒定O(1)时间的先前结果可能会非常有优势。

Likewise, if your server has to make multiple network requests and API calls in order to compose the data that gets sent back to the requester, then caching data could reduce the number of network calls, and thus the latency.

同样，如果您的服务器必须进行多个网络请求和API调用才能组成发送回请求者的数据，则缓存数据可以减少网络调用的次数，从而减少延迟。

If your system has a client (front end), and a server and databases (backend) then caching can be inserted on the client (e.g. browser storage), between the client and the server (e.g. CDNs), or on the server itself. This would reduce over-the-network calls to the database.

如果您的系统具有客户端(前端)，服务器和数据库(后端)，则可以在客户端(例如浏览器存储)，客户端和服务器之间(例如CDN)或服务器本身上插入缓存。这将减少对数据库的网络调用。

So caching can occur at multiple points or levels in the system, including at the hardware (CPU) level.

因此，缓存可以在系统中的多个点或级别上发生，包括在硬件(CPU)级别上。

处理陈旧数据 (Handling Stale Data)

You may have noticed that the above examples are implicitly handy for "read" operations. Write operations are not that different, in main principles, with the following added considerations:

您可能已经注意到，上面的示例对于“读取”操作是隐式方便的。写入操作在主要原则上并没有什么不同，但还要增加以下注意事项：

write operations require keeping the cache and your database in sync
写操作需要保持缓存和数据库同步
this may increase complexity because there are more operations to perform, and new considerations around handling un-synced or "stale" data need to be carefully analyzed
这可能会增加复杂性，因为要执行更多的操作，并且需要仔细分析有关处理未同步或“过时”数据的新注意事项
new design principles may need to be implemented to handle that syncing - should it be done synchronously, or asynchronously? If async, then at what intervals? Where does data get served from in the mean time? How often does the cache need to be refreshed, etc...
可能需要实施新的设计原则来处理该同步-应该同步还是异步完成？如果异步，那么以什么间隔？在此期间从何处获取数据？缓存需要多久刷新一次，等等。
data "eviction" or turnover and refreshes of data, to keep cached data fresh and up-to-date. These include techniques like LIFO, FIFO, LRU and LFU.

数据“逐出”或更新和刷新数据，以使缓存的数据保持最新和最新状态。这些包括LIFO ， FIFO ， LRU和LFU之类的技术。

So let's end with some high-level, and non-binding conclusions. Generally, caching works best when used to store static or infrequently changing data, and when the sources of change are likely to be single operations rather than user-generated operations.

因此，让我们以一些概括性的，非约束性的结论作为结尾。通常，当用于存储静态或不经常更改的数据时，并且当更改的源可能是单个操作而不是用户生成的操作时，缓存效果最佳。

Where consistency and freshness in data is critical, caching may not be an optimal solution, unless there is another element in the system that efficiently refreshes the caches are intervals that do not adversely impact the purpose and user experience of the application.

在数据的一致性和新鲜度至关重要的情况下，缓存可能不是最佳的解决方案，除非系统中有另一个有效刷新缓存的元素是不会对应用程序的目的和用户体验产生不利影响的间隔。

第5节：代理 (Section 5: Proxies)

Proxy. What? Many of us have heard of proxy servers. We may have seen configuration options on some of our PC or Mac software that talk about adding and configuring proxy servers, or accessing "via a proxy".

代理。什么？我们许多人都听说过代理服务器。我们可能已经在某些PC或Mac软件上看到了配置选项，这些选项讨论添加和配置代理服务器或“通过代理”访问。

So let's understand that relatively simple, widely used and important piece of tech. This is a word that exists in the English language completely independent of computer science, so let's start with that definition.

因此，让我们了解一下相对简单，广泛使用且重要的技术。这是一个完全独立于计算机科学的英语单词，因此让我们从该定义开始。

Now you can eject most of that out of your mind, and hold on to one key word: "substitute".

现在，您可以将大部分内容从头脑中弹出，然后按住一个关键字：“替换”。

In computing, a proxy is typically a server, and it is a server that acts as a middleman between a client and another server. It literally is a bit of code that sits between client and server. That's the crux of proxies.

在计算中，代理通常是服务器，并且它是充当客户端和另一台服务器之间的中间人的服务器。从字面上看，这是位于客户端和服务器之间的一些代码。这就是代理的症结所在。

In case you need a refresher, or aren't sure of the definitions of client and server, a "client" is a process (code) or machine that requests data from another process or machine (the "server"). The browser is a client when it requests data from a backend server.

如果您需要复习，或者不确定客户端和服务器的定义，则“客户端”是从另一个进程或计算机(“服务器”)请求数据的进程(代码)或计算机。当浏览器从后端服务器请求数据时，它是客户端。

The server serves the client, but can also be a client - when it retrieves data from a database. Then the database is the server, the server is the client (of the database) and also a server for the front-end client (browser).

服务器为客户端提供服务，但也可以是客户端-从数据库检索数据时。然后，数据库是服务器，服务器是(数据库的)客户端，也是前端客户端(浏览器)的服务器。

As you can see from the above, the client-server relationship is bi-directional. So one things can be both the client and server. If there was a middleman server that received requests, then sent them to another service, then forwards the response it got from that other service back to the originator client, that would be a proxy server.

从上面可以看到，客户端-服务器关系是双向的。因此，客户端和服务器都是一回事。如果有一个中间人服务器接收到请求，然后将它们发送到另一个服务，然后将它从另一个服务获得的响应转发回原始客户端，即代理服务器。

Going forward we will refer to clients as clients, servers as servers and proxies as the thing between them.

展望未来，我们将客户端称为客户端，将服务器称为服务器，将代理称为它们之间的事物。

So when a client sends a request to a server via the proxy, the proxy may sometimes mask the identity of the client - to the server, the IP address that comes through in the request may be the proxy and not the originating client.

因此，当客户端通过代理向服务器发送请求时，代理有时会掩盖客户端的身份-到服务器时，请求中通过的IP地址可能是代理而不是原始客户端。

For those of you who access sites or download things that otherwise are restricted (from the torrent network for example, or sites banned in your country), you may recognize this pattern - it's the principle on which VPNs are built.

对于那些访问网站或下载其他限制内容的人(例如从torrent网络或您所在国家/地区禁止的网站)，您可能会意识到这种模式-这是建立VPN的原理。

Before we move a bit deeper, I want to call something out - when generally used, the term proxy refers to a "forward" proxy. A forward proxy is one where the proxy acts on behalf of (substitute for) the client in the interaction between client and server.

在深入探讨之前，我想先介绍一下-通常使用的术语“代理”是指“转发”代理。转发代理是指代理在客户端与服务器之间的交互中代表客户端(代替客户端)进行操作的代理。

This is distinguished from a reverse proxy - where the proxy acts on behalf of a server. On a diagram it would look the same - the proxy sits between the client and the server, and the data flows are the same client <-> proxy <-> server.

这与反向代理不同-反向代理代表服务器。在图中，它看起来是一样的-代理位于客户端和服务器之间，并且数据流是相同的客户端<->代理<->服务器。

The key difference is that a reverse proxy is designed substitute for the server. Often clients won't even know that the network request got routed through a proxy and the proxy passed it on to the intended server (and did the same thing with the server's response).

关键区别在于反向代理被设计为替代服务器。通常，客户甚至都不知道网络请求是通过代理路由的，并且代理将其传递给了预期的服务器(并且对服务器的响应做了同样的事情)。

So, in a forward proxy, the server won't know that the client's request and its response are traveling through a proxy, and in a reverse proxy the client won't know that the request and response are routed through a proxy.

因此，在正向代理中，服务器将不知道客户端的请求及其响应正在通过代理，而在反向代理中，客户端将不知道请求和响应是通过代理进行路由的。

Proxies feel kinda sneaky :)

代理感到有点偷偷摸摸:)

But in systems design, especially for complex systems, proxies are useful and reverse proxies are particularly useful. Your reverse proxy can be delegated a lot of tasks that you don't want your main server handling - it can be a gatekeeper, a screener, a load-balancer and an all around assistant.

但是在系统设计中，特别是对于复杂系统，代理是有用的，反向代理是特别有用的。您的反向代理可以委派许多您不希望主服务器处理的任务-它可以是网守，筛选器，负载平衡器和全方位助手。

So proxies can be useful but you may not be sure why. Again, if you've read my other stuff you'd know that I firmly believe that you can understand things properly only when you know why they exist - knowing what they do is not enough.

因此代理可能有用，但您可能不确定原因。同样，如果你读过我的其他的东西，你知道，我坚信你能正确地只了解事情的时候，你知道他们为什么存在-知道他们做的是不够的。

We've talked about VPNs (for forward proxies) and load-balancing (for reverse proxies), but there are more examples here - I particularly recommend Clara Clarkson's high level summary.

我们已经谈到的VPN(用于转发代理)和负载平衡(反向代理服务器)，但也有更多的例子在这里 -我特别推荐克拉拉Clarkson的高度概括。

第6节：负载平衡 (Section 6: Load Balancing)

If you think about the two words, load and balance, you will start to get an intuition as to what this does in the world of computing. When a server simultaneously receives a lot of requests, it can slow down (throughput reduces, latency rises). After a point it may even fail (no availability).

如果您考虑一下负载和平衡这两个词，您将开始对它在计算领域的作用有一个直觉。当服务器同时接收到大量请求时，它可能会减慢速度(吞吐量降低，延迟增加)。在一个点之后，它甚至可能失败(无可用性)。

You can give the server more muscle power (vertical scaling) or you can add more servers (horizontal scaling). But now you got to work out how the income requests get distributed to the various servers - which requests get routed to which servers and how to ensure they don't get overloaded too? In other words, how do you balance and allocate the request load?

您可以赋予服务器更多的力量(垂直缩放)，也可以添加更多服务器(水平缩放)。但是现在您必须弄清楚收入请求如何分配到各个服务器-哪些请求被路由到哪些服务器以及如何确保它们也不会过载？换句话说，如何平衡和分配请求负载？

Enter load balancers. Since this article is an introduction to principles and concepts, they are, of necessity, very simplified explanations. A load balancer's job is to sit between the client and server (but there are other places it can be inserted) and work out how to distribute incoming request loads across multiple servers, so that the end user (client's) experience is consistently fast, smooth and reliable.

输入负载均衡器。由于本文是对原理和概念的介绍，因此它们在必要时必须经过非常简化的解释。负载平衡器的工作是坐在客户端和服务器之间(但可以在其他位置插入它)，并研究如何在多个服务器之间分配传入请求负载，以便最终用户(客户端)的体验始终快速，流畅和可靠。

So load balancers are like traffic managers who direct traffic. And they do this to maintain availability and throughput.

因此，负载均衡器就像是引导流量的流量管理器。他们这样做是为了保持可用性和吞吐量。

When understanding where a load balancer is inserted in the system's architecture, you can see that load balancers can be thought of as reverse proxies. But a load balancer can be inserted in other places too - between other exchanges - for example, between your server and your database.

了解负载均衡器在系统体系结构中的插入位置后，您会发现负载均衡器可以被视为反向代理。但是负载平衡器也可以插入其他位置，例如在其他交换机之间，例如在服务器和数据库之间。

平衡法-服务器选择策略 (The Balancing Act - Server Selection Strategies)

So how does the load balancer decide how to route and allocate request traffic? To start with, every time you add a server, you need to let your load balancer know that there is one more candidate for it to route traffic to.

那么，负载均衡器如何决定如何路由和分配请求流量？首先，每次添加服务器时，都需要让负载均衡器知道它还有一个候选者可以将流量路由到该负载均衡器。

If you remove a server, the load balancer needs to know that too. The configuration ensures that the load balancer knows how many servers it has in its go-to list and which ones are available. It is even possible for the load balancer to be kept informed on each server's load levels, status, availability, current task and so on.

如果卸下服务器，则负载平衡器也需要知道这一点。该配置可确保负载均衡器知道其转到列表中有多少台服务器以及哪些服务器可用。甚至可以使负载平衡器随时了解每台服务器的负载级别，状态，可用性，当前任务等。

Once the load balancer is configured to know what servers it can redirect to, we need to work out the best routing strategy to ensure there is proper distribution amongst the available servers.

将负载均衡器配置为知道可以重定向到哪些服务器后，我们需要制定出最佳路由策略，以确保在可用服务器之间进行适当的分配。

A naive approach to this is for the load balancer to just randomly pick a server and direct each incoming request that way. But as you can imagine, randomness can cause problems and "unbalanced" allocations where some servers get more loaded than others, and that could affect performance of the overall system negatively.

一个简单的方法是让负载均衡器随机选择一个服务器，然后以这种方式定向每个传入的请求。但是正如您可以想象的那样，随机性可能会导致问题和“不平衡”分配，其中某些服务器的负载要比其他服务器更多，这可能会对整个系统的性能产生负面影响。

循环赛和加权循环赛 (Round Robin and Weighted Round Robin)

Another method that can be intuitively understood is called "round robin". This is the way many humans process lists that loop. You start at the first item in the list, move down in sequence, and when you're done with the last item you loop back up to the top and start working down the list again.

可以直观理解的另一种方法称为“循环”。这是许多人处理列表循环的方式。您从列表中的第一项开始，依次向下移动，当最后一项完成后，您将循环回到顶部，然后再次开始处理列表。

The load balancer can do this too, by just looping through available servers in a fixed sequence. This way the load is pretty evenly distributed across your servers in a simple-to-understand and predictable pattern.

负载均衡器也可以做到这一点，只需按固定顺序遍历可用服务器即可。这样，负载就以一种易于理解和可预测的模式在服务器上平均分配。

You can get a little more "fancy" with the round robin by "weighting" some services over others. In the normal, standard round robin, each server is given equal weight (let's say all are given a weighting of 1). But when you differently weight servers, then you can have some servers with a lower weighting (say 0.5, if they're less powerful), and others can be higher like 0.7 or 0.9 or even 1.

通过将某些服务“加权”于其他服务，可以使轮询变得更“花哨”。在正常的标准轮询中，每个服务器的权重相等(假设所有服务器的权重均为1)。但是，当您对服务器加权时，可以使某些服务器的权重较低(例如，如果它们的功能较弱，则为0.5)，而其他服务器的权重可能更高，例如0.7或0.9甚至是1。

Then the total traffic will be split up in proportion to those weights and allocated accordingly to the servers that have power proportionate to the volume of requests.

然后，总流量将按这些权重成比例分配，并相应地分配给功率与请求量成正比的服务器。

基于负载的服务器选择 (Load-based server selection)

More sophisticated load balancers can work out the current capacity, performance, and loads of the servers in their go-to list and allocate dynamically according to current loads and calculations as to which will have the highest throughput, lowest latency etc. It would do this by monitoring the performance of each server and deciding which ones can and cannot handle the new requests.

更复杂的负载均衡器可以在其go-to列表中计算出服务器的当前容量，性能和负载，并根据当前负载和计算动态分配，从而得出吞吐量最高，延迟最低等。 by monitoring the performance of each server and deciding which ones can and cannot handle the new requests.

IP Hashing based selection (IP Hashing based selection)

You can configure your load balancer to hash the IP address of incoming requests, and use the hash value to determine which server to direct the request too. If I had 5 servers available, then the hash function would be designed to return one of five hash values, so one of the servers definitely gets nominated to process the request.

IP hash based routing can be very useful where you want requests from a certain country or region to get data from a server that is best suited to address the needs from within that region, or where your servers cache requests so that they can be processed fast.

In the latter scenario, you want to ensure that the request goes to a server that has previously cached the same request, as this will improve speed and performance in processing and responding to that request.

If your servers each maintain independent caches and your load balancer does not consistently send identical requests to the same server, you will end up with servers re-doing work that has already been done in as previous request to another server, and you lose the optimization that goes with caching data.

Path or Service based selection (Path or Service based selection)

You can also get the load balancer to route requests based on their "path" or function or service that is being provided. For example if you're buying flowers from an online florist, requests to load the "Bouquets on Special" may be sent to one server and credit card payments may be sent to another server.

If only one in twenty visitors actually bought flowers, then you could have a smaller server processing the payments and a bigger one handling all the browsing traffic.

Mixed Bag (Mixed Bag)

And as with all things, you can get to higher and more detailed levels of complexity. You can have multiple load balancers that each have different server selection strategies! And if yours is a very large and highly trafficked system, then you may need load balancers for load balancers...

Ultimately, you add pieces to the system until your performance is tuned to your needs (your needs may look flat, or slow upwards mildly over time, or be prone to spikes!).

Section 7: Consistent Hashing (Section 7: Consistent Hashing)

One of the slightly more tricky concepts to understand is hashing in the context of load balancing. So it gets its own section.

In order to understand this, please first understand how hashing works at a conceptual level. The TL;DR is that hashing converts an input into a fixed-size value, often an integer value (the hash).

In order to understand this, please first understand how hashing works at a conceptual level . The TL;DR is that hashing converts an input into a fixed-size value, often an integer value (the hash).

One of the key principles for a good hashing algorithm or function is that the function must be deterministic, which is a fancy way for saying that identical inputs will generate identical outputs when passed into the function. So, deterministic means - if I pass in the string "Code" (case sensitive) and the function generates a hash of 11002, then every time I pass in "Code" it must generate "11002" as an integer. And if I pass in "code" it will generate a different number (consistently).

One of the key principles for a good hashing algorithm or function is that the function must be deterministic , which is a fancy way for saying that identical inputs will generate identical outputs when passed into the function. So, deterministic means - if I pass in the string "Code" (case sensitive) and the function generates a hash of 11002, then every time I pass in "Code" it must generate "11002" as an integer. And if I pass in "code" it will generate a different number (consistently).

Sometimes the hashing function can generate the same hash for more than one input - this is not the end of the world and there are ways to deal with it. In fact it becomes more likely the more the range of unique inputs are. But when more than one input deterministically generates the same output, it's called a "collision".

With this in firmly in mind, let's apply it to routing and directed requests to servers. Let's say you have 5 servers to allocate loads across. An easy to understand method would be to hash incoming requests (maybe by IP address, or some client detail), and then generate hashes for each request. Then you apply the modulo operator to that hash, where the right operand is the number of servers.

For example, this is what your load balancers' pseudo code could look like:

request#1 => hashes to 34
request#2 => hashes to 23
request#3 => hashes to 30
request#4 => hashes to 14// You have 5 servers => [Server A, Server B ,Server C ,Server D ,Server E]// so modulo 5 for each request...request#1 => hashes to 34 => 34 % 5 = 4 => send this request to servers[4] => Server Erequest#2 => hashes to 23 => 23 % 5 = 3 => send this request to servers[3] => Server Drequest#3 => hashes to 30 => 30 % 5 = 0 => send this request to  servers[0] => Server Arequest#4 => hashes to 14 => 14 % 5 = 4 => send this request to servers[4] => Server E

As you can see, the hashing function generates a spread of possible values, and when the modulo operator is applied it brings out a smaller range of numbers that map to the server number.

You will definitely get different requests that map to the same server, and that's fine, as long as there is "uniformity" in the overall allocation to all the servers.

You will definitely get different requests that map to the same server, and that's fine, as long as there is " uniformity " in the overall allocation to all the servers.

Adding Servers, and Handling Failing Servers (Adding Servers, and Handling Failing Servers)

So - what happens if one of the servers that we are sending traffic to dies? The hashing function (refer to the pseudo code snippet above) still thinks there are 5 servers, and the mod operator generates a range from 0-4. But we only have 4 servers now that one has failed, and we are still sending it traffic. Oops.

Inversely, we could add a sixth server but that would never get any traffic because our mod operator is 5, and it will never yield a number that would include the newly added 6th server. Double oops.

// Let's add a 6th server
servers => [Server A, Server B ,Server C ,Server D ,Server E, Server F]// let's change the modulo operand to 6
request#1 => hashes to 34 => 34 % 6 = 4 => send this request to servers[4] => Server Erequest#2 => hashes to 23 => 23 % 6 = 5 => send this request to servers[5] => Server Frequest#3 => hashes to 30 => 30 % 6 = 0 => send this request to  servers[0] => Server Arequest#4 => hashes to 14 => 14 % 6 = 2 => send this request to servers[2] => Server C

We note that the server number after applying the mod changes (though, in this example, not for request#1 and request#3 - but that is just because in this specific case the numbers worked out that way).

In effect, the result is that half the requests (could be more in other examples!) are now being routed to new servers altogether, and we lose the benefits of previously cached data on the servers.

For example, request#4 used to go to Server E, but now goes to Server C. All the cached data relating to request#4 sitting on Server E is of no use since the request is now going to Server C. You can calculate a similar problem for where one of your servers dies, but the mod function keeps sending it requests.

It sounds minor in this tiny system. But on a very large scale system this is a poor outcome. #SystemDesignFail.

So clearly, a simple hashing-to-allocate system does not scale or handle failures well.

A popular solution - consistent hashing (A popular solution - consistent hashing)

Unfortunately this is the part where I feel word descriptions will not be enough. Consistent hashing is best understood visually. But the purpose of this post so far is to give you an intuition around the problem, what it is, why it arises, and what the shortcomings in a basic solution might be. Keep that firmly in mind.

The key problem with naive hashing, as we discussed, is that when (A) a server fails, traffic still gets routed to it, and (B) you add a new server, the allocations can get substantially changed, thus losing the benefits of previous caches.

There are two very important things to keep in mind when digging into consistent hashing:

Consistent hashing does not eliminate the problems, especially B. But it does reduce the problems a lot. At first you might wonder what the big deal is in consistent hashing, as the underlying downside still exists - yes, but to a much smaller extent, and that itself is a valuable improvement in very large scale systems.

Consistent hashing does not eliminate the problems , especially B. But it does reduce the problems a lot. At first you might wonder what the big deal is in consistent hashing, as the underlying downside still exists - yes, but to a much smaller extent, and that itself is a valuable improvement in very large scale systems.
Consistent hashing applies a hash function to incoming requests and the servers. The resulting outputs therefore fall in a set range (continuum) of values. This detail is very important.

Consistent hashing applies a hash function to incoming requests and the servers . The resulting outputs therefore fall in a set range (continuum) of values. This detail is very important.

Please keep these in mind as you watch the below recommended video that explains consistent hashing, as otherwise its benefits may not be obvious.

I strongly recommend this video as it embeds these principles without burdening you with too much detail.

If you're having a little trouble really understanding why this strategy is important in load balancing, I suggest you take a break, then return to the load balancing section and then re-read this again. It's not uncommon for all this to feel very abstract unless you've directly encountered the problem in your work!

Section 8: Databases (Section 8: Databases)

We briefly considered that there are different types of storage solutions (databases) designed to suit a number of different use-cases, and some are more specialized for certain tasks than others. At a very high level though, databases can be categorized into two types: Relational and Non-Relational.

Relational Databases (Relational Databases)

A relational database is one that has strictly enforced relationships between things stored in the database. These relationships are typically made possible by requiring the database to represented each such thing (called the "entity") as a structured table - with zero or more rows ("records", "entries") and and one or more columns ("attributes, "fields").

By forcing such a structure on an entity, we can ensure that each item/entry/record has the right data to go with it. It makes for better consistency and the ability to make tight relationships between the entities.

You can see this structure in the table recording "Baby" (entity) data below. Each record ("entry) in the table has 4 fields, which represent data relating to that baby. This is a classic relational database structure (and a formalized entity structure is called a schema).

So the key feature to understand about relational databases is that they are highly structured, and impose structure on all the entities. This structure in enforced by ensuring that data added to the table conforms to that structure. Adding a height field to the table when its schema doesn't allow for it will not be permitted.

Most relational databases support a database querying language called SQL - Structured Query Language. This is a language specifically designed to interact with the contents of a structured (relational) database. The two concepts are quite tightly coupled, so much so that people often referred to a relational database as a "SQL database" (and sometimes pronounced as "sequel" database).

Most relational databases support a database querying language called SQL - Structured Query Language . This is a language specifically designed to interact with the contents of a structured (relational) database. The two concepts are quite tightly coupled, so much so that people often referred to a relational database as a "SQL database" (and sometimes pronounced as "sequel" database).

In general, it is considered that SQL (relational) databases support more complex queries (combining different fields and filters and conditions) than non-relational databases. The database itself handles these queries and sends back matching results.

Many people who are SQL database fans argue that without that function, you would have to fetch all the data and then have the server or the client load that data "in memory" and apply the filtering conditions - which is OK for small sets of data but for a large, complex dataset, with millions of records and rows, that would badly affect performance. However, this is not always the case, as we will see when we learn about NoSQL databases.

Many people who are SQL database fans argue that without that function, you would have to fetch all the data and then have the server or the client load that data " in memory " and apply the filtering conditions - which is OK for small sets of data but for a large, complex dataset, with millions of records and rows, that would badly affect performance. However, this is not always the case, as we will see when we learn about NoSQL databases.

A common and much-loved example of a relational database is the PostgreSQL (often called "Postgres") database.

ACID (ACID)

ACID transactions are a set of features that describe the transactions that a good relational database will support. ACID = "Atomic, Consistent, Isolation, Durable". A transaction is an interaction with a database, typically read or write operations.

ACID transactions are a set of features that describe the transactions that a good relational database will support. ACID = "Atomic, Consistent, Isolation, Durable" . A transaction is an interaction with a database, typically read or write operations.

Atomicity requires that when a single transaction comprises of more than one operation, then the database must guarantee that if one operation fails the entire transaction (all operations) also fail. It's "all or nothing". That way if the transaction succeeds, then on completion you know that all the sub-operations completed successfully, and if an operation fails, then you know that all the operations that went with it failed.

For example if a single transaction involved reading from two tables and writing to three, then if any one of those individual operations fails the entire transaction fails. This means that none of those individual operations should complete. You would not want even 1 out of the 3 write transactions to work - that would "dirty" the data in your databases!

Consistency requires that each transaction in a database is valid according to the database's defined rules, and when the database changes state (some information has changed), such change is valid and does not corrupt the data. Each transaction moves the database from one valid state to another valid state. Consistency can be thought of as the following: every "read" operation receives the most recent "write" operation results.

Isolation means that you can "concurrently" (at the same time) run multiple transactions on a database, but the database will end up with a state that looks as though each operation had been run serially ( in a sequence, like a queue of operations). I personally think "Isolation" is not a very descriptive term for the concept, but I guess ACCD is less easy to say than ACID...

Durability is the promise that once the data is stored in the database, it will remain so. It will be "persistent" - stored on disk and not in "memory".

Durability is the promise that once the data is stored in the database, it will remain so. It will be " persistent " - stored on disk and not in "memory".

Non-relational databases (Non-relational databases)

In contrast, a non-relational database has a less rigid, or, put another way, a more flexible structure to its data. The data typically is presented as "key-value" pairs. A simple way of representing this would be as an array (list) of "key-value" pair objects, for example:

// baby names
[{ name: "Jacob",rank: ##,gender: "M",year: ####},{ name: "Isabella",rank: ##,gender: "F",year: ####},{//...},// ...
]

Non relational databases are also referred to as "NoSQL" databases, and offer benefits when you do not want or need to have consistently structured data.

Similar to the ACID properties, NoSQL database properties are sometimes referred to as BASE:

Basically Available which states that the system guarantees availability

Soft State mean means the state of the system may change over time, even without input

Eventual Consistency states that the system will become consistent over a (very short) period of time unless other inputs are received.

Since, at their core, these databases hold data in a hash-table-like structure, they are extremely fast, simple and easy to use, and are perfect for use cases like caching, environment variables, configuration files and session state etc. This flexibility makes them perfect for using in memory (e.g. Memcached) and also in persistent storage (e.g. DynamoDb).

Since, at their core, these databases hold data in a hash-table-like structure, they are extremely fast, simple and easy to use, and are perfect for use cases like caching, environment variables, configuration files and session state etc. This flexibility makes them perfect for using in memory (eg Memcached ) and also in persistent storage (eg DynamoDb ).

There are other "JSON-like" databases called document databases like the well-loved MongoDb, and at the core these are also "key-value" stores.

There are other "JSON-like" databases called document databases like the well-loved MongoDb , and at the core these are also "key-value" stores.

Database Indexing (Database Indexing)

This is a complicated topic so I will simply skim the surface for the purpose of giving you a high level overview of what you need for systems design interviews.

Imagine a database table with 100 million rows. This table is used mainly to look up one or two values in each record. To retrieve the values for a specific row you would need to iterate over the table. If it's the very last record that would take a long time!

Indexing is a way of short cutting to the record that has matching values more efficiently than going through each row. Indexes are typically a data structure that is added to the database that is designed to facilitate fast searching of the database for those specific attributes (fields).

So if the census bureau has 120 million records with names and ages, and you most often need to retrieve lists of people belonging to an age group, then you would index that database on the age attribute.

Indexing is core to relational databases and is also widely offered on non-relational databases. The benefits of indexing are thus available in theory for both types of databases, and this is hugely beneficial to optimise lookup times.

Replication and Sharding (Replication and Sharding)

While these may sound like things out of a bio-terrorism movie, you're more likely to hear them everyday in the context of database scaling.

Replication means to duplicate (make copies of, replicate) your database. You may remember that when we discussed availability.

Replication means to duplicate (make copies of, replicate) your database. You may remember that when we discussed availability .

We had considered the benefits of having redundancy in a system to maintain high availability. Replication ensures redundancy in the database if one goes down. But it also raises the question of how to synchronize data across the replicas, since they're meant to have the same data. Replication on write and update operations to a database can happen synchronously (at the same time as the changes to the main database) or asynchronously .

The acceptable time interval between synchronising the main and a replica database really depends on your needs - if you really need state between the two databases to be consistent then the replication needs to be rapid. You also want to ensure that if the write operation to the replica fails, the write operation to the main database also fails (atomicity).

But what do you do when you've got so much data that simply replicating it may solve availability issues but does not solve throughput and latency issues (speed)?

At this point you may want to consider "chunking down" your data, into "shards". Some people also call this partitioning your data (which is different from partitioning your hard drive!).

Sharding data breaks your huge database into smaller databases. You can work out how you want to shard your data depending on its structure. It could be as simple as every 5 million rows are saved in a different shard, or go for other strategies that best fit your data, needs and locations served.

Section 9: Leader Election (Section 9: Leader Election)

Let's move back to servers again for a slightly more advanced topic. We already understand the principle of Availability, and how redundancy is one way to increase availability. We have also walked through some practical considerations when handling the routing of requests to clusters of redundant servers.

Let's move back to servers again for a slightly more advanced topic. We already understand the principle of Availability , and how redundancy is one way to increase availability. We have also walked through some practical considerations when handling the routing of requests to clusters of redundant servers.

But sometimes, with this kind of setup where multiple servers are doing much the same thing, there can arise situations where you need only one server to take the lead.

For example, you want to ensure that only one server is given the responsibility for updating some third party API because multiple updates from different servers could cause issues or run up costs on the third-party's side.

In this case you need to choose that primary server to delegate this update responsibility to. That process is called leader election.

In this case you need to choose that primary server to delegate this update responsibility to. That process is called leader election .

When multiple servers are in a cluster to provide redundancy, they could, amongst themselves, be configured to have one and only one leader. They would also detect when that leader server has failed, and appoint another one to take its place.

The principle is very simple, but the devil is in the details. The really tricky part is ensuring that the servers are "in sync" in terms of their data, state and operations.

There is always the risk that certain outages could result in one or two servers being disconnected from the others, for example. In that case, engineers end up using some of the underlying ideas that are used in blockchain to derive consensus values for the cluster of servers.

In other words, a consensus algorithm is used to give all the servers an "agreed on" value that they can all rely on in their logic when identifying which server is the leader.

Leader Election is commonly implemented with software like etcd, which is a store of key-value pairs that offers both high availability and strong consistency (which is valuable and an unusual combination) by using Leader Election itself and using a consensus algorithm.

Leader Election is commonly implemented with software like etcd , which is a store of key-value pairs that offers both high availability and strong consistency (which is valuable and an unusual combination) by using Leader Election itself and using a consensus algorithm.

So engineers can rely on etcd's own leader election architecture to produce leader election in their systems. This is done by storing in a service like etcd, a key-value pair that represents the current leader.

Since etcd is highly available and strongly consistent, that key-value pair can always be relied on by your system to contain the final "source of truth" server in your cluster is the current elected leader.

Section 10: Polling, Streaming, Sockets (Section 10: Polling, Streaming, Sockets)

In the modern age of continuous updates, push notifications, streaming content and real-time data, it is important to grasp the basic principles that underpin these technologies. To have data in your application updated regularly or instantly requires the use of one of the two following approaches.

Polling (Polling)

This one is simple. If you look at the wikipedia entry you may find it a bit intense. So instead take a look at its dictionary meaning, especially in the context of computer science. Keep that simple fundamental in mind.

Polling is simply having your client "check" send a network request to your server and asking for updated data. These requests are typically made at regular intervals like 5 seconds, 15 seconds, 1 minute or any other interval required by your use case.

Polling every few seconds is still not quite the same as real-time, and also comes with the following downsides, especially if you have a million plus simultaneous users:

almost-constant network requests (not great for the client)
almost-constant network requests (not great for the client)
almost constant inbound requests (not great for the server loads - 1 million+ requests per second!)
almost constant inbound requests (not great for the server loads - 1 million+ requests per second!)

So polling rapidly is not really efficient or performant, and polling is best used in circumstances when small gaps in data updates is not a problem for your application.

For example, if you built an Uber clone, you may have the driver-side app send driver location data every 5 seconds, and your rider-side app poll for the driver's location every 5 seconds.

流媒体 (Streaming)

Streaming solves the constant polling problem. If constantly hitting the server is necessary, then it's better to use something called web-sockets.

Streaming solves the constant polling problem. If constantly hitting the server is necessary, then it's better to use something called web-sockets .

This is a network communication protocol that is designed to work over TCP. It opens a two-way dedicated channel (socket) between a client and server, kind of like an open hotline between two endpoints.

Unlike the usual TCP/IP communication, these sockets are "long-lived" so that its a single request to the server that opens up this hotline for the two-way transfer of data, rather than multiple separate requests. By long-lived, we meant that the socket connection between the machines will last until either side closes it, or the network drops.

You may remember from our discussion on IP, TCP and HTTP that these operate by sending "packets" of data, for each request-response cycle. Web-sockets mean that there is a single request-response interaction (not a cycle really if you think about it!) and that opens up the channel through which two-data is sent in a "stream".

The big difference with polling and all "regular" IP based communication is that whereas polling has the client making requests to the server for data at regular intervals ("pulling" data), in streaming, the client is "on standby" waiting for the server to "push" some data its way. The server will send out data when it changes, and the client is always listening for that. Hence, if the data change is constant, then it becomes a "stream", which may be better for what the user needs.

For example, while using collaborative coding IDEs, when either user types something, it can show up on the other, and this is done via web-sockets because you want to have real-time collaboration. It would suck if what I typed showed up on your screen after you tried to type the same thing or after 3 minutes of you waiting wondering what I was doing!

For example, while using collaborative coding IDEs , when either user types something, it can show up on the other, and this is done via web-sockets because you want to have real-time collaboration. It would suck if what I typed showed up on your screen after you tried to type the same thing or after 3 minutes of you waiting wondering what I was doing!

Or think of online, multiplayer games - that is a perfect use case for streaming game data between players!

To conclude, the use case determines the choice between polling and streaming. In general, you want to stream if your data is "real-time", and if it's OK to have a lag (as little as 15 seconds is still a lag) then polling may be a good option. But it all depends on how many simultaneous users you have and whether they expect the data to be instantaneous. A commonly used example of a streaming service is Apache Kafka.

Section 11: Endpoint Protection (Section 11: Endpoint Protection)

When you build large scale systems it becomes important to protect your system from too many operations, where such operations are not actually needed to use the system. Now that sounds very abstract. But think of this - how many times have you clicked furiously on a button thinking it's going to make the system more responsive? Imagine if each one of those button clicks pinged a server and the server tried to process them all! If the throughput of the system is low for some reason (say a server was struggling under unusual load) then each of those clicks would have made the system even slower because it has to process them all!

Sometimes it's not even about protecting the system. Sometimes you want to limit the operations because that is part of your service. For example, you may have used free tiers on third-party API services where you're only allowed to make 20 requests per 30 minute interval. if you make 21 or 300 requests in a 30 minute interval, after the first 20, that server will stop processing your requests.

That is called rate-limiting. Using rate-limiting, a server can limit the number of operations attempted by a client in a given window of time. A rate-limit can be calculated on users, requests, times, payloads, or other things. Typically, once the limit is exceeded in a time window, for the rest of that window the server will return an error.

Ok, now you might think that endpoint "protection" is an exaggeration. You're just restricting the users ability to get something out of the endpoint. True, but it is also protection when the user (client) is malicious - like say a bot that is smashing your endpoint. Why would that happen? Because flooding a server with more requests than it can handle is a strategy used by malicious folks to bring down that server, which effectively brings down that service. That's exactly what a Denial of Service (D0S) attack is.

While DoS attacks can be defended against in this way, rate-limiting by itself won't protect you from a sophisticated version of a DoS attack - a distributed DoS. Here distribution simply means that the attack is coming from multiple clients that seem unrelated and there is no real way to identify them as being controlled by the single malicious agent. Other methods need to be used to protect against such coordinated, distributed attacks.

But rate-limiting is useful and popular anyway, for less scary use-cases, like the API restriction one I mentioned. Given how rate-limiting works, since the server has to first check the limit conditions and enforce them if necessary, you need to think about what kind of data structure and database you'd want to use to make those checks super fast, so that you don't slow down processing the request if it's within allowed limits. Also, if you have it in-memory within the server itself, then you need to be able to guarantee that all requests from a given client will come to that server so that it can enforce the limits properly. To handle situations like this it's popular to use a separate Redis service that sits outside the server, but holds the user's details in-memory, and can quickly determine whether a user is within their permitted limits.

Rate limiting can be made as complicated as the rules you want to enforce, but the above section should cover the fundamentals and most common use-cases.

Section 12: Messaging & Pub-Sub (Section 12: Messaging & Pub-Sub)

When you design and build large-scale and distributed systems, for that system to work cohesively and smoothly, it is important to exchange information between the components and services that make up the system. But as we have seen before, systems that rely on networks suffer from the same weakness as networks - they are fragile. Networks fail and its not an infrequent occurrence. When networks fail, components in the system are not able to communicate may degrade the system (best case) or cause the system to fail altogether (worst case). So distributed systems need robust mechanisms to ensure that the communication continues or recovers where it left off, even if there is an "arbitrary partition" (i.e. failure) between components in the system.

When you design and build large-scale and distributed systems , for that system to work cohesively and smoothly, it is important to exchange information between the components and services that make up the system. But as we have seen before, systems that rely on networks suffer from the same weakness as networks - they are fragile. Networks fail and its not an infrequent occurrence. When networks fail, components in the system are not able to communicate may degrade the system (best case) or cause the system to fail altogether (worst case). So distributed systems need robust mechanisms to ensure that the communication continues or recovers where it left off, even if there is an "arbitrary partition" (ie failure) between components in the system.

Imagine, as an example, that you're booking airline tickets. You get a good price, choose your seats, confirm the booking and you've even paid using your credit card. Now you're waiting for your ticket PDF to arrive in your inbox. You wait, and wait, and it never comes. Somewhere, there was a system failure that didn't get handled or recover properly. A booking system will often connect with airline and pricing APIs to handle the actual flight selection, fare summary, date and time of flight etc. All that gets done while you click through the site's booking UI. But it doesn't have to send you the PDF of the tickets until a few minutes later. Instead the UI can simply confirm that your booking is done, and you can expect the tickets in your inbox shortly. That's a reasonable and common user experience for bookings because the moment of paying and the receipt of the tickets does not have to be simultaneous - the two events can be asynchronous. Such a system would need messaging to ensure that the service (server endpoint) that asynchronously generates the PDF gets notified of a confirmed, paid-for booking, and all the details, and then the PDF can be auto-generated and emailed to you. But if that messaging system fails, the email service would never know about your booking and no ticket would get generated.

Publisher / Subscriber Messaging

This is a very popular paradigm (model) for messaging. The key concept is that publishers 'publish' a message and a subscriber subscribes to messages. To give greater granularity, messages can belong to a certain "topic" which is like a category. These topics are like dedicated "channels" or pipes, where each pipe exclusives handles messages belonging to a specific topic. Subscribers choose which topic they want to subscribe to and get notified of messages in that topic. The advantage of this system is that the publisher and the subscriber can be completely de-coupled - i.e. they don't need to know about each other. The publisher announces, and the subscriber listens for announcements for topics that it is on the lookout for.

This is a very popular paradigm (model) for messaging. The key concept is that publishers 'publish' a message and a subscriber subscribes to messages. To give greater granularity, messages can belong to a certain "topic" which is like a category. These topics are like dedicated "channels" or pipes, where each pipe exclusives handles messages belonging to a specific topic. Subscribers choose which topic they want to subscribe to and get notified of messages in that topic. The advantage of this system is that the publisher and the subscriber can be completely de-coupled - ie they don't need to know about each other. The publisher announces, and the subscriber listens for announcements for topics that it is on the lookout for.

A server is often the publisher of messages and there are usually several topics (channels) that gets published to. The consumer of a specific topic subscribes to those topics. There is no direct communication between the server (publisher) and the subscriber (could be another server). The only interaction is between publisher and topic, and topic and subscriber.

The messages in the topic are just data that needs to be communicated, and can take on whatever forms you need. So that gives you four players in Pub/Sub: Publisher, Subscriber, Topics and Messages.

Better than a database (Better than a database)

So why bother with this? Why not just persist all data to a database and consume it directly from there? Well, you need a system to queue up the messages because each message corresponds to a task that needs to be done based on that message's data. So in our ticketing example, if a 100 people make a booking in 35 minutes, putting all that in the database doesn't solve the problem of emailing those 100 people. It just stores a 100 transactions. Pub/Sub systems handle the communication, the task sequencing and the messages get persisted in a database. So the system can offer useful features like "at least once" delivery (messages won't be lost), persistent storage, ordering of messages, "try-again", "re-playability" of messages etc. Without this system, just storing the messages in the database will not help you ensure that the message gets delivered (consumed) and acted upon to successfully complete the task.

Sometimes the same message may get consumed more than once by a subscriber - typically because the network dropped out momentarily, and though the subscriber consumed the message, it didn't let the publisher know. So the publisher will simply re-send it to the subscriber. That's why the guarantee is "at least once" and not "once and only once". This is unavoidable in distributed systems because networks are inherently unreliable. This can raise complications, where the message triggers an operation on the subscriber's side, and that operation could change things in the database (change state in the overall application). What if a single operation gets repeated multiple times, and each time the application's state changes?

Controlling Outcomes - one or many outcomes? (Controlling Outcomes - one or many outcomes?)

The solution to this new problem is called idempotency - which is a concept that is important but not intuitive to grasp the first few times you examine it. It is a concept that can appear complex (especially if you read the wikipedia entry), so for the current purpose, here is a user-friendly simplification from StackOverflow:

In computing, an idempotent operation is one that has no additional effect if it is called more than once with the same input parameters.

In computing, an idempotent operation is one that has no additional effect if it is called more than once with the same input parameters.

So when a subscriber processes a message two or three times, the overall state of the application is exactly what it was after the message was processed the first time. If, for example, at the end of booking your flight tickets and after you entered your credit card details, you clicked on "Pay Now" three times because the system was slow ... you would not want to pay 3X the ticket price right? You need idempotency to ensure that each click after the first one doesn't make another purchase and charge your credit card more than once. In contrast, you can post an identical comment on your best friend's newsfeed N number of times. They will all show up as separate comments, and apart from being annoying, that's not actually wrong. Another example is offering "claps" on Medium posts - each clap is meant to increment the number of claps, not be one and only one clap. These latter two examples do not require idempotency, but the payment example does.

There are many flavours of messaging systems, and the choice of system is driven by the use-case to be solved for. Often, people will refer to "event based" architecture which means that the system relies on messages about "events" (like paying for tickets) to process operations (like emailing the ticket). The really commonly talked about services are Apache Kafka, RabbitMQ, Google Cloud Pub/Sub, AWS SNS/SQS.

Section 13: Smaller Essentials (Section 13: Smaller Essentials)

记录中 (Logging)

Over time your system will collect a lot of data. Most of this data is extremely useful. It can give you a view of the health of your system, its performance and problems. It can also give you valuable insight into who uses your system, how they use it, how often, which parts get used more or less, and so on.

This data is valuable for analytics, performance optimization and product improvement. It is also extremely valuable for debugging, not just when you log to your console during development, but in actually hunting down bugs in your test and production environments. So logs help in traceability and audits too.

The key trick to remember when logging is to view it as a sequence of consecutive events, which means the data becomes time-series data, and the tools and databases you use should be specifically designed to help work with that kind of data.

监控方式 (Monitoring)

This is the next steps after logging. It answers the question of "What do I do with all that logging data?". You monitor and analyze it. You build or use tools and services that parse through that data and present you with dashboards or charts or other ways of making sense of that data in a human-readable way.

By storing the data in a specialized database designed to handle this kind of data (time-series data) you can plug in other tools that are built with that data structure and intention in mind.

Alerting (Alerting)

When you are actively monitoring you should also put a system in place to alert you of significant events. Just like having an alert for stock prices going over a certain ceiling or below a certain threshold, certain metrics that you're watching may warrant an alert being sent if they go too high or too low. Response times (latency) or errors and failures are good ones to set up alerting for if they go above an "acceptable" level.

The key to good logging and monitoring is to ensure your data is fairly consistent over time, as working with inconsistent data could result in missing fields that then break the analytical tools or reduce the benefits of the logging.

资源资源 (Resources)

As promised, some useful resources are as follows:

A fantastic Github repo full of concepts, diagrams and study prep

A fantastic Github repo full of concepts, diagrams and study prep
Tushar Roy's introduction to Systems Design

Tushar Roy's introduction to Systems Design
Gaurav Sen's YouTube playlist

Gaurav Sen's YouTube playlist
SQL vs NoSQL

SQL vs NoSQL

I hope you enjoyed this long-form guide!

You can ask me questions on Twitter.

You can ask me questions on Twitter .

Postscript for freeCodeCamp students

Postscript f or f reeCodeCamp students

I really, truly believe your most precious resources are your time, effort and money. Of these, the single most important resource is time, because the other two can be renewed and recovered. So if you’re going to spend time on something make sure it gets you closer to this goal.

I really, truly believe your most precious resources are your time, effort and money. Of these, the single most important resource is time, because the other two can be renewed and recovered. So if you're going to spend time on something make sure it gets you closer to this goal.

With that in mind, if you want to invest 3 hours with me to find your shortest path to learning to code (especially if you’re a career changer, like me), then head to my course site and use the form there sign up (not the popup!). If you add the words “I LOVE CODE” to the message, I will know you’re a freeCodeCamp reader, and I will send you a promo code, because just like you, freeCodeCamp gave me a solid start.

With that in mind, if you want to invest 3 hours with me to find your shortest path to learning to code (especially if you're a career changer, like me), then head to my course site and use the form there sign up (not the popup!). If you add the words “I LOVE CODE” to the message, I will know you're a freeCodeCamp reader, and I will send you a promo code, because just like you, freeCodeCamp gave me a solid start.

Also if you would like to learn more, check out episode 53 of the freeCodeCamp podcast, where Quincy (founder of FreeCodeCamp) and I share our experiences as career changers that may help you on your journey. You can also access the podcast on iTunes, Stitcher, and Spotify.

Also if you would like to learn more, check out episode 53 of the freeCodeCamp podcast , where Quincy (founder of FreeCodeCamp) and I share our experiences as career changers that may help you on your journey. You can also access the podcast on iTunes , Stitcher , and Spotify .

翻译自: https://www.freecodecamp.org/news/systems-design-for-interviews/

面试系统设计