It is inevitable that something will fail in a distributed system, and we should plan as if it is a normal occurrence. One solution to this problem is to run multiple instances of a service. That way, if one fails, the others can take over.

在分布式系统中不可避免地会发生某些故障，因此我们应该像正常情况那样进行计划。 解决此问题的一种方法是运行服务的多个实例。 这样，如果一个失败，其他人就可以接管。

In this article, we will explore some of the different ways we can achieve this on Kubernetes (K8s).

在本文中，我们将探讨在Kubernetes (K8s)上实现此目标的一些不同方法。

没有 (None)

Redundancy has a cost to it, and we should consider this when deciding how much resiliency we need. If your customers can put up with a small amount of outage and it won't affect their experience too much, you may not need any.

冗余为此付出了代价，我们在决定所需的弹性时应考虑这一点。 如果您的客户可以忍受少量中断，并且不会对他们的体验造成太大影响，那么您可能不需要任何中断。

When talking about uptime of a service, it is often specified in a number of nines (e.g. 99.9% uptime). This means that for every 1,000 requests, only one may fail. As a rule of thumb, every nine you add on to service will cost ten times as much to achieve.

在谈论服务的正常运行时间时，通常以九个数字来指定(例如，正常运行时间为99.9％)。 这意味着每1000个请求中，只有一个失败。 根据经验，每增加九个服务，您将花费十倍的成本。

As long as your application is not constantly crashing, you may be able to run one pod and rely on K8s rescheduling it if something does go wrong. This does assume that one pod can handle the load the service is receiving.

只要您的应用程序不会经常崩溃，您就可以运行一个pod并依靠K8重新安排它的运行时间，以免出现问题。 这确实假设一个Pod可以处理服务正在接收的负载。

ñ (N)

As your service starts requiring more pods to handle the load, you can scale it out. If the traffic profile changes over time (e.g. a lunchtime rush), you should have enough pods to handle the load at peak times. This strategy only gives you more resilience when the pods are receiving significantly less traffic.

当您的服务开始需要更多的Pod来处理负载时，您可以进行扩展。 如果流量配置文件随时间变化(例如，午餐时间高峰)，则您应该有足够的吊舱以在高峰时间处理负载。 仅当Pod收到的流量明显减少时，此策略才能为您提供更大的弹性。

If a pod fails during peak times, the requests will spread out across the remaining pods and potentially overwhelm them. During lower-traffic times, the remaining pods may have enough capacity to handle the load.

如果某个Pod在高峰时段发生故障，则请求将分散到其余Pod上，并可能使它们不堪重负。 在交通流量较低的时候，其余的吊舱可能有足够的容量来承受负荷。

When talking about scaling, you may hear the terms in, out, up, and down. Generally, scaling up and down means keeping the same number of instances but increasing the CPU and/or memory of the servers. In and out is the process of introducing or removing instances of a service but keeping the resources the same. This allows you to scale further, as you may be limited by the maximum CPU or memory you can use.

在讨论缩放时，您可能会听到术语in ， out ， up和down 。 通常，向上和向下扩展意味着保持相同数量的实例，但增加了CPU和/或服务器的内存。 导入和导出是引入或删除服务实例，但保持资源不变的过程。 这使您可以进一步扩展，因为您可能会受到可使用的最大CPU或内存的限制。

N + 1 (N + 1)

As before with n, we need to understand how many pods are needed to handle peak traffic. This time, we add one extra pod. This gives us protection against one pod failing during peak times. The cost of this resilience is the cost of one pod, as this is extra and only needed in failure scenarios. If one pod can handle all of the traffic, you should still have an extra pod.

与n ，我们需要了解需要多少个Pod来处理高峰流量。 这次，我们添加了一个额外的广告连播。 这为我们提供了保护，以防止高峰期间出现一个吊舱失效。 这种弹性的成本就是一个吊舱的成本，因为这是额外的成本，只有在故障情况下才需要。 如果一个Pod可以处理所有流量，则您仍然应该有一个额外的Pod。

缩放比例 (Scaling)

Rather than manually calculating the number of pods we need at peak times, we can let K8s do this for us. Given a scaling metric, K8s can scale our service in and out based on the current demand. This reduces costs by scaling down during lower demand and up when we need it. Scaling in itself does not give us resilience from pod failure but does protect against increasing demand. Keeping pods small in terms of memory and CPU allows us to more precisely scale and cut costs even more.

与其手动计算高峰时段所需的吊舱数量，不如让K8代为完成。 有了扩展指标，K8可以根据当前需求扩展和扩展我们的服务。 通过在需求降低时缩小规模并在需要时扩大规模，从而降低了成本。 扩展本身并不能使我们抵御吊舱故障，但可以防止需求增加。 使Pod在内存和CPU方面较小，这使我们可以更精确地扩展规模并降低成本。

For information on how to find out when to scale your service, check out my article on Performance Testing for K8s Autoscaling.

有关如何确定何时扩展服务的信息，请参阅有关K8s自动扩展性能测试的文章 。

When scaling a service, you should include some headroom, meaning you should not push your pods to the limit. Your pods will take some time to start up, and autoscaling metrics are calculated periodically. Your application needs to be able to handle the requests between requesting to scale up and actually scaling up. If you have lots of pods, the headroom can shrink, as it is spread out across all pods.

扩展服务时，应留出一定的净空，这意味着您不应将Pod推到极限。 您的Pod需要一些时间才能启动，并且会自动计算自动缩放指标。 您的应用程序需要能够处理在请求扩展与实际扩展之间的请求。 如果您有很多吊舱，则净空空间会缩小，因为它散布在所有吊舱中。

Horizontal Pod Autoscalers (HPAs) are used to autoscale our services in K8s. For more information on how these can be set up, check out the documentation. For more information on which metrics are not a good idea to use for autoscaling, check out this article.

水平Pod自动缩放器(HPA)用于在K8s中自动缩放我们的服务。 有关如何进行设置的更多信息，请查阅文档 。 有关哪些指标不适用于自动缩放的更多信息，请参阅本文 。

75％缩放 (75% Scaling)

Once we have autoscaling working and know when we should scale our service, we can control how much resiliency we want. By scaling at 75% of the capacity of the service, we can lose 25% of the pods and only be at capacity. The more pods we have, the more pods we can lose — but also the more we pay for underused pods. When running a large number of pods, you may consider decreasing the resiliency percentage, as you will still have a large number of pods that are able to fail.

一旦我们可以进行自动扩展并知道何时扩展服务，就可以控制所需的弹性。 通过将服务容量扩展到75％，我们可以损失25％的Pod，而仅保持容量不变。 我们拥有的豆荚越多，我们损失的豆荚就越多-但我们为未充分利用的豆荚支付的费用也就越多。 当运行大量的Pod时，您可能会考虑降低弹性百分比，因为您仍然会有大量Pod发生故障。

This technique is particularly useful if a service’s traffic profile is particularly spikey. Spikes can be a particular nuisance for autoscaling, as they may be so short in time that the autoscaler does not have time to react. If you know roughly how big the spikes are, you can plan this into where a service scales.

如果服务的流量配置文件特别棘手，则此技术特别有用。 尖峰信号可能会对自动缩放产生特别的麻烦，因为它们的时间可能太短，以至于自动缩放器没有时间做出React。 如果您大致知道峰值是多少，则可以将其计划到服务扩展的地方。

缩放比例+ N (Scaling + N)

If we want to control our redundancy more precisely than a percentage, we can scale to capacity and then introduce n extra pods. This would allow n pods to fail but still leave us with enough capacity. This allows us to control exactly how many redundant pods we pay for.

如果我们要控制冗余度而不是某个百分比，则可以扩展到容量，然后引入n额外的Pod。 这将使n pod失败，但仍然给我们留下足够的容量。 这使我们能够精确控制我们为多少冗余吊舱付款。

K8s services use labels to decide which pods’ requests are routed to. This allows us to deploy two replica sets of the same service with different configurations. One replica set will scale using a horizontal pod autoscaler, while we will configure the other to have n pods. Both replica sets will tag the pods with the same label and a service will route to this label. The service will evenly split the traffic across all pods, allowing service to scale while maintaining n redundant pods.

K8s服务使用标签来决定将哪些吊舱的请求路由到。 这使我们可以使用不同的配置部署同一服务的两个副本集。 一个副本集将使用水平容器自动缩放器缩放，而我们将另一个副本集配置为具有n容器。 两个副本集都将使用相同的标签标记容器，并且服务将路由到该标签。 该服务将在所有Pod中平均分配流量，从而在保持n冗余Pod的同时扩展服务。

多个集群 (Multiple Clusters)

Taking things to the next level, we could deploy our application to two K8s clusters. This would allow a whole cluster to fail and still maintain services. A load balancer would sit in front of the clusters and route traffic between the clusters. If you are hosting a cluster on a cloud provider such as AWS, the cluster automatically gets deployed across multiple availability zones. This protects against a physical failure such as a power cut, natural disaster, or worse, so there needs to be a good reason to have multiple clusters.

将事情提升到一个新的水平，我们可以将我们的应用程序部署到两个K8s集群中。 这将使整个群集出现故障，并且仍可以维护服务。 负载均衡器将位于群集的前面，并在群集之间路由流量。 如果您在AWS等云提供商上托管集群，则集群会自动跨多个可用性区域进行部署。 这样可以防止出现物理故障(例如断电，自然灾害或更严重的情况)，因此需要有多个群集的充分理由。

When running multiple clusters, autoscaling becomes more challenging, as generally scaling metrics are not shared across clusters. If a cluster does fail, the still-working cluster will receive all the traffic from the failed cluster with little notice. There are multiple ways of dealing with this sudden increase in load. Your autoscaling may be able to react quickly enough to handle the traffic or you may take the hit and have reduced performance while things scale up.

当运行多个集群时，自动缩放变得更具挑战性，因为通常缩放指标不会在集群之间共享。 如果某个群集确实发生故障，则仍在工作的群集将在几乎不通知的情况下接收来自发生故障的群集的所有流量。 有多种方法可以应对这种突然增加的负载。 您的自动缩放功能可能能够足够Swift地做出React以处理流量，或者您可能会受到打击并在事情不断扩展时降低性能。

Depending on how much redundancy is needed, the clusters could be run in active-passive mode, meaning one cluster receives all the requests. Services would have to scale up from zero in this setup unless the scaling metrics were shared across clusters.

根据所需的冗余程度，群集可以以主动-被动模式运行，这意味着一个群集可以接收所有请求。 除非在群集之间共享缩放指标，否则在此设置中服务必须从零开始扩展。

数量与冗余 (Quantity vs. Redundancy)

The more pods you have, the less a single pod failing affects the other pods. Let's say we have ten pods that can serve 100rps. If we scale at 90rps and one pod fails, the remaining pods will receive 100rps, putting them at their maximum. If we have 20 pods and one fails, the remaining pods would only have to deal with 95rps. Both of these scenarios assume the services are receiving exactly the number of requests where the service should scale. In reality, services normally receive slightly less traffic or slightly more if the service is trying to scale up.

您拥有的豆荚越多，单个豆荚失败对其他豆荚的影响就越小。 假设我们有十个可以服务100rps的广告连播。 如果我们以90rps的比例缩放并且一个Pod失败，则其余Pod将获得100rps，将它们最大化。 如果我们有20个吊舱，但有一个失败，其余的吊舱仅需处理95rps。 这两种情况都假定服务正在准确接收应扩展服务的请求数。 实际上，服务通常会略微减少流量，或者如果尝试扩大规模则会略微增加流量。

结论 (Conclusion)

Autoscaling is a complicated beast and there are lots of options to consider. I hope you have a better understanding of the options available to you and you can use this to reduce the costs of your cloud bill whilst maintaining service uptime.

自动缩放是一个复杂的野兽，有很多选择要考虑。 我希望您对可用的选项有更好的了解，并且可以使用它来减少云账单的成本，同时保持服务的正常运行时间。

Thanks for reading!

谢谢阅读！