异步api

by Garrett Vargas

通过Garrett Vargas

如何设计无服务器异步API (How To Design a Serverless Async API)

I recently ran a workshop to teach developers how to create an Alexa skill. The workshop material centered around a project to return car rental search results. Because I wanted to focus the learning on developing the conversational flow and not the mechanics of doing a car search, I decided to encapsulate the search logic behind an API. In addition, since the car search request can take 10 or more seconds to complete, I wanted the call to be asynchronous so we could build a conversation like:

我最近举办了一个讲习班，教开发人员如何创建Alexa技能。 研讨会资料围绕一个项目返回了租车搜索结果。 因为我想将学习重点放在开发对话流程上，而不是在进行汽车搜索的机制上，所以我决定将搜索逻辑封装在API后面。 此外，由于汽车搜索请求可能需要10秒钟或更长时间才能完成，因此我希望通话是异步的，因此我们可以建立如下对话：

“Find a Car in New York next weekend”

“下周末在纽约找车”

“What size car would you like for your trip in New York next weekend?”

“下周末您要在纽约旅行时选择哪种尺寸的汽车？”

“A small car”

“一辆小汽车”

“Is there a specific company you’d like to rent from?”

“您想租一家特定的公司吗？”

“No”

“没有”

The implementation of the asynchronous API was a pretty interesting project in and of itself, and in this blog post I’m going to tell you how I did this in a serverless way using API Gateway, Lambda functions, and S3.

异步API的实现本身就是一个非常有趣的项目，在这篇博客文章中，我将告诉您如何使用API​​网关，Lambda函数和S3以无服务器方式进行此操作。

S3水桶 (S3 Bucket)

The S3 bucket in this architecture serves as a cache that stores search results to be retrieved later. Callers of the API are given a token when they start a search, and the basic design is to use that token as part of the S3 object name to allow you to retrieve the contents on a subsequent call. To prevent the bucket from filling up with search results, you can set an expiration policy that is appropriate for the lifecycle of your API results (i.e. how long do you want the asynchronous results to stay alive?).

此体系结构中的S3存储桶用作缓存，用于存储以后要检索的搜索结果。 API的调用者开始搜索时会得到一个令牌，并且基本设计是将该令牌用作S3对象名称的一部分，以允许您在后续调用中检索内容。 为了防止存储桶填满搜索结果，您可以设置一个适用于您的API结果生命周期的过期策略(即，您希望异步结果保持多长时间？)。

Note that the expiration policy can only be set in day increments, so even if you have results that should be considered stale after 30 minutes, with this approach you’ll still have the object in storage for at least a day.

请注意，过期策略只能以天为单位进行设置，因此即使您将30分钟后的结果认为是过时的，使用这种方法，您仍将对象存储至少一天。

You can associate a timestamp with the object and check it in your code to make sure that the result is ignored if it is more than a certain age, but the object itself will persist for at least a day.

您可以将时间戳记与对象关联，并在代码中对其进行检查，以确保结果(如果超过一定期限)将被忽略，但是对象本身将持续至少一天。

To set up your bucket, take the following steps in the AWS Management Console:

要设置存储桶，请在AWS管理控制台中执行以下步骤：

• Click Create Bucket from S3

  单击从S3 创建存储桶

• Enter the name of the bucket and make note of which region you create it in (you’ll need to make sure your Lambda functions and API Gateway are all set up in this same region). Note that S3 bucket names are globally unique, which means a name like “test” will likely be taken. You’ll need to pick something that no other user has created before, so something that incorporates your name or the current date would be a good starting point输入存储桶的名称，并记下创建存储桶的区域(您需要确保在同一区域中都设置了Lambda函数和API网关)。 请注意，S3存储桶名称是全局唯一的，这意味着可能会使用“ test”之类的名称。 您需要选择以前没有其他用户创建过的内容，因此可以使用包含您的姓名或当前日期的内容作为一个很好的起点
• You can keep the bucket with the default permissions and no versioning — you’ll explicitly grant your Lambda function permission to this bucket, so don’t expose it publicly to the world (in fact, that would be a bad idea!)您可以将存储桶保留为默认权限，而无需进行版本控制-您将向该存储桶明确授予Lambda函数权限，因此不要将其公开暴露给世界(实际上，这是个坏主意！)

• Once the bucket is created, click on it and go to the Management tab

  创建存储桶后，单击它并转到“ 管理”选项卡

• Enter a lifecycle by clicking Add Lifecycle Rule

  通过单击添加生命周期规则输入生命周期

• Enter a name and skip through the Transitions screen to end up on the Expiration screen

  输入名称并跳过“过渡”屏幕以显示在“ 到期”屏幕上

• Since we didn’t add versioning to our S3 bucket, you only need to configure an expiration rule for the Current Version as illustrated below

  由于我们没有向S3存储桶添加版本控制，因此您只需为“ 当前版本”配置到期规则，如下所示

Lambda函数 (Lambda Functions)

I had the basic idea to use a Lambda function to perform a search, return a token to the caller, and write the results into an S3 bucket. The results could then be retrieved by a subsequent call, passing in the token and any additional filtering information (for example, “small car” in the example above).

我的基本想法是使用Lambda函数执行搜索，将令牌返回给调用者，然后将结果写入S3存储桶。 然后可以通过后续调用，传递令牌和任何其他过滤信息(例如，上例中的“小型汽车”)来检索结果。

However, I quickly realized that my Lambda function would return after I validated the input parameters and called back with a token, which meant that I wasn’t able to keep it alive to write the search results out to S3.

但是，我很快意识到，在验证输入参数并使用令牌进行回调之后，Lambda函数将返回，这意味着我无法保持活动状态，无法将搜索结果写入S3。

What I needed was a way to continue code execution after I had the token so I could return to the API caller. To do this, I created two Lambda functions, one to validate the parameters, and the other to perform the search and lookup the cached results.

我需要的是在获得令牌后继续执行代码的方法，这样我就可以返回到API调用者。 为此，我创建了两个 Lambda函数，一个用于验证参数，另一个用于执行搜索和查找缓存结果。

The first function validates the parameters, and once it has done so, it invokes the second Lambda function asynchronously to kick off the search, then returns with a generated token back to the caller while the second Lambda function churns away. The token that my generateToken function used in my workshop was just a timestamp since I didn’t have scalability considerations, but it could also be a UUID or other generated ID.

第一个函数验证参数，一旦完成验证，它将异步调用第二个Lambda函数以开始搜索，然后在第二个Lambda函数失效时以生成的令牌返回给调用者。 由于我没有可伸缩性考虑，所以我的generateToken函数在我的讲习班中使用的令牌只是一个时间戳，但是它也可以是UUID或其他生成的ID。

Because this Lambda function invokes another Lambda function, you’ll need to give it the appropriate permissions to make this call. You do this by creating the appropriate IAM role following these steps:

由于此Lambda函数将调用另一个Lambda函数，因此您需要为其授予适当的权限才能进行此调用。 您可以按照以下步骤通过创建适当的IAM角色来做到这一点：

• In your Lambda function, under Execution role, select Create a Custom Role in the dropdown

  在Lambda函数的“执行角色”下，从下拉列表中选择“ 创建自定义角色 ”

• This will launch IAM in a new tab这将在新标签页中启动IAM

• Select Create a new IAM Role from the dropdown for IAM Role and provide it with a name

  从“ IAM角色 ”下拉列表中选择“ 创建新的IAM角色” ，并为其提供名称。

• In the full list of IAM roles, select this new role and click Attach Policies

  在IAM角色的完整列表中，选择此新角色，然后单击“ 附加策略”。

• Filter for the AWSLambdaRole policy and add it to this role. Optionally, you can modify the JSON to give it access only to the second Lambda function after you create it in the following step, by referring to its ARN in the Resource field

  过滤AWSLambdaRole策略并将其添加到该角色。 (可选)您可以修改JSON，以使其在接下来的步骤中通过在Resource字段中引用其ARN使其仅访问第二个Lambda函数。

The second Lambda function has two responsibilities — to perform the search and save the results into an S3 bucket, and to retrieve the results from an S3 bucket when called with a valid token. I could have separated this logic and created three Lambda functions, but I felt that it was a better design to have the logic that accessed the cache and knew how to encode/decode the object in one place.

Lambda的第二个功能有两个职责-执行搜索并将结果保存到S3存储桶中，以及在使用有效令牌调用时从S3存储桶中检索结果。 我本可以分开这个逻辑并创建三个 Lambda函数，但是我认为让逻辑访问缓存并知道如何在一处编码/解码对象是一个更好的设计。

Because API Gateway allows you to map query parameters, you’ll find it easy to differentiate between the cases when this function is being called to perform a new search and when it is being asked to retrieve a search result (I’ll demonstrate how to do that later). Note that this function calls a lengthy internal doSearch function which writes results to S3 based on token provided from the previous function.

由于API Gateway允许您映射查询参数，因此您将很容易区分在调用此函数以执行新搜索和要求其检索搜索结果的情况之间(我将演示如何稍后再执行)。 请注意，此函数调用一个冗长的内部doSearch函数，该函数根据前一个函数提供的令牌将结果写入S3。

Because this Lambda function makes a call to read and write from S3, you’ll have to set the appropriate permissions for this Lambda function — which will be different than the first one. Follow the same set of steps to create an IAM role, only this time rather than the AWSLambdaRole policy, you’ll want to associate the AmazonS3FullAccess policy — again optionally providing the ARN of the specific S3 bucket that you want to provide full access to.

由于此Lambda函数会调用S3进行读取和写入操作，因此您必须为此Lambda函数设置适当的权限-这将与第一个函数不同。 遵循同一组步骤来创建IAM角色，只是这次而不是AWSLambdaRole策略，您将要关联AmazonS3FullAccess策略-再次有选择地提供要提供完全访问权限的特定S3存储桶的ARN。

API网关 (API Gateway)

With the Lambda functions out of the way, the next step is to create an API Gateway around these functions so that a user has a REST API to call into. Remember, the flow I wanted to build from a client perspective was:

在不使用Lambda函数的情况下，下一步是围绕这些函数创建API网关，以便用户可以调用REST API。 记住，我要从客户角度构建的流程是：

• POST call to the API with the desired search parameters使用所需的搜索参数对API进行POST调用
• Get a token back as a response取回令牌作为回应
• Ask the user some additional questions to help narrow the results向用户询问一些其他问题，以帮助缩小结果范围
• GET call to the API with the token and additional filter criteria to get the actual result set使用令牌和其他过滤条件对API进行GET调用，以获取实际结果集

API Gateway makes it easy and convenient to access your Lambda functions as desired.

API网关使您可以根据需要轻松便捷地访问Lambda函数。

The first step is to create your new API. In the AWS Management Console, you can navigate to API Gateway and click Create API to create a new API. Once you’ve given it a name, you need to create the methods that you want to use in accessing the API. In my case, that meant selecting POST and GET as new methods from the Actions drop down menu.

第一步是创建新的API。 在AWS管理控制台中，您可以导航到API网关，然后单击创建API以创建新的API。 为其命名后，需要创建要用于访问API的方法。 就我而言，这意味着从“ 操作”下拉菜单中选择POST和GET作为新方法。

Let’s start with the POST. Once you’ve selected the POST method, you’ll be asked the Integration Type you’d like to use. Select Lambda Function, and then fill in the details with the first Lambda function you created to validate parameters and kick off the search. You don’t need to point API Gateway to the second Lambda function that does the search — that is done by by the validation function for you.

让我们从POST开始。 选择POST方法后，系统将询问您要使用的集成类型 。 选择Lambda Function ，然后使用您创建的第一个Lambda函数填写详细信息以验证参数并开始搜索。 您无需将API Gateway指向执行搜索的第二个Lambda函数，这是由验证函数为您完成的。

After you set these parameters, you’ll see the full API flow, along with a TEST sidebar that will allow you to pass a test payload to your API to see if it executes properly.

设置完这些参数后，您将看到完整的API流程以及一个TEST侧边栏，该边栏将允许您将测试有效负载传递给API，以查看其是否正常执行。

For the GET call, do similar, though in this case you’re going to call the second Lambda function passing in a token so it knows to retrieve the results from your S3 cache. Also, in this case you won’t have a JSON payload to pass on — rather the expectation is that the customer supplies query parameters in the URL. API Gateway allows you to do this transformation easily via a Mapping Template.

对于GET调用，请执行类似操作，尽管在这种情况下，您将调用传递令牌的第二个Lambda函数，以便它知道从S3缓存中检索结果。 同样，在这种情况下，您将没有传递JSON有效负载的方法，而是期望客户在URL中提供查询参数。 API Gateway使您可以通过映射模板轻松进行此转换。

The basic steps are as follows:

基本步骤如下：

• Add a GET method under the Actions drop down menu

  在“ 操作”下拉菜单下添加GET方法

• Set Integration Type to Lambda Function

  将积分类型设置为Lambda函数

• Enter in the details of the second Lambda function输入第二个Lambda函数的详细信息

• Once created, click on the Integration Request step of your GET method execution

  创建完成后，单击GET方法执行的Integration Request步骤

• Expand the Mapping Templates section and click Add mapping template

  展开“ 映射模板”部分，然后单击“ 添加映射模板”。

• Type application/json into the edit box and click the check to confirm

  在编辑框中输入application / json ，然后单击检查以确认

• Enter in the mapping from query parameters to JSON request — you’ll do this with keys of the form "field": “$input.params('queryparam')" This will map the query param named queryparam to a field named field

  输入从查询参数到JSON请求的映射-您将使用"field": “$input.params('queryparam')"形式的键来执行此操作"field": “$input.params('queryparam')"这会将名为queryparam的查询参数queryparam到名为field

The nice thing here is that you don’t have to have the same name for the query parameters exposed to your client as for internal usage in your Lambda function. For example, in my case I expect parameters of User, CarSize, SupplierRating, and UpgradeClass but I map these to id, size, rating, and upgrade respectively for internal use.

这里的好处是，不必为暴露给客户端的查询参数使用与Lambda函数内部使用的名称相同的名称。 例如，在我的情况下，我希望使用User，CarSize，SupplierRating和UpgradeClass的参数，但是我将它们分别映射到id，大小，等级和升级以供内部使用。

构建和部署 (Build and Deploy)

Now that you’ve integrated your Lambda function into API Gateway, you’re ready to build and deploy. Under the Actions menu, select Deploy API. API Gateway will ask you for a Deployment Stage; choose [New Stage] to create a new stage, and provide it a name like Beta. After you deploy the API, API Gateway will tell you the URL to use to invoke your API. You use the same URL for both the POST and GET functions. Pretty easy, isn’t it?

现在，您已将Lambda函数集成到API Gateway中，就可以开始构建和部署了。 在“操作”菜单下，选择“ 部署API” 。 API网关将要求您进行部署； 选择[新阶段]创建一个新阶段，并提供一个类似Beta的名称。 部署API后，API网关将告诉您用于调用API的URL。 POST和GET函数使用相同的URL。 很简单，不是吗？

结语 (Wrap Up)

What I described here is the foundation for an asynchronous serverless API. There are a lot of edge cases and error handling that I glossed over, as well as techniques within API Gateway to harden the API that I didn’t address such as validating that all required parameters are set before invoking the Lambda function or requiring an access token as opposed to creating a wide-open API.

我在这里描述的是异步无服务器API的基础。 我掩盖了很多极端情况和错误处理，以及API网关中用于强化我未解决的API的技术，例如在调用Lambda函数或要求访问之前验证是否已设置所有必需的参数令牌而不是创建一个广泛开放的API。

In addition, this use case was for a small workshop environment. You’d have to look at your own use case to understand the scale you need and whether this approach would work for you. You’d want to pay particular attention to the concurrent execution settings for your Lambda function. If the underlying call you are trying to make takes a minute to run for example, even with a limit of 1000 concurrent executions you’d only be able to make 16 calls per second, which may not prove sufficient for a full production workload.

另外，该用例用于小型车间环境。 您必须查看自己的用例，以了解所需的规模以及此方法是否对您有用。 您需要特别注意Lambda函数的并发执行设置。 例如，如果您要进行的基础调用需要一分钟才能运行，即使限制了1000次并发执行，则您每秒只能进行16次调用，这可能不足以满足全部生产工作负荷。

Caveats aside, for the right use cases this approach can be a simple and powerful way to create an async API without having to stand up dedicated servers or a caching solution.

除了警告之外，对于正确的用例，此方法可以是一种创建异步API的简单而强大的方法，而无需使用专用服务器或缓存解决方案。

翻译自: https://www.freecodecamp.org/news/how-to-design-a-serverless-async-api-6cfd68939459/

异步api