magic feature

by Ruslan Talpă

通过RuslanTalpă

停止将PostgREST称为“ MAGIC”！ (Stop calling PostgREST “MAGIC”!)

While it’s flattering for developers to have “magic” associated with their work, it might be damaging to the adoption of PostgREST. Magic means unknown and people fear the unknown. Let’s take 10 minutes to understand how it works internally.

对于开发人员来说，与其工作相关联的“魔术”虽然很讨人喜欢，但可能会对采用PostgREST造成损害。 魔术意味着未知，人们害怕未知。 让我们花10分钟的时间来了解其内部工作方式。

I started contributing to the PostgREST core 2 years ago. Since then, I’ve been checking into what people are saying about it . There are two camps. The die hard fans who call it “magic” and are ready to gang up on the non-believers. And then you have the skeptics.

2年前，我开始为 PostgREST核心做出贡献 。 从那时起，我一直在研究人们在说些什么。 有两个营地。 顽固的歌迷称其为“魔术”，并准备好与非信徒结盟。 然后您便有了怀疑论者。

This narrative addresses the skeptics camp. It stems from the fact that people don’t understand how PostgREST works!

该叙述针对怀疑论者阵营。 这是因为人们不了解PostgREST的工作原理！

It’s all simple, and beautiful.

一切都简单而美丽。

(REST)URL中有什么？ (What’s in a (REST) URL ?)

Let’s take this example

让我们举个例子

GET /items/1

If you strip everything away, like authentication and authorization, the essence that remains is this

如果剥离所有内容，例如身份验证和授权，剩下的本质就是

SELECT * FROM items WHERE id=1

PostgREST takes a HTTP request, looks at it, translates it to SQL and executes it. That’s almost all there is to it. It’s one big pure function. I hope I don’t get crucified by Haskell people for abusing the notation and ignoring there’s actually IO going on.

PostgREST接收一个HTTP请求，对其进行处理，将其转换为SQL并执行。 几乎就是全部。 这是一大功能。 我希望我不要因为Haskell的滥用行为而被钉在十字架上，而忽略了实际上正在发生的IO。

postgrest :: Schema -> HTTP -> SQL

Here’s a slightly more complicated example:

这是一个稍微复杂的示例：

GET /items?select=id,name&id=gt.10&order=name

gets translated to

被翻译成

SELECT id, name FROM items WHERE id > 10 ORDER BY name

三种魔法成分✨ (The three magic ingredients ✨)

There are thee core concepts on which PostgREST is built, everything around them is just bells and whistles. If you understand them, you understand PostgREST.

PostgREST建立在这些核心概念上，它们周围的一切只是一团糟。 如果您了解它们，您就会了解PostgREST。

JSON编码 (JSON encoding)

The first brilliant idea that Joe Nelson had was to recognize that PostgreSQL can JSON encode the response. Now you might be thinking “Why would one want to do that, put more load on the component that is the hardest to scale?”

Joe Nelson的第一个绝妙主意是认识到PostgreSQL可以对响应进行JSON编码。 现在您可能在想：“为什么要这样做，给最难扩展的组件增加更多负载？”

Well, here are the reasons.

好吧，这是原因。

PostgreSQL JSON encoder is fast, it’s C fast. The bigger your response is, the bigger the gain. It can be 2x/10x faster than ActiveRecord or even 160x for large responses.

PostgreSQL JSON编码器速度很快，它的速度很快。 您的回应越大，收益就越大。 它可以比ActiveRecord快2倍/ 10倍，对于大响应甚至可以达到160倍。

Let’s think for a minute. What are the implications of having the database return the final response, aside from making it do more work?

让我们思考一分钟。 除了使数据库做更多的工作之外，让数据库返回最终响应还有什么意义？

Whatever is sitting in front of the database no longer has to deserialize the response coming from the database, and turn that into internal representation of the language for that data. Then, after the data is in the memory, it does not need to serialize back to JSON. This means on the way out, your code does not need to do anything, just pass along whatever it gets from the database.

不管坐在数据库前面的是什么，都不再需要反序列化来自数据库的响应，并将其转换为该数据语言的内部表示形式。 然后，将数据放入内存后，无需序列化回JSON。 这意味着在出路时，您的代码无需执行任何操作，只需传递从数据库中获取的所有内容即可。

We shifted some of the load to the database from the application code. This is nothing for the mighty PostgreSQL, it’s not where the heavy lifting happpens. If we look at the entire system, we’ve reduced the total amount of work that needs to be done by removing one serialize/deserialize step. This is one of the secrets of why PostgREST is so fast. It lets the big guy do it’s thing.

我们将一些负载从应用程序代码转移到了数据库。 对于强大的PostgreSQL来说，这没什么，不是繁重的事情发生的地方。 如果我们查看整个系统，则可以通过减少一个序列化/反序列化步骤来减少需要完成的工作总量。 这就是为什么PostgREST如此之快的秘密之一。 它让大个子做这件事。

Let’s look at what that actually looks like in SQL. This query is like the ones PostgREST generates.

让我们看一下SQL中的实际情况。 此查询类似于PostgREST生成的查询。

WITH essence AS (  SELECT id, name FROM items WHERE id > 10 ORDER BY name)SELECT   coalesce(    array_to_json(array_agg(row_to_json(response))),    '[]'  )::character varying AS BODYFROM (SELECT * FROM essence) response

We’ve wrapped the essence query. On the other end, we get our result as a single row with one column called BODY which contains the JSON response. Easy!

我们包装了essence查询。 另一方面，我们将结果作为一行包含一个名为BODY列作为单行，其中包含JSON响应。 简单！

认证/授权 (Authentication / Authorization)

This is the part that most newcomers have troubles with when first encountering PostgREST. They have the (perfectly natural) question: “How does PostgREST implement authentication/authorization?”.

这是大多数新手在初次接触PostgREST时遇到的麻烦。 他们有一个(完全自然的)问题：“ PostgREST如何实现身份验证/授权？”。

The short answer is, it doesn’t.

简短的答案是， 不是 。

Again, one of Joe’s brilliant ideas, and I’d even say it’s more fundamental then the JSON encoding.

再次，这是Joe的一个绝妙的主意之一，我什至会说，这比JSON编码更为根本。

PostgreSQL has a rich role system. In conjunction with such features as views and Row Level Security (RLS), you can have fine grained control over access to your data down to an individual cell. Yet, People treat their databases as “dumb data stores”. And what’s even worse, they connect to them using roles with admin privileges.

PostgreSQL具有丰富的角色系统 。 结合视图和行级安全性 (RLS)等功能，您可以对访问单个单元格的数据进行精细控制。 但是，人们将其数据库视为“哑数据存储”。 更糟糕的是，他们使用具有管理员权限的角色连接到他们。

What would be the point of a SQL injection attack if the role with which the application is connecting to the database has privileges only to specific tables and rows? At worst, the attacker would be deleting his own data. But i am digressing …

如果应用程序连接到数据库的角色仅具有特定表和行的特权，那么SQL注入攻击的意义何在？ 最糟糕的是，攻击者将删除自己的数据。 但是我离题了……

When first PostgREST connects to the database, it connects to it using a role that we usually call authenticator. That role has no privileges attached to it aside from the ability to login.

当第一个PostgREST连接到数据库时，它使用我们通常称为authenticator的角色连接到数据库。 除登录功能外，该角色没有任何特权。

Whenever an authenticated request comes in, and there is anAuthorization header that contains a JWT token, PostgREST will decode the token, verify that it’s valid using the secret key, and look at it’s payload for a particular field called role. Let’s say that role has the value of alice.

每当收到经过身份验证的请求，并且有一个包含JWT令牌的Authorization标头时，PostgREST都会对该令牌进行解码，使用私钥验证其是否有效，并查看其特定负载(称为role.的有效负载role. 假设role具有alice的价值。

This means that this request needs to be executed with the privileges of alice. To do this we use one PostgreSQL neat little trick. You can switch the current user, sort of like doing sudo alice. And even nicer, you can do that in the context of a transaction so there is no chance of one request interfering with another. We are not creating a new database connection, this happens in the same connection. And although alice is a database role, it does not have login privileges.

这意味着此请求需要使用alice的特权来执行。 为此，我们使用了一个PostgreSQL整洁的小技巧。 您可以切换当前用户，有点像在做sudo alice. 甚至更好的是，您可以在事务的上下文中执行此操作，因此一个请求不会干扰另一个请求。 我们没有创建新的数据库连接，这是在同一连接中发生的。 虽然alice 是数据库角色，没有登录权限。

Here is the order of how things happen:

这是事情发生的顺序：

BEGIN;SET LOCAL role TO 'alice';-- the main query goes hereCOMMIT;

This does not mean that PostgREST switches to whatever it likes. For this to work, you have to explicitly say that the authenticator role has the right to assume the alice role by:

这并不意味着PostgREST可以切换到喜欢的任何东西。 为此，您必须明确声明authenticator角色有权通过以下方式承担alice角色：

GRANT alice TO authenticator;

So just like that, PostgREST gained the ability to do authorization by leveraging the underlying database role system. The implications and advantages of this approach are huge.

这样一来，PostgREST通过利用基础数据库角色系统获得了进行授权的能力。 这种方法的含义和优点是巨大的。

The PostgREST codebase remains small and simple. You get to declare the privileges for each user using SQL with no ugly imperative code all over the place. And, no matter what future code accesses the database, the rules are consistently applied.

PostgREST代码库仍然小而简单。 您可以使用SQL为每个用户声明特权，而无需在各处使用难看的命令性代码。 而且，无论将来使用什么代码访问数据库，都将始终遵循规则。

There is also one other big advantage that is not immediately obvious.

还有另一项尚未立即显而易见的重大优势。

In the traditional way of building APIs, when a request comes in you start checking if the current user has the right to access that information. That means extra queries. Those types of queries have a big part to play in the request overall latency, and the load on the database. This gives the impression that the databases are slow and don’t scale very well.

按照传统的构建API的方式，当有请求进入时，您将开始检查当前用户是否有权访问该信息。 这意味着额外的查询。 这些类型的查询在请求总体延迟和数据库负载中起着很大的作用。 这给人的印象是数据库很慢并且扩展性不是很好。

By giving PostgreSQL the full picture of who is issuing the query (role), what are his privileges (grants) and his restrictions (RLS), the query planner can do a much better job, utilize all the indexes and even get faster over time.

通过向PostgreSQL提供有关谁发出查询(角色)，他的特权(授予)和他的限制(RLS)是什么的全貌，查询计划者可以做得更好，可以利用所有索引，甚至可以随着时间的推移变得更快。

The first objection raised is that one would need to have a database role for each application user and that is not a very good design. They would be right, but that is not what PostgREST is asking you to do. You can use database roles to set user groups (admin, employee, customer) and then use RLS to specify user rules based on their username, id or email.

提出的第一个反对意见是，每个应用程序用户都需要拥有一个数据库角色，而这并不是一个很好的设计。 他们是正确的，但这不是PostgREST要求您执行的操作。 您可以使用数据库角色来设置用户组(管理员，员工，客户)，然后使用RLS根据用户名，ID或电子邮件指定用户规则 。

资源嵌入 (Resource Embedding)

You might have the impression that although PostgREST has a nice solution for authentication and exposing REST endpoints, it’s just your basic CRUD capabilities. And that is not enough for a modern API. After all, that’s why we have things like GraphQL.

您可能会有一种印象，尽管PostgREST有一个不错的身份验证和公开REST终结点的解决方案，但这只是您的基本CRUD功能。 对于现代API而言，这还不够。 毕竟，这就是我们拥有诸如GraphQL之类的原因的原因。

You are right, if all it did was basic CRUD then it would be a nice tool for prototyping or simple projects. But PostgREST has one last trick up it’s sleeve, and I’m proud to call this my biggest contribution. The trick is the&select= parameter. It not only allows you to specify what columns you want returned from your table, but you can also ask for related resources:

您是对的，如果它所做的只是基本的CRUD，那么它将是用于原型设计或简单项目的好工具。 但是PostgREST却有其绝招，我为此感到自豪，这是我最大的贡献 。 诀窍是&select=参数。 它不仅允许您指定要从表中返回哪些列，还可以要求相关资源：

GET /items?select=id,name,subitems(id,name)

If you replace () with {} and squint your eyes a bit, you can almost see GraphQL. PostgREST interface is comparable in expressive powers to GraphQL when it comes to fetching data from your database.

如果用{}代替()并稍微斜视一下，几乎可以看到GraphQL。 从数据库中获取数据时，PostgREST接口在表达能力上可与GraphQL媲美。

It took us a complete core rewrite and then a year of making the interface uniform for all paths. But we finally got there.

我们花了完整的内核重写 ，然后花了一年时间使所有路径的接口统一。 但是我们终于到了。

So how does this work?

那么这是如何工作的呢？

At first boot time, PostgREST runs a bunch of queries on your database. This is to understand what entities live there and the relations between them, based on the foreign keys you have defined. After this, whenever you saysubitems(...) it knows the table items is related to table subitems through a foreign key called item_id. Based on that information it knows how to generate the correct join query. This works similarly for parent relations and many-to-many.

在第一次启动时，PostgREST在您的数据库上运行一堆查询 。 这是根据您定义的外键来了解那里存在哪些实体以及它们之间的关系。 此后，每当您说出subitems(...)它都会通过称为item_id的外键知道表items与表subitems相关。 基于这些信息，它知道如何生成正确的联接查询。 父母关系和多对多关系也是如此。

A simplified (essence) query of that looks like this

一个简化的(本质)查询看起来像这样

SELECT    items.id, items.name,    COALESCE(         (            SELECT array_to_json(array_agg(row_to_json(subitems)))            FROM (                SELECT subitems.id, subitems.name                FROM subitems                WHERE subitems.item_id = items.id             ) subitems         ),         '[]'    ) AS subitemsFROM items

On first look one might think this is very inefficient since it seems to do a sub-select for every item row but we are using PostgreSQL and it’s awesome query planner, it knows what you are actually asking for and it’s got your back. Don’t be afraid of joins and sub-select.

乍一看，可能会认为这是非常低效的，因为它似乎对每个项目行都进行了子选择，但是我们使用的是PostgreSQL，而且它是出色的查询计划程序，它知道您的实际要求并得到了支持。 不要害怕联接和子选择。

Once, after showing this query and explaining the query planner knows how to correctly handle it, I’ve received a response like

有一次，在显示此查询并解释了查询计划者知道如何正确处理它之后，我收到了如下答复

“Ha-ha, yeah, you are relying on a query optimizer’s mercy.”

“哈哈，是的，您要依靠查询优化器的摆布。”

I’ll take that any day, thank you very much!

我会接受任何一天，非常感谢！

I’ll rely on a software developed over a 20 years span by people with PHD’s. I don’t have the arrogance to think that somehow my for loop is superior technology to the PostgreSQL query optimizer.

我将依靠由PHD的人员开发20多年的软件。 我毫不气ance地认为，我的for循环是比PostgreSQL查询优化器优越的技术。

但是为什么(我们需要PostgREST)？ ？ (But why (do we need PostgREST)? ?)

Now if you followed along and understand how things come together, you might be thinking:

现在，如果您继续研究并了解事物如何融合在一起，您可能会想到：

“All PostgREST does is generate one SQL query, wrap it in a transaction with some context and execute it. Why don’t I just skip the middle man, write a script that takes raw SQL as input and does the same thing?”

“ PostgREST所做的只是生成一个SQL查询，将其包装在具有一定上下文的事务中并执行它。 为什么我不跳过中间人，编写一个以原始SQL作为输入并做同样事情的脚本？”

You could do that. Indeed no one would be able to access the data they are not supposed to and things will be fine for a while. And then, one night you’ll wake up to an alert that your database is down. After some investigation, you’ll find as a culprit, this query:

你可以那样做。 确实，没有人能够访问他们不应该访问的数据，并且一段时间后一切都会好起来的。 然后，有一天晚上，您将醒来，提醒您数据库已关闭。 经过一番调查，您会发现以下查询为罪魁祸首：

SELECT   crypt(    encode(digest(gen_random_bytes(1024), 'sha512'), 'base64'),       gen_salt('bf', 20)  )FROM   generate_series(1, 1000000)

WT#!

WT＃！

What’s worse, I don’t need any privileges to any tables to run this query!

更糟糕的是，我不需要任何表特权即可运行此查询！

For all it’s might, PostgreSQL was not built to deflect denial-of-service (DoS) attacks. This is where PostgREST saves you. It strikes a balance between exposing a lot of SQL flexibility to the clients to be useful, but limits that power to prevent malicious queries. It will not allow at will joins, only the ones defined by foreign keys which should have the proper indexes on them.

PostgreSQL并非尽其所能地抵制拒绝服务(DoS)攻击。 这就是PostgREST拯救您的地方。 它在向客户端提供大量SQL灵活性以使其有用之间取得平衡，但限制了防止恶意查询的能力。 它不会允许随意加入，只应该对他们有适当的索引外键定义的值。

一件事做好吗？ (Do one thing well ?)

So you are sold! You’ve downloaded the binary, pointed it to your database and BAM! You have a REST API! Things are moving along great. Most of your API needs are covered. Then you get to a few last bits to implement and suddenly you stop!

所以你被卖了！ 您已经下载了二进制文件，并将其指向数据库和BAM！ 您有一个REST API！ 事情进展顺利。 您的大多数API需求都可以满足。 然后，您需要执行最后几步，然后突然停下来！

“How do I actually send an email with this thing? How do I make a call to this 3rd party API when a user logs in? How do I write my tests?

“实际上我该如何发送电子邮件？ 用户登录后如何调用该第三方API？ 我该如何编写测试？

Damn it, I knew this was a bad idea! Hey PostgREST devs, can you help me out and implement this functionality?”

该死的，我知道这是个坏主意！ 嗨，PostgREST开发人员，您能帮我实施这个功能吗？”

And the answer you’ll most likely get is “You Aren’t Gonna Need It” (YAGNI). That’s not because we are d#$%#. It’s because PostgREST is a component of your stack, it’s not “the stack”. It has one single job and it tries to do it well. On it’s own, it will not completely replace your favorite backend MVC framework. But with some help from friends like PostgreSQL, OpenResty, RabbitMQ, it will do that for you and with great results. Take a look at the Starter Kit to see how it fits into the stack.

您最有可能得到的答案是“您不需要它”(YAGNI)。 那不是因为我们是d＃$％＃。 这是因为PostgREST是堆栈的组成部分，而不是“堆栈”。 它只有一项工作，并且会尽力做好。 就其本身而言，它不会完全取代您喜欢的后端MVC框架。 但是在诸如PostgreSQL，OpenResty，RabbitMQ之类的朋友的帮助下，它将为您实现这一目标，并取得了不错的效果。 看一下入门工具包 ，看看它如何适合堆栈。

You’ll no longer write APIs, you’ll be defining and configuring them.

您将不再编写API，而是要定义和配置它们。

超越REST？ (Beyond REST ?)

Lately the front-end community has been taken over by the React & GraphQL craze, and for a good reason. It might seems like REST will soon be left behind, taking PostgREST along with it. Yet the ideas here transcend REST, it’s the protocol you use to talk to PostgREST.

最近，有一个很好的理由，前端社区已被React＆GraphQL热潮所接管。 似乎REST将很快被遗弃，并将PostgREST与它一起使用。 然而，这里的想法超越了REST，它是您用来与PostgREST进行通信的协议。

You might’ve heard of PostGraphQL. It’s based on the same ideas with a completely different implementation. The author is also one of the top PostgREST contributors. Inspired by PostgREST and discussions about GraphQL within PostgREST community, he decided to put his own spin on it.

您可能听说过PostGraphQL 。 它基于相同的思想，但实现方式却完全不同。 作者还是PostgREST的主要贡献者之一。 受PostgREST和PostgREST社区中有关GraphQL的讨论的启发，他决定在上面发表自己的看法。

I decided to take a different route. I used PostgREST and built GraphQL on top of it, instead of re-implementing the same logic in another language. After all, this was my goal from the very beginning. Build up PostgREST capabilities to a point where it can support GraphQL.

我决定走另一条路。 我使用PostgREST并在其之上构建了GraphQL，而不是用另一种语言重新实现相同的逻辑。 毕竟，这从一开始就是我的目标。 将PostgREST功能建立到可以支持GraphQL的程度。

It’s been a long journey developing my original idea for subZero, a GraphQL & REST API for your database. But I learned a lot along the way.

开发我的subZero原始想法经历了漫长的历程， subZero是用于数据库的GraphQL和REST API。 但是我在此过程中学到了很多东西。

Take subZero for a spin. I hope the additional software we developed along the way is useful to you.

以subZero旋转。 希望我们在开发过程中开发的其他软件对您有用。

Enjoy!

请享用！

If you have any PostgREST or subZero questions, you can always reach me using email or slack.

如果您有任何PostgREST或subZero问题，可以随时通过电子邮件或slack与我联系。

翻译自: https://www.freecodecamp.org/news/stop-calling-postgrest-magic-8f3e1d5e5dd1/

magic feature