Aditya Pal | Applied Science, Chantat Eksombatchai | Applied Science, Yitong Zhou | User Understanding, Bo Zhao | User Understanding, Charles Rosenberg | Applied Science, Jure Leskovec | Applied Science

Aditya Pal | 应用科学，Chantat Eksombatchai | 应用科学，周一彤| 用户理解，赵波| 用户理解，Charles Rosenberg | 应用科学，Jure Leskovec | 应用科学

As we build a visual discovery engine that powers 2B+ Pins, it’s crucial to understand user interests and preferences in order to serve relevant content. One standard approach to encode user preferences is via an embedding-based representation in a high dimensional space. Most prior methods tried at Pinterest infer a single high-dimensional embedding for each user in compatibility with the content embedding. This is a good starting point but falls short in delivering a full understanding of the user.

当我们构建为2B +引脚供电的视觉发现引擎时，了解用户的兴趣和偏好以提供相关内容至关重要。 一种对用户偏好进行编码的标准方法是通过在高维空间中进行基于嵌入的表示。 在Pinterest上尝试的大多数现有方法都会为每个用户推断与内容嵌入兼容的单个高维嵌入。 这是一个很好的起点，但不足以全面了解用户。

In this work, we postulate that a single embedding is not sufficient for encoding multiple facets of a user’s interests that might have no obvious linkage between them. They can evolve, with some interests persisting long term while others span a short time period. Recommended items are also represented in the same embedding space. A good embedding must encode a user’s multiple tastes, interests, styles, etc., whereas a recommended item (a video, an image, a news article, a house listing, a pin, etc.) typically only has a single focus. Hence it becomes important to represent a user with multiple embeddings, with each embedding capturing a specific aspect of their interest.

在这项工作中，我们假设单个嵌入不足以对用户兴趣的多个方面进行编码，而这些方面之间可能没有明显的联系。 它们可以发展，某些利益可以长期持续，而其他利益则可以在很短的时间内出现。 推荐项目也显示在相同的嵌入空间中。 良好的嵌入必须对用户的多种口味，兴趣，风格等进行编码，而推荐项(视频，图像，新闻，房屋列表，图钉等)通常只有一个重点。 因此，重要的是用多个嵌入来表示用户，每个嵌入都捕获他们感兴趣的特定方面。

PinnerSage模型 (PinnerSage Model)

In order to better understand our users’ preferences, we developed PinnerSage, a highly scalable, flexible and extensible recommender system that internally represents each user with multiple embeddings. Figure 1 provides an end-to-end overview of the PinnerSage recommendation model. The starting point for our model is to organize the repins and clicks of a user into multiple interest clusters by running the Ward clustering model and then generating a summary of each of those clusters using a medoid, an embedding, and a cluster importance score. Next, a subset of these clusters are picked by the online cluster selection, and it employs a nearest-neighbor index to generate recommendations to the user. Users’ actions are processed in real-time to update the interest clusters. In order for PinnerSage to provide relevant recommendations to our 400M+ monthly active users and adapt in real-time, we made several model design choices that we describe next.

为了更好地了解我们的用户的偏好，我们开发了PinnerSage，这是一种高度可扩展，灵活且可扩展的推荐系统，可以在内部代表每个用户多个嵌入对象。 图1提供了PinnerSage推荐模型的端到端概述。 我们模型的出发点是，通过运行Ward聚类模型 ，然后使用medoid，嵌入和聚类重要性评分，将用户的拨动和点击组织到多个兴趣聚类中，以生成每个聚类的摘要。 接下来，通过在线集群选择来选择这些集群的子集，并且它使用最近邻居索引来生成对用户的推荐。 实时处理用户的操作以更新兴趣群。 为了让PinnerSage为我们的400M +月活跃用户提供相关建议并实时进行调整，我们做出了以下几个模型设计选择。

设计选择1：固定引脚嵌入 (Design Choice 1: Pin Embeddings are Fixed)

The interest clusters in Figure 1 are generated by clustering the embeddings of repins and clicks of a user. The embeddings of repins and clicks are trained via the PinSage model that optimizes for contextual and visual similarity between Pins via a Graph convolutional model. Since our goal is to project users in the same space as the Pin embedding space, we consider the Pin embeddings to be fixed. This design choice simplifies our models considerably and allows us to run inference pipelines in parallel for each user.

图1中的兴趣聚类是通过聚类用户的图钉和点击的嵌入而生成的。 通过PinSage模型训练图钉和点击的嵌入，该模型通过Graph卷积模型优化图钉之间的上下文和视觉相似性。 由于我们的目标是在与Pin嵌入空间相同的空间中投影用户，因此我们认为Pin嵌入是固定的。 这种设计选择大大简化了我们的模型，并允许我们为每个用户并行运行推理管道。

Joint embedding inference models, where both user and Pin embeddings are inferred together, can be too complex and hard to scale. Moreover, we posit that in practice they compromise recommendation relevance, as some spurious connections between pins can be established via the users. To see this point, consider the example in Figure 2.

联合嵌入推理模型(将用户嵌入和Pin嵌入一起推断)可能太复杂且难以扩展。 此外，我们认为在实践中它们会损害推荐的相关性，因为可以通过用户在引脚之间建立一些虚假的连接。 要了解这一点，请考虑图2中的示例。

In the above example figure, a user is interested in painting, shoes, and sci-fi. Jointly learned users and Pin embeddings would bring pin embeddings on these disparate topics closer, which can compromise the relevance of the nearest neighbor-based recommender. Pin embeddings should only operate on the underlying principle of bringing similar pins closer while keeping the rest of the pins as far as possible. For this reason, we use PinSage, which precisely achieves this objective without any dilution.

在上面的示例图中，用户对绘画，鞋子和科幻小说感兴趣。 共同学习的用户和Pin嵌入将使这些不同主题上的pin嵌入更加紧密，这可能会损害最近的基于邻居的推荐者的相关性。 引脚嵌入仅应遵循使相似的引脚靠近的基本原则，同时将其余的引脚保持尽可能远。 因此，我们使用PinSage，无需任何稀释即可精确实现此目标。

设计选择2：无限的用户嵌入 (Design Choice 2: Unlimited User Embeddings)

Prior work either fixes the number of embeddings to a small number or puts an upper bound on them. At best, such restrictions hinder developing a full understanding of the users and, at worst, merge different concepts together, leading to bad recommendations. For example, merging embeddings could yield an embedding that lies in a very different region. Figure 2 shows that a merger of three disparate pin embeddings results in an embedding that is best represented by the concept energy boosting breakfast. Needless to say, recommendations based on such a merger can be problematic.

先前的工作要么将嵌入数量固定为少量，要么将嵌入数量设置为上限。 最好的情况是，这样的限制阻碍了对用户的全面了解，最坏的情况是将不同的概念融合在一起，从而导致不好的建议。 例如，合并嵌入可能会产生位于非常不同区域中的嵌入。 图2显示，三个不同的针状嵌入物的合并产生的嵌入物最好用能量增强早餐的概念来表示。 不用说，基于这样的合并的建议可能会有问题。

PinnerSage generates as many interest clusters as the underlying data supports. This is achieved by clustering users’ actions into conceptually coherent clusters via a hierarchical agglomerative clustering algorithm (Ward). A light user might get represented by 3–5 clusters, whereas a heavy user might get represented by 75–100 clusters.

PinnerSage会生成基础数据支持的尽可能多的兴趣集群。 这是通过层次化的聚集聚类算法(Ward)将用户的行为聚类为概念上一致的聚类来实现的。 轻量级用户可能由3–5个群集代表，而重度用户可能由75–100个群集代表。

设计选择3：基于Medoid的集群表示 (Design Choice 3: Medoid-based Cluster Representation)

Typically, clusters are represented by centroid, which requires storing an embedding. Additionally, centroid can be sensitive to outliers in the cluster. To compactly represent a cluster, we pick a cluster member pin, called medoid. Medoid, by definition, is a member of the user’s originally interacted pin set. Hence it avoids the pit-fall of topic drift and is robust to outliers. From a systems perspective, medoid is a concise way of representing a cluster, as it only requires storage of medoid’s pin id, and leads to cross-user and even cross-application cache sharing. It also allows our system to be compatible with other non-embedding-based recommendation systems such as Pixie.

通常，群集以质心表示，需要存储嵌入。 此外，质心可能对聚类中的离群值敏感。 为了紧凑地表示群集，我们选择一个群集成员销钉，称为medoid。 根据定义，Medoid是用户最初交互的密码集的成员。 因此，它避免了主题漂移的陷阱，并且对异常值具有鲁棒性。 从系统角度来看，medoid是表示集群的一种简洁方法，因为它仅需要存储medoid的pin ID，并导致跨用户甚至跨应用程序的缓存共享。 它还使我们的系统与其他基于非嵌入的推荐系统(例如Pixie)兼容。

设计选择4：候选检索的Medoid采样 (Design Choice 4: Medoid Sampling for Candidate Retrieval)

PinnerSage provides a rich representation of a user via cluster medoids. However, in practice we cannot use all the medoids simultaneously for candidate retrieval due to cost concerns. Additionally, the user would be bombarded with too many different items. To address these concerns, we sample 3 medoids proportional to their importance scores and recommend their nearest neighboring pins. The importance scores of medoids are updated daily, and they can adapt with the user’s changing tastes.

PinnerSage通过聚类medoids提供了丰富的用户表示。 但是，实际上，由于成本方面的考虑，我们无法同时使用所有类固醇进行候选检索。 另外，用户会被太多不同的物品轰炸。 为了解决这些问题，我们对3种类固醇按其重要性得分成比例进行采样，并推荐与它们最接近的相邻引脚。 类固醇的重要性评分每天都会更新，它们可以适应用户不断变化的口味。

设计选择5：处理实时更新的两管齐下的方法 (Design Choice 5: Two-Pronged Approach for Handling Real-Time Updates)

It is important for a recommender system to adapt to the current needs of its users. At the same time, an accurate representation of users requires looking at their past 60–90 days of activities. Sheer size of the data and the speed at which it grows makes it hard to consider both aspects together. We address this issue by combining two methods: (a) a daily batch inference job that infers multiple medoids per user based on their long-term interaction history, and (b) an online version of the same model that infers medoids based on the users’ interactions on the current day. As new activity comes in, only the online version is updated. At the end of the day, the batch version consumes the current day’s activities and resolves any inconsistencies. This approach ensures that our system adapts quickly to the users’ current needs and at the same time does not compromise their long-term interests.

推荐系统必须适应其用户的当前需求，这一点很重要。 同时，要准确地表示用户，需要查看他们过去60-90天的活动。 数据的庞大规模及其增长速度使得很难同时考虑这两个方面。 我们通过以下两种方法来解决此问题：(a)每日批处理推断工作，根据其长期交互历史来推断每个用户的多个类固醇；以及(b)相同模型的在线版本，根据用户来推断类固醇当天的互动。 随着新活动的到来，仅在线版本被更新。 在一天结束时，批处理版本将消耗当日的活动并解决所有不一致问题。 这种方法可确保我们的系统快速适应用户的当前需求，同时又不损害其长期利益。

A / B测试 (A/B Tests)

PinnerSage is currently deployed in production and used by many products within Pinterest, ranging from Homefeed, Related Pins, Ads, Shopping, and Creators, in both their retrieval and ranking ML models. Our wins on the initial A/B test on two surfaces are highlighted in Table 1.

PinnerSage目前已在生产中部署，并且在Pinterest内的许多产品(包括Homefeed，相关的Pins，广告，购物和Creators)的检索和排名ML模型中都使用。 表1突出显示了我们在两个表面上进行初始A / B测试的胜利。

Table 1 shows that PinnerSage provides significant engagement gains on increasing overall engagement volume (repins and clicks) as well as increasing engagement propensity (repins and clicks per user). Any gain can be directly attributed to increased quality and diversity of PinnerSage recommendations.

表1显示，PinnerSage通过增加总体参与量(回复和点击)以及提高参与倾向(每位用户的回复和点击)可以显着提高参与度。 任何收益都可以直接归因于PinnerSage建议的质量提高和多样性。

结论 (Conclusion)

We proposed an end-to-end system, called PinnerSage, that powers personalized recommendation at Pinterest. In contrast to prior production systems that are based on a single embedding-based user representation, PinnerSage proposes a multi-embedding-based user representation scheme. Our proposed clustering scheme ensures that we get full insight into the needs of a user and understand them better. To make this happen, we adopt several design choices that allow our system to run efficiently and effectively. Our large A/B tests show that PinnerSage provides significant gains in user engagement. Much of the improvements delivered by our model can be attributed to its better understanding of user interests and its quick response to their needs.

我们提出了一个名为PinnerSage的端到端系统，该系统可为Pinterest提供个性化推荐。 与基于单个基于嵌入的用户表示的现有生产系统相比，PinnerSage提出了一种基于多嵌入的用户表示方案。 我们提出的集群方案可确保我们全面了解用户需求并更好地理解它们。 为了实现这一目标，我们采用了几种设计选择，使我们的系统能够高效运行。 我们的大型A / B测试表明，PinnerSage可显着提高用户参与度。 我们的模型提供的许多改进都可以归因于其对用户兴趣的更好理解以及对用户需求的快速响应。

附录 (Appendix)

PinnerSage paper is to appear in KDD 2020. Read more details about the paper here: https://arxiv.org/abs/2007.03634

PinnerSage论文将出现在KDD 2020中。有关此论文的更多详细信息，请访问： https ：//arxiv.org/abs/2007.03634

致谢 (Acknowledgements)

We would like to extend our appreciation to Homefeed and Shopping teams for helping in setting up online A/B experiments. Our special thanks to the embedding infrastructure team for powering embedding nearest neighbor search.

我们要感谢Homefeed和Shopping团队帮助建立在线A / B实验。 我们特别感谢嵌入基础架构团队为嵌入最近邻居搜索提供支持。