架构垂直伸缩和水平伸缩区别_简单的可伸缩图神经网络

架构垂直伸缩和水平伸缩区别

巨型图上的深度学习 (Deep learning on giant graphs)

TL;DR: One of the challenges that have so far precluded the wide adoption of graph neural networks in industrial applications is the difficulty to scale them to large graphs such as the Twitter follow graph. The interdependence between nodes makes the decomposition of the loss function into individual nodes’ contributions challenging. In this post, we describe a simple graph neural network architecture developed at Twitter that can work on very large graphs.

TL; DR： 迄今为止，阻碍图神经网络在工业应用中广泛采用的挑战之一是难以将它们缩放到大型图(例如Twitter跟随图)。 节点之间的相互依赖性使损失函数分解成单个节点的贡献变得困难。 在这篇文章中，我们描述了一种在Twitter上开发的简单图神经网络架构，该架构可以处理非常大的图。

This post was co-authored with Fabrizo Frasca and Emanuele Rossi.

这篇文章与 Fabrizo Frasca 和 Emanuele Rossi 合着。

Graph Neural Networks (GNNs) are a class of ML models that have emerged in recent years for learning on graph-structured data. GNNs have been successfully applied to model systems of relation and interactions in a variety of different domains, including social science, computer vision and graphics, particle physics, chemistry, and medicine. Until recently, most of the research in the field has focused on developing new GNN models and testing them on small graphs (with Cora, a citation network containing only about 5K nodes, still being widely used [1]); relatively little effort has been invested in dealing with large-scale applications. On the other hand, industrial problems often deal with giant graphs, such as Twitter or Facebook social networks containing hundreds of millions of nodes and billions of edges. A big part of methods described in the literature are unsuitable for these settings.

摹拍摄和神经网络(GNNS)是一类ML车型已经出现在最近几年的学习上图的结构化数据。 GNN已成功应用于各种不同领域的关系和相互作用的模型系统，包括社会科学，计算机视觉和图形，粒子物理学，化学和医学。直到最近，该领域的大多数研究都集中在开发新的GNN模型并在小图形上对其进行测试(使用仅包含约5K节点的引用网络Cora，至今仍在广泛使用[1])；在处理大规模应用程序上投入了相对较少的精力。另一方面，工业问题通常涉及巨型图，例如包含数亿节点和数十亿边缘的Twitter或Facebook社交网络。文献中描述的方法的很大一部分不适用于这些设置。

In a nutshell, graph neural networks operate by aggregating the features from local neighbour nodes. Arranging the d-dimensional node features into an n×d matrix X (here n denotes the number of nodes), the simplest convolution-like operation on graphs implemented in the popular GCN model [2] combines node-wise transformations with feature diffusion across adjacent nodes

简而言之，图神经网络通过聚集来自本地邻居节点的特征进行操作。将d维节点特征排列到n × d矩阵X中 (此处n表示节点数)，流行的GCN模型 [2]中实现的对图的最简单的类似卷积运算将结点变换与整个特征扩散相邻节点

Y = ReLU(AXW).

Y = ReLU( AXW )。

Here W is a learnable matrix shared across all nodes and A is a linear diffusion operator amounting to a weighted average of features in a neighbourhood [3]. Multiple layers of this form can be applied in sequence like in traditional CNNs. Graph neural networks can be designed to make predictions at the level of nodes (e.g. for applications such as detecting malicious users in a social network), edges (e.g. for link prediction, a typical scenario in recommender systems), or the entire graphs (e.g. predicting chemical properties of molecular graphs). The node-wise classification task can be carried out, for instance, by a two-layer GCN of the form

这里W是在所有节点上共享的可学习矩阵， A是线性扩散算子，等于邻域中特征的加权平均值[3]。这种形式的多层可以像传统的CNN一样顺序应用。可以将图神经网络设计为在节点(例如，用于诸如检测社交网络中的恶意用户的应用程序)，边缘(例如，用于链接预测，推荐系统中的典型场景)，或整个图(例如，预测分子图的化学性质)。节点分类任务可以例如通过以下形式的两层GCN来执行

Y = softmax(A ReLU(AXW)W’).

Y = softmax( A ReLU( AXW ) W ')。

Why is scaling graph neural networks challenging? In the aforementioned node-wise prediction problem, the nodes play the role of samples on which the GNN is trained. In traditional machine learning settings, it is typically assumed that the samples are drawn from some distribution in a statistically independent manner. This, in turn, allows to decompose the loss function into the individual sample contributions and employ stochastic optimisation techniques working with small subsets (mini-batches) of the training data at a time. Virtually every deep neural network architecture is nowadays trained using mini-batches.

w ^ HY被缩放图形神经网络的挑战？在上述的逐节点预测问题中，节点扮演训练GNN的样本的角色。在传统的机器学习设置中，通常假设样本是从某种分布中以统计独立的方式提取的。反过来，这又允许将损失函数分解为单个样本贡献，并采用随机优化技术来一次处理训练数据的小子集(微型批次)。如今，几乎每个深度神经网络体系结构都使用小型批次进行培训。

In graphs, on the other hand, the fact that the nodes are inter-related via edges creates statistical dependence between samples in the training set. Moreover, because of the statistical dependence between nodes, sampling can introduce bias — for instance it can make some nodes or edges appear more frequently than on others in the training set — and this ‘side-effect’ would need proper handling. Last but not least, one has to guarantee that the sampled subgraph maintains a meaningful structure that the GNN can exploit.

另一方面，在图中，节点通过边相互关联的事实在训练集中的样本之间产生了统计依赖性。此外，由于节点之间的统计依赖性，采样可能会引入偏差(例如，它可能使某些节点或边缘比训练集中的其他节点或边缘出现得更频繁)，并且这种“副作用”需要适当处理。最后但并非最不重要的一点是，必须保证采样的子图保持GNN可以利用的有意义的结构。

In many early works on graph neural networks, these problems were swept under the carpet: architectures such as GCN and ChebNet [2], MoNet [4] and GAT [5] were trained using full-batch gradient descent. This has led to the necessity to hold the whole adjacency matrix of the graph and the node features in memory. As a result, for example, an L-layer GCN model has time complexity

架构垂直伸缩和水平伸缩区别_简单的可伸缩图神经网络相关推荐

从RAID看垂直伸缩到水平伸缩的演化
learn from 从0开始学大数据(极客时间) 大规模数据存储问题: 容量问题,数据量超过磁盘容量读写速度,磁盘读写慢数据可靠性,磁盘寿命问题 RAID(独立磁盘冗余阵列) 是将多块普通磁盘组 ...
图神经网络代码_第一篇：图神经网络(GNN)计算框架绪论
写在开头: 这个专栏是为了总结我本科毕业设计中所设计的题目<基于GPU的图神经网络算法库的设计钰实现>.这半年来一直在这个方向上啃代码,读论文,真的学到了很多东西.尤其是阅读了大佬团队写的 ...
linux桌面版和服务器版区别_简单普及一些linux和windows的区别，减少新入手一些云服务时的不适应...
近一年,发的linux或者服务器之类的教程资源相对比较多,被问到一些令人头大的一些问题,也不知道怎么回答,可能需要稍微普及一点linux和windows的区别,虽然自己也是新人半桶水. 1.windo ...
python write和writelines的区别_简单了解Python write writelines区别
一.传入的参数类型要求不同: 1. file.write(str)需要传入一个字符串做为参数,否则会报错. write( "字符串") with open('20200222.tx ...
线框图和原型图的区别_有用的线框图和原型制作工具–综述
从年轻人到老年人,全球化的概念以不断的创造力和想象力冲击着世界. 从创造力在营销,线框设计和原型制作领域中创造机会这一事实来看,它绝对在列表中. 这些工具不仅可以使您的网站看起来漂亮整洁,而且可以吸引 ...
HTML元素的水平/垂直居中方式（简单代码和图）
1.水平居中text-align:center; 在没有浮动的情况下,我们可以让需要居中的块级元素设为inline/inline-block,然后在其父元素上添加text-aline:center;即 ...
探索图神经网络的网络架构和训练方法
摘要:本文我们将主要介绍各种典型的图神经网络的网络架构和训练方法. 本文我们将主要介绍各种典型的图神经网络的网络架构和训练方法.文章<A Comprehensive Survey on Grap ...
k8s HPA(HorizontalPodAutoscaler)-自动水平伸缩
Horizontal Pod Autoscaling in Kubernetes 写在前面我们平时部署web服务,当服务压力大撑不住的时候,我们会加机器(加钱):一般没有上容器编排是手动加的,临时加 ...
HPA 自动水平伸缩 POD
前戏我们知道,初始Pod的数量是可以设置的,同时业务也分流量高峰和低峰,那么怎么即能不过多的占用K8s的资源,又能在服务高峰时自动扩容pod的数量呢,在K8s上的答案是Horizontal Po ...

架构垂直伸缩和水平伸缩区别_简单的可伸缩图神经网络

巨型图上的深度学习 (Deep learning on giant graphs)

架构垂直伸缩和水平伸缩区别_简单的可伸缩图神经网络相关推荐

最新文章

热门文章