想看原文戳这里

1. introduction

(1) the definition of KG

A KG is a multi-relational graph composed of entities (nodes) and relations (different types of edges).

(2) examples

Freebase, DBpedia, YAGO, NELL

(3) key idea of KGE

The key idea is to embed components of a KG including entities and relations into continuous vector spaces, so as to simplify the manipulation while preserving the inherent structure of the KG.

(4) traditional procedure of KGE

Given a KG, such a technique first represents entities and relations in a continuous vector space, and defines a scoring function on each fact to measure its plausibility. Entity and relation embeddings can then be obtained by maximizing the total plausibility of observed facts.

2. notations

…

3. KG Embedding with Facts Alone

(1) facts

Facts observed in the KG are stored as a collection of triples D+={(h,r,t)}. Each triple is composed of a head entity h∈E, a tail entity t∈E, and a relation r∈R between them.

(2) steps

① representing entities and relations

Entities are usually represented as vectors, i.e., deterministic points in the vector space. Relations are typically taken as operations in the vector space.

② defining a scoring function

A scoring function fr(h,t) is defined on each fact (h,r,t) to measure its plausibility. Facts observed in the KG tend to have higher scores than those that have not been observed.

③ learning entity and relation representations

The third step solves an optimization problem that maximizes the total plausibility of observed facts.

(3) embedding techniques

① Translational Distance Models

exploit distance-based scoring functions; measure the plausibility of a fact as the distance between the two entities, usually after a translation carried out by the relation

(i) TransE

-represents both entities and relations as vectors in the same space, say Rd

-Given a fact (h,r,t), the relation is interpreted as a translation vector r so that the embedded entities h and t can be connected by r with low error, i.e., h+r≈t when (h,r,t) holds.

-TransE has flaws in dealing with 1-to-N, N-to-1, and N-to-N relations : given a 1-to-N relation, e.g., DirectorOf, TransE might learn very similar vector representations for Psycho, Rebecca, and RearWindow which are all films directed by AlfredHitchcock, even though they are totally different entities

(ii) TransE’s Extensions

Relation-Specific Entity Embeddings: allow an entity to have distinct representations when involved in different relations

- TransH

* relation-specific hyperplanes

* models entities as vectors

* models each relation r as a vector r on a hyperplane with wr as the normal vector

* By introducing the mechanism of projecting to relation-specific hyperplanes, TransH enables different roles of an entity in different relations.

- TransR

* relation-specific spaces

* entities are represented as vectors in an entity space Rd, and each relation is associated with a specific space Rk and modeled as a translation vector in that space

* Mr∈Rk×d is a projection matrix from the entity space to the relation space of r

- TransD

* simplifies TransR

- TranSparse (share)

* simplifies TransR

- TranSparse (separate)

* simplifies TransR

Relaxing Translational Requirement: relaxing the overstrict requirement of h+r≈t

- TransM

- ManifoldE

- TransF

* only requires t to lie in the same direction with h+r, and meanwhile h in the same direction with t-r

- TransA

(iii) Gaussian Embeddings

take into account entities’ uncertainties, and model them as random variables

- KG2E

- TransG

(iv) Other Distance Models

- UM (unstructured model)

* a naive version of TransE by setting all r=0

- SE (structured embedding)

② Semantic Matching Models

exploit similarity-based scoring functions; measure plausibility of facts by matching latent semantics of entities and relations

(i) RESCAL and Its Extensions

- RESCAL

- TATEC

* models not only the three-way interaction h⊤Mrt but also two-way interactions, e.g., those between an entity and a relation

- DistMult

* this over-simplified model can only deal with symmetric relations

- HolE

* combines the expressive power of RESCAL with the efficiency and simplicity of DistMult

- ComplEx

* extends DistMult by introducing complex-valued embeddings so as to better model asymmetric relations

* entity and relation embeddings h,r,t no longer lie in a real space but a complex space

* every ComplEx has an equivalent HolE

- ANALOGY

* extends RESCAL so as to further model the analogical properties of entities and relations

(ii) Matching with Neural Networks

- Semantic Matching Energy (SME)

- Neural Tensor Network (NTN)

- Multi-Layer Perceptron (MLP)

- Neural Association Model (NAM)

③ Model Training

(i) Training under Open World Assumption

- KGs contain only true facts and non-observed facts can be either false or just missing. D+ stores only positive examples.

- Given the positive set D+ and a negative set D- constructed accordingly, we can learn entity and relation representations Θ by minimizing the logistic loss/ pairwise ranking loss.

- logistic loss :

- pairwise ranking loss :

- Minimizing the pairwise ranking loss has an additional advantage: it does not assume that negative examples are necessarily false, just that they are more invalid than those positive ones.

- the logistic loss generally yields better results for semantic matching models such as DistMult and ComplEx, while the pairwise ranking loss may be more suitable for translational distance models like TransE

- steps

* Initializing Entity and Relation Embeddings

~ usually initialized randomly from uniform distributions or Gaussian distributions

~ also initialized with results of simple models such as TransE

~ also represent an entity as the average word vector of its name or description, and initialize word vectors with those pre-trained on a text corpus

* Generating Negative Training Examples

~ To reduce false-negative examples, (i) set different probabilities for replacing the head and the tail, i.e., to give more chance to replacing the head if the relation is 1-to-N and the tail if the relation is N-to-1 or (ii) given a positive fact, it corrupts a position (i.e., head or tail) using only entities that have appeared in that position with the same relation

~ 50 negatives per positive example is a good trade-off between accuracy and training time

(ii) Training under Closed World Assumption

- all facts that are not contained in D+ are false

- squared loss :

- other possible loss functions include the logistic loss and absolute loss

④ Model Comparison

⑤ Other Approaches

learn representations for head-tail entity pairs rather than individual entities

4. Incorporating Additional Information

(1) Entity Types

① a straight forward method : take IsA as an ordinary relation and the corresponding triples as ordinary training examples

② semantically smooth embedding (SSE)

③ type-embodied knowledge representation learning (TKRL)

(i) can handle hierarchical entity categories and multiple category labels

(ii) a translational distance model with type-specific entity projections

(2) Relation Paths

A relation path is typically defined as a sequence of relations r1→⋯→rℓ through which two entities can be connected on the graph. A key challenge then is how to represent such paths in the same vector space along with entities and relations.

① a straight forward method : represent a path as a composition of the representations of its constituent relations; typical composition operations include addition, multiplication, and recurrent neural network (RNN)

② path-based TransE (PTransE)

…

(3) Textual Descriptions

① align the given KG with an auxiliary text corpus, and then jointly conduct KG embedding and word embedding

② description-embodied knowledge representation learning (DKRL)

③ a text-enhanced KG embedding model, referred to as TEKE

(4) Logical Rules

systems such as WARMR, ALEPH, and AMIE can extract logical rules automatically from KGs

① utilize rules to refine embedding models during KG completion

② a joint model which embeds KG facts and logical rules simultaneously; a key ingredient of this approach, called KALE, is to represent and model facts and rules in a unified framework

③ vector embeddings are introduced for entity pairs rather than individual entities, making it particularly useful for relation extraction

(5) Other Information

Entity Attributes; Temporal Information; Graph Structures; Evidence from Other Relational Learning Methods

5. Applications in Downstream Tasks

…待补充

6. Concluding Remarks

…待补充

Knowledge Graph Embedding: A Survey of Approaches and Applications 摘要相关推荐

Knowledge Graph Embedding: A Survey of Approaches and Applications (2)
本文是对知识图谱相关文章阅读翻译,<Knowledge Graph Embedding: A Survey of Approaches and Applications>只针对文字部分,图 ...
Knowledge Graph Embedding: A Survey of Approaches and Applications【翻译】
原文地址:https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8047276 论文名称:Knowledge Graph Embed ...
论文阅读《Knowledge Graph Refinement: A Survey of Approaches and Evaluation Methods》
论文链接一.谷歌的知识图谱谷歌的知识图谱是在2012年向公众推出的,也是在这个时候,"知识图谱"这个术语被创造出来. 谷歌本身对于知识图谱的构建是相当保密的:只有少数外部来源讨 ...
《knowledge graph embedding：a survey of approaches and applications》论文阅读
发表于TKDE 2017. knowledge graph embedding:a survey of approaches and applications abstract 1. introduc ...
【实体对齐·BootEA】Bootstrapping Entity Alignment with Knowledge Graph Embedding
文章目录 0.总结 1.动机 2. 贡献+方法 3.应用场景 4.其他模型 5.数据集 6.效果以下的是组内比较 BootEA: "Bootstrapping Entity Alignme ...
学习Knowledge Graph Embedding Based Question Answering代码笔记
前言最近被导师安排学习一下[Knowledge Graph Embedding Based Question Answering] 这篇paper,这篇paper的重点在于运用了Knowledge ...
AutoSF- Searching Scoring Functions for Knowledge Graph Embedding
AutoSF: Searching Scoring Functions for Knowledge Graph Embedding Abstract 衡量知识图(KG)中三元组合理性的得分函数(SFs ...
知识图谱-第三方工具：LibKGE（用于Knowledge Graph Embedding）【包含多种模型：TransE、DistMult、ComplEx、ConvE、Transformer等】
用于Knowledge Graph Embedding的向量表示库有很多,比如: LibKGE GraphVite AmpliGraph OpenKE pykeen pykg2vec LibKGE的主 ...
知识图谱论文阅读（十三）【2020 arXiv】Attentive Knowledge Graph Embedding for Personalized Recommendation
题目: Attentive Knowledge Graph Embedding for Personalized Recommendation 论文链接: 代码链接: 知识图谱特征学习在推荐系统中的应 ...

Knowledge Graph Embedding: A Survey of Approaches and Applications 摘要

1. introduction

(1) the definition of KG

(2) examples

(3) key idea of KGE

(4) traditional procedure of KGE

2. notations

3. KG Embedding with Facts Alone

(1) facts

(2) steps

(3) embedding techniques

4. Incorporating Additional Information

(1) Entity Types

(2) Relation Paths

(3) Textual Descriptions

(4) Logical Rules

(5) Other Information

5. Applications in Downstream Tasks

6. Concluding Remarks

Knowledge Graph Embedding: A Survey of Approaches and Applications 摘要相关推荐

最新文章

热门文章