一、Abstract

1、推理是人工智能的关键体现

2、SNLI为推理模型提供数据支撑

3、文中提出的模型不需要复杂的模型结构且效果较好

4、融合语法树模型，效果更佳

二、Introduction

主要介绍数据集和推理的相关背景知识

三、Related Work

简单的介绍了相关工作以及数据集的相关应用情况

四、Hybrid Neural Inference Models

模型结构输入层-输入编码层，即输入的两个匹配文本数值矩阵通过双向LSTM获得输出，与此同时还给出Tree-LSTM网络结构

局部推理建模层: 相当于注意力层，计算注意力矩阵并通过element-wise添加特征维度

推理组合层: 将两句话特征矩阵通过池化操作得到最终特征向量，实现特征聚合

输出预测层: 利用全连接 + softmax进行分类

五、Experiment Setup

实验参数的一些设置，包括学习率、batchsize大小、wordEmbedding维度等

六、Results

实验进行了多组变量分析，最后分析发现 ESIM + tree-LSTM组合效果最好

七、Conclusions and Future Work

主要是对文章进行总结并对未来进行展望

八、Code

# -*- coding: utf-8 -*-# @Time : 2021/2/16 下午6:54
# @Author : TaoWang
# @Description : ESIM网络结构import torch
import torch.nn as nn
import numpy as np
import pandas as pd
from torchtext import data
from torchtext.vocab import Vectors
from torchtext.data import Field, Example, Iteratorclass ESIM(nn.Module):def __init__(self, text):super(ESIM, self).__init__()self.embedding = nn.Embedding(*text.vocab.vectors.size())self.embedding.weight.data.copy_(text.vocab.vectors)# 输入编码层self.a_bilstm_input = nn.LSTM(text.vocab.vectors.size()[1], hidden_size, batch_first=True, bidirectional=True)self.b_bilstm_input = nn.LSTM(text.vocab.vectors.size()[1], hidden_size, batch_first=True, bidirectional=True)# 推理组合层self.a_bilstm_infer = nn.LSTM(8 * hidden_size, hidden_size, batch_first=True, bidirectional=True)self.b_bilstm_infer = nn.LSTM(8 * hidden_size, hidden_size, batch_first=True, bidirectional=True)# 预测层全连接网络self.linear = nn.Sequential(nn.Linear(8 * hidden_size, 2 * hidden_size),nn.ReLU(True),nn.Linear(2 * hidden_size, linear_size),nn.ReLU(True),nn.Linear(linear_size, classes))def forward(self, a, b):""":param a: :param b: :return: """# 词嵌入emb_a, emb_b = self.embedding(a), self.embedding(b)# 输入编码层a_ba, _ = self.a_bilstm_input(emb_a)b_ba, _ = self.b_bilstm_input(emb_b)# 局部推理建模层e = torch.matmul(a_ba, b_ba.permute(0, 2, 1))a_ti = torch.matmul(torch.softmax(e, dim=2), b_ba)b_ti = torch.matmul(torch.softmax(e, dim=1).permute(0, 2, 1), a_ba)# 矩阵拼接ma = torch.cat([a_ba, a_ti, a_ba - a_ti, a_ba * a_ti], dim=2)mb = torch.cat([b_ba, b_ti, b_ba - b_ti, b_ba * b_ti], dim=2)# 推理组合层va, _ = self.a_bilstm_infer(ma)vb, _ = self.b_bilstm_infer(mb)va_avg, va_max = torch.mean(va, dim=1), torch.max(va, dim=1)[0]vb_avg, vb_max = torch.mean(vb, dim=1), torch.max(vb, dim=1)[0]v = torch.cat([va_avg, va_max, vb_avg, vb_max], dim=1)# 输出预测层out = self.linear(v)return out

ESIM (Enhanced LSTM for Natural Language Inference)相关推荐

论文笔记翻译——Enhanced LSTM for Natural Language Inference(ESIM)
文章目录 0.前言 1. 模型结构 1.1 input encoding 1.2 local inference modeling 1.3 inference composition 2. 模型实现 ...
ESIM：Enhanced LSTM for Natural Language Inference
原文链接:https://aclanthology.org/P17-1152.pdf ACL 2017 概述对于自然语言推理任务,Bowman等人在2015年提出了一个大数据集,大多数工作就开始使用 ...
Enhanced LSTM for Natural Language Inference-学习笔记
Enhanced LSTM for Natural Language Inference 自然语言推理(NLI: natural language inference)问题:即判断能否从一个前提p中推 ...
NATURAL LANGUAGE INFERENCE OVER INTERACTION SPACE
文章目录 ABSTRACT 1 INTRODUCTION 2 RELATED WORK 3 MODEL 3.1 INTERACTIVE INFERENCE NETWORK 3.2 DENSELY IN ...
论文解读: Exploiting Cloze Questions for Few Shot Text Classiﬁcation and Natural Language Inference
前言:在自然语言研究领域,某些自然语言处理任务可以利用"预训练模型+任务描述"方式的无监督学习来解决.然而,已有的"预训练模型+任务描述"的无监督学习方法不如有 ...
论文解读：Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference
论文解读:Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference 随着 ...
【提示学习】Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference
论文信息名称内容论文标题 Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inf ...
EMNLP2020 | 近期必读Natural Language Inference精选论文
**AMiner平台**由清华大学计算机系研发,拥有我国完全自主知识产权.平台包含了超过2.3亿学术论文/专利和1.36亿学者的科技图谱,提供学者评价.专家发现.智能指派.学术地图等科技情报专业化服务 ...
Natural Language Inference Over Interaction Space(DIIN)阅读笔记
文章目录 Introduction Model Interactive Inference Network(IIN) Densely Interactive Inference Network(DII ...

ESIM (Enhanced LSTM for Natural Language Inference)