Named Entity Recognition with Small Strongly Labeled and Large Weakly Labeled Data

整体评价

模型的工程性较强，面向实际工程中的：标注数据集少，通过弱监督方式得到弱监督数据集的情形。---------have both a small amount of strongly labeled data and a large amount of weakly labeled data

弱标记数据在训练过程中与强标记数据简单结合时，会降低模型性能

模型结构

在有限的label data上，已有的研究成果

在无标注数据集上，重新迭代训练已有的预训练模型，以使得更好的适用于新的domain。

模型方法

**（1）采用PLM在target domain 的数据集上做预训练
（2）使用domain领域的知识库生成弱监督数据集（弱监督有标注数据集），并使用（1）中的模型，在弱监督数据集和labeled data 上继续微调。**这个过程称为noise-aware-weakly supervised learning，包括两个部分： weak label completion procedure and noise-aware loss function.

Weak Label Completion
the encoder θenc is initialized from Stage I and NER CRF head is randomly initialized.
对于一个给定的弱监督的标注句子，其自身已有的label表示为Yw,利用模型补充的label为Yc,
**定义的规则是：**当Yw是O时，采用CRF预测的label作为补充的label，其他情况下，采用弱监督的label作为其label。

Noise-Aware Loss Function——true label为Y，weak label为Yc
在weak data上的损失函数计算公式为：

在weak和labeled data上的，总的损失函数的计算公式是：

（3）将（2）预训练后的model在labeled data 重新微调。

实验部分

参数搜索（grid search）
ablaltion study

• WLC: Weak label completion.
• NAL: Noise-aware loss function, i.e., Eq.(4).
Since NAL is built on top of WLC, the two components need to be used together.
• FT: Final fine-tuning on strongly labeled data
(Stage III).

分析部分
- size of weak data
- Size of Strongly Labeled Data.
- Weak Label Errors
  - Label Distribution Mismatch.
  - Entity BIO Sequence Mismatch in Weak Label
    Completion. Another error of the weakly labels is
    the mismatched entity BIO sequence in the weak
    label completion step, e.g., B-productType followed by I-color

论文：Named Entity Recognition with Small Strongly Labeled and Large Weakly Labeled Data相关推荐

ACL21 - Named Entity Recognition with Small Strongly Labeled and Large Weakly Labeled Data
Jiang H, Zhang D, Cao T, et al. Named Entity Recognition with Small Strongly Labeled and Large Weakl ...
复现：Named Entity Recognition with Small Strongly Labeled and Large Weakly Labeled Data
信达雅的原则: 数据->模型->训练. Named Entity Recognition with Small Strongly Labeled and Large Weakly Labe ...
【论文笔记-NER综述】A Survey on Deep Learning for Named Entity Recognition
本笔记理出来综述中的点,并将大体的论文都列出,方便日后调研使用查找,详细可以看论文. 神经网络的解释: The forward pass com- putes a weighted sum of th ...
[论文阅读笔记14]Nested named entity recognition revisited
一, 题目 Nested Named Entity Recognition Revisited 重访问的嵌套命名实体识别二, 作者 Arzoo Katiyar and Claire Cardie D ...
论文阅读笔记-FGN: Fusion Glyph Network for Chinese Named Entity Recognition
论文地址:paper:https://arxiv.org/ftp/arxiv/papers/2001/2001.05272.pdf github地址:github:https://github.com ...
论文阅读：A Survey on Deep Learning for Named Entity Recognition
这是一篇2020年发的命名实体识别的综述性论文,从NER的语料库,定义,评估指标,到深度学习中的NER的技术都有涉及到. A Survey on Deep Learning for Named Ent ...
论文解读：Example-Based Named Entity Recognition
论文解读:Example-Based Named Entity Recognition 核心要点: 本篇工作是比较早的给出了如何定义和实现few-shot NER任务,后续诸多论文均引用该工作: 本文 ...
论文笔记：Few-Shot Named Entity Recognition: An Empirical Baseline Study
Few-Shot Named Entity Recognition: An Empirical Baseline Study 看法整体感觉,这篇文章是工程实践中可以参考的,大杂烩的感觉,复现一次,应 ...
[论文笔记]AAAI-2021-Continual Learning for Named Entity Recognition
[论文笔记] 2021-AAAI-Continual Learning for Named Entity Recognition 0 写在前面什么是持续学习? 我们人类有能够将一个任务的知识用到另一 ...

论文：Named Entity Recognition with Small Strongly Labeled and Large Weakly Labeled Data