一种新的基于深度学习的重叠关系联合抽取模型（英文）

doi:10.7523/j.ucas.2020.0026

中国科学院大学学报 ›› 2022, Vol. 39 ›› Issue (2): 240-251.DOI: 10.7523/j.ucas.2020.0026

• 电子信息与计算机科学 • 上一篇下一篇

一种新的基于深度学习的重叠关系联合抽取模型（英文）

赵敏钧¹, 赵亚伟¹, 赵雅捷², 罗刚²

1 中国科学院大学工程科学学院, 北京 100049;
2 北京知因智慧科技有限公司AI实验室, 北京 100088

收稿日期:2020-03-23 修回日期:2020-05-25 发布日期:2021-06-01
通讯作者: 赵亚伟
基金资助:
Supported by the National Natural Science Foundation of China (61872331) and University of Chinese Academy of Sciences

A new joint model for extracting overlapping relations based on deep learning

ZHAO Minjun¹, ZHAO Yawei¹, ZHAO Yajie², LUO Gang²

1 School of Engineering Science, University of Chinese Academy of Sciences, Beijing 100049, China;
2 AI Lab of KnowLeGene Intelligent Technology Co, Ltd, Beijing 100088, China

Received:2020-03-23 Revised:2020-05-25 Published:2021-06-01
Supported by:
Supported by the National Natural Science Foundation of China (61872331) and University of Chinese Academy of Sciences

摘要/Abstract

摘要： 随着互联网技术的快速发展和移动设备的普及，我们每时每刻都被各种各样的信息包围着。如何从海量的数据中挖掘出具有价值的信息一直是国内外研究的热点。其中，关系抽取是信息抽取的一个重要子任务，目的是从文本中识别出实体之间的关系，从而挖掘出文本中的结构化信息，即事实三元组。在文本中，实体重叠和关系重叠是非常普遍的现象，但是现有的联合抽取模型不能够有效地解决这类问题，因此提出一种新的联合抽取模型，将关系抽取任务看作由2个子任务实体识别和关系识别组成，并分别使用序列标注的方法和多分类方法进行识别。在联合抽取过程中，为充分挖掘文本语义信息，在模型的输入层添加词性（POS）和句法依存关系（Deprel）特征，同时为消除随着句子长度增加带来的长距离依赖问题，在模型中引入注意力机制。最后，论文在NYT数据集和WebNLG数据集上进行关系抽取实验，结果表明论文提出的模型能够有效地解决关系重叠的问题，并取得最佳抽取效果。

关键词: 关系抽取, 实体重叠, 联合抽取模型, 深度学习

Abstract: With the rapid developments of Internet technologies and popularization of Internet among daily activities, we are surrounded by all kinds of information every moment. Hence, to mine valuable information from massive data has always been a hotspot of research at home and abroad. In this environment, relationship extraction is an important subtask of information extraction, which purpose is to identify the relationship between entities from the text, so as to mine the structured information in the text, that is, fact triplet. In the text, entity overlapping and relationship overlapping are very common phenomena, but the existing joint extraction model cannot effectively solve such problems, so the paper proposes a new joint extraction model, which regards the relationship extraction task as consisting of entity recognition and relationship recognition of two subtasks. The two subtasks are identified using sequence labeling method and multi-classification method, respectively. In the joint extraction process, in order to fully mine the semantic information of the text, the part of speech (POS) and syntactic dependency (Deprel) features were added to the input layer of the model. Attention mechanism is also introduced in the model, which can eliminate the problem of long-distance dependence as sentence length increases. Finally, the paper conducts relationship extraction experiments on the NYT dataset and the WebNLG dataset. The experimental results show that the model proposed in the paper can effectively solve the problem of overlapping relationships and obtain the best extraction effect.

Key words: relation extraction, entity overlapped, joint extraction model, deep learning

中图分类号:

赵敏钧, 赵亚伟, 赵雅捷, 罗刚. 一种新的基于深度学习的重叠关系联合抽取模型（英文）[J]. 中国科学院大学学报, 2022, 39(2): 240-251.

ZHAO Minjun, ZHAO Yawei, ZHAO Yajie, LUO Gang. A new joint model for extracting overlapping relations based on deep learning[J]. Journal of University of Chinese Academy of Sciences, 2022, 39(2): 240-251.

参考文献

[1] Zelenko D, Aone C, Richardella A. Kernel methods for relation extraction[C]//Proceedings of the ACL-02 conference on Empirical methods in natural language processing-EMNLP’02. Not Known. Morristown, NJ, USA: Association for Computational Linguistics, 2002: 71-78. DOI:10.3115/1118693.1118703.
[2] Chan Y S, Roth D. Exploiting syntactico-semantic structures for relation extraction[C]//Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1. Portland, Oregon, 2011: 551-560.
[3] Li Q, Ji H. Incremental joint extraction of entity mentions and relations[C]//Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Baltimore, Maryland. Stroudsburg, PA,USA: Association for Computational Linguistics, 2014: 402-412. DOI:10.3115/v1/p14-1038.
[4] Yu X, Lam W. Jointly identifying entities and extracting relations in encyclopedia text via a graphical model approach[C]//Proceedings of the 23rd International Conference on Computational Linguistics: Posters. Association for Computational Linguistics. August 23-27,2010, Beijing, China. 2010: 1399-1407.
[5] Miwa M, Bansal M. End-to-end relation extraction using LSTMs on sequences and tree structures[C]// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Berlin, Germany. Stroudsburg, PA, USA: Association for Computational Linguistics, 2016: 1105-1116. DOI:10.18653/v1/p16-1105.
[6] Dai D, Xiao X Y, Lyu Y J, et al. Joint extraction of entities and overlapping relations using position-attentive sequence labeling[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2019, 33:6300-6308. DOI:10.1609/aaai.v33i01.33016300.
[7] Zeng X R, Zeng D J, He S Z, et al. Extracting relational facts by an end-to-end neural model with copy mechanism[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Melbourne, Australia. Stroudsburg, PA, USA: Association for Computational Linguistics, 2018: 506-514. DOI:10.18653/v1/p18-1047.
[8] Collobert R, Weston J, Bottou L, et al. Natural language processing (almost) from scratch[J]. IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences, 2011, abs/1103.0398: 2493-2537.
[9] Ma X Z, Hovy E. End-to-end sequence labeling via Bi-directional LSTM-CNNs-CRF[EB/OL]. 2016: arXiv: 1603.01354[cs.LG].(2016-03-04) [2020-05-20] https://arxiv.org/abs/1603.01354.
[10] Auer S, Bizer C, Kobilarov G, et al. DBpedia: a nucleus for a web of open data[C]//The Semantic Web. Springer, Berlin, Heidelberg, 2007: 722-735. DOI:10.1007/978-3-540-76298-0_52.
[11] Bollacker K, Evans C, Paritosh P, et al. Freebase: a collaboratively created graph database for structuring human knowledge[C]//SIGMOD ′08: Proceedings of the 2008 ACM SIGMOD international conference on Management of data. 2008: 1247-1250. DOI:10.1145/1376616.1376746.
[12] Nadeau D, Sekine S. A survey of named entity recognition and classification[J]. Lingvisticae Investigationes, 2007, 30(1): 3-26. DOI:10.1075/li.30.1.03nad.
[13] Rink B, Harabagiu S. Utd: classifying semantic relations by combining lexical and semantic resources[C]//Proceedings of the 5th International Workshop on Semantic Evaluation. Association for Computational Linguistics, 2010: 256-259.
[14] Luo G, Huang X J, Lin C Y, et al. Joint entity recognition and disambiguation[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Lisbon, Portugal. Stroudsburg, PA, USA: Association for Computational Linguistics, 2015: 879-888. DOI:10.18653/v1/d15-1104.
[15] Lample G, Ballesteros M, Subramanian S, et al. Neural architectures for named entity recognition[C]//Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. San Diego, California. Stroudsburg, PA, USA: Association for Computational Linguistics, 2016: 260-270. DOI:10.18653/v1/n16-1030.
[16] Huang Z H, Xu W, Yu K. Bidirectional LSTM-CRF models for sequence tagging[J]. arXiv:1508.01991,2015.
[17] Zheng S C, Wang F, Bao H Y, et al. Joint extraction of entities and relations based on a novel tagging scheme[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Vancouver, Canada. Stroudsburg, PA, USA: Association for Computational Linguistics, 2017: 1227-1236. DOI:10.18653/v1/p17-1113.
[18] Fu T J, Li P H, Ma W Y. GraphRel: modeling text as relational graphs for joint entity and relation extraction[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Florence, Italy. Stroudsburg, PA, USA: Association for Computational Linguistics, 2019: 1409-1418. DOI:10.18653/v1/p19-1136.
[19] Mikolov T, Chen K, Corrado G, et al. Efficient estimation of word representations in vector space[J]. arXiv:1301.3781,2013.
[20] Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[C]//Advances in neural information processing systems. 2017: 5998-6008.
[21] Riedel S, Yao L M, McCallum A. Modeling relations and their mentions without labeled text[C]//Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, Berlin, Heidelberg, 2010: 148-163. DOI:10.1007/978-3-642-15939-8_10.
[22] Gardent C, Shimorina A, Narayan S, et al. Creating training corpora for NLG micro-planners[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).Vancouver, Canada. Stroudsburg, PA, USA: Association for Computational Linguistics, 2017: 179-188. DOI:10.18653/v1/p17-1017.
[23] Hochreiter S, Schmidhuber J. Long short-term memory[J]. Neural Computation, 1997, 9(8):1735-1780. DOI:10.1162/neco.1997.9.8.1735.
[24] Kingma D P, Ba J. Adam: a method for stochastic optimization[J]. arXiv:1412.6980,2014.

一种新的基于深度学习的重叠关系联合抽取模型（英文）

A new joint model for extracting overlapping relations based on deep learning

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 6

编辑推荐

Metrics

本文评价

访问统计

联系我们

[1]	周龙, 王伟强, 吕科. FACR：一种快速且准确的车辆识别器[J]. 中国科学院大学学报, 2021, 38(1): 130-136.
[2]	李森, 彭玲, 胡媛, 池天河. 基于FD-RCF的高分辨率遥感影像耕地边缘检测[J]. 中国科学院大学学报, 2020, 37(4): 483-489.
[3]	卢凯旋, 李国清, 陈正超, 昝露洋, 李柏鹏, 高建威. 基于负样本多通道优化SSD网络的钢铁厂提取[J]. 中国科学院大学学报, 2020, 37(3): 352-359.
[4]	闫凯, 沈汀, 陈正超, 闫弘轩. 基于深度学习的SSD模型尾矿库自动提取[J]. 中国科学院大学学报, 2020, 37(3): 360-367.
[5]	宋旭鸣, 沈逸飞, 石远明. 基于深度学习的智能移动边缘网络缓存[J]. 中国科学院大学学报, 2020, 37(1): 128-135.
[6]	刘洋, 罗晨希, 罗铁坚. 和积网络的性质分析及其有效性验证算法[J]. 中国科学院大学学报, 2020, 37(1): 136-143.