广东工业大学学报 ›› 2024, Vol. 41 ›› Issue (01): 69-78.doi: 10.12052/gdutxb.220132

• 计算机科学与技术 • 上一篇    下一篇

融合标签语义嵌入和图卷积的短文本特征扩展及分类方法

张灵1, 李荣臻1, 郑苏2   

  1. 1. 广东工业大学 计算机学院, 广东 广州 510006;
    2. 宁夏大学 教育学院, 宁夏 银川 750001
  • 收稿日期:2022-08-29 出版日期:2024-01-25 发布日期:2023-11-20
  • 作者简介:张灵(1968−)，女，教授，博士，主要研究方向为数据挖掘、计算机视觉、无线传感，E-mail：june4567@21cn.com
  • 基金资助:
    广东省交通运输厅科技项目(科技-2016-02-030)

Short Text Feature Extension and Classification Method Based on Semantic Embedding of Tags and Graph Convolution Network

Zhang Ling1, Li Rong-zhen1, Zheng Su2   

  1. 1. School of Computer Science and Technology, Guangdong University of Technology, Guangzhou 510006, China;
    2. College of Education, Ningxia University, Yinchuan 750001, China
  • Received:2022-08-29 Online:2024-01-25 Published:2023-11-20

HTML

PDF (PC)

290

摘要： 针对短文本长度过短、关键词偏少和标签信息利用不足造成的分类过程中面临特征稀疏和语义不明确的问题，提出了融合标签语义嵌入的图卷积网络模型。首先，在传统的术语频率和逆文档频率算法基础上，提出了融合单词所属文本的类间、类内分布关系的全局词频提取算法。其次，利用融合标签嵌入的方法，将每条训练文本与相对应的标签引入到同一个特征空间内，通过筛选聚合提取更能突显文本类别的近义词嵌入，作为文本图的文档节点的嵌入表示。最后，将文本图输入到图卷积神经网络学习后，获得的特征与预训练模型提取文本上下文的特征相融合，提升短文本的分类质量以及整个模型的泛化能力，在4个短文本数据集MR、web_snippets、R8和R52上对本文模型和14个基线算法进行了对比实验，结果表明本文提出的模型相比于对比模型具有更优的结果，在识别精度、召回率以及F1值上有着更好的表现。

关键词: 短文本, 标签语义, 特征空间, 图卷积网络, 预训练模型

Abstract: In short text classification, too short text length, fewer keywords and underutilization of the label information leads to the severe problems of sparse features and ambiguous semantics, which can affect the performance of short text classification. Agraph convolution network model based on tag semantic embedding is proposed for the problem. Firstly, according to TF/IDF, a new word frequency method is proposed, which comprehensively considers the inter-class and intra-class distribution of words in the global corpus. Then, through By label embedding method, each training text with the corresponding label is mapped into one feature space in the text graph. After filtering and aggregation in one feature space, synonyms embedded of label information can highlight the category representation. Finally, the text graph is input into the graph convolution neural network to learn new feature. Both the learned new feature Both the learned new feature and the features from the pre-training model can improve the classification accuracy of short texts and the generalization ability of the whole model. We choose four short text datasets such as MR, web_snippets, R8 and R52, to evaluate the performance of our proposed algorithm and fourteen benchmark models. The experimental results show that the proposed model in this paper is superior to others in classification accuracy, recall ratio and F1-score.

Key words: short text, semantics of label, feature space, graph convolution network, pre-training model

中图分类号: 

  • TP391
[1] ZAREMBA W, SUTSKEVER I, VINYALS O. Recurrent neural network regularization[EB/OL]. (2014-09-08) [2022-08-20]. https://arxiv.org/abs/1409.2329v5.
[2] TURKOGLU M, HANBAY D, SENGUR A. Multi-model L- STM-based convolutional neural networks for detection of apple diseases and pests [J]. Journal of Ambient Intelligence and Humanized Computing, 2022, 13(1): 3335-3345.
[3] AGARAP A F. A neural network architecture combining gated recurrent unit (GRU) and support vector machine(SVM) for intrusion detection in network traffic data[EB/OL]. (2017-09-10) [2022-08-20]. https://arxiv.org/abs/1709.0302.
[4] CANIZO M, TRIGUERO I, CONDE A, et al. Multi-head CNN-RNN for multi-time series anomaly detection: an ind- ustrial case study [J]. Neurocomputing, 2019, 363: 246-260.
[5] KIM Y. Convolutional neural network for sentence classification[EB/OL]. arXiv: 1408.5882 (2014-09-03) [2022-08-20]. https://arxiv.org/abs/1408.5882.
[6] NIEPERT M, AHMED M, KUTZKOV K. Learning convolutional neural networks for graphs[EB/OL]. arXiv: 1605.05273(2016-06-08) [2022-08-20]. https://arxiv.org/abs/1605.05273.
[7] XU K, HU W, LESKOVEC J, et al. How powerful are graph neural networks?[C]//International Conference on Learning Representation. New Orleans: ICLR, 2019: 1-17.
[8] KIPF, WELLING M, THOMAS N. Semi-supervisedclassification with graph convolutional networks[C]//International Conference on Learning Representations. Toulon, France: ICLR, 2017: 1-14.
[9] YAO L, MAO C S, LUO Y. Graph convolutional networks for text classification[C]//33rd AAAI Conference on Artificial Intelligence. Honolulu: AAAI, 2019: 7370-7377.
[10] DRASKO R, BOZO K. Review spam detection using mac- hine learning[C]//23th International Scientific-Professional Conference on Information Technology. New Delhi, India: IT, 2018: 1-4.
[11] BAKSHI R K, KAUR N, KAUR R, et al. Opinion mining and sentiment analysis[C]//Computing for Sustainable Glo-bal Development. New Delhi, India: INDIACom, 2016: 452-455.
[12] BOUAZIZ A, DARTIGUES-PALLEZ C, PEREIRA C D C, et al. Short text classification using semantic random forest [J]. Springer International Publishing, 2014, 8646: 288-299.
[13] 方澄, 李贝, 韩萍. 基于全局特征图的半监督微博文本情感分类[J]. 信号处理, 2021, 37(6): 1066-1074.
FANG C, LI B, HAN P. Semi-supervised microblog text sentiment classification based on global feature graph [J]. Journal of Signal Processing, 2021, 37(6): 1066-1074.
[14] 崔婉秋, 杜军平, 寇菲菲, 等. 面向微博短文本的社交与概念化语义扩展搜索方法[J]. 计算机研究与发展, 2018, 55(8): 1641-1652.
CUI W Q, DU J P, KOU F F, et al. The social and conceptual semantic extended search method for microblog short text [J]. Journal of Computer Reasearch and Development, 2018, 55(8): 1641-1652.
[15] WANG G, LI C, WANG W, et al. Joint embedding of words and labels for text classification[C]//Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. New Orleans: NAACL-HLT, 2018: 461-469.
[16] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Conference and Workshop on Neural Information Processing Systems. Long Beach, California, U- SA: ACM, 2017: 6000-6010.
[17] 张万杰. 引入标签语义信息的多标签文本分类[J]. 计算机应用, 2021, 8: 1672-9528.
[18] JI F, YANG J L, ZHANG Q, et al. GraphFlow: a new graph convolutional network based on parallel flows [C]//Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing. Barcelona: ICASSP, 2020: 3332-3336.
[19] 郑诚, 董春阳, 黄夏炎. 基于BTM图卷积网络的短文本分类方法[J]. 计算机工程与应用, 2021, 57(4): 155-160.
ZHENG C, DONG C Y, HUANG X Y. Short text classification method based on BTM graph convolution network [J]. Computer Engineering and Application, 2021, 57(4): 155-160.
[20] 辛媛. 基于图神经网络的单标签文本分类[D]. 合肥: 中国科技技术大学, 2021, 1-61.
[21] 申艳光, 贾耀清. 基于词共现与图卷积的文本分类方法[J]. 计算机工程与应用, 2021, 57(11): 173-178.
SHEN Y G, JIA Y Q. Text categorization method based on word co-occurrence and graph convolution [J]. Computer Engineering and Application, 2021, 57(11): 173-178.
[22] 郑诚, 陈杰, 董春阳. 结合图卷积的深层神经网络用于文本分类[J]. 计算机工程与应用, 2022, 58(7): 206-212.
ZHENG C, CHEN J, DONG C Y. Deep neural network combined with graph convolution for text classification [J]. Computer Engineering and Application, 2022, 58(7): 206-212.
[23] LIU X, YOU X, ZHANG X, et al. Tensor graph convolutional networks for text classification[C]//Proceedings of the AAAI Conference on Artificial Intelligence. New York, US-A: AAAI, 2020: 8409-8416.
[24] WU F, ZHANG T, SOUZA A, et al. Simplifying graph convolutional networks[C]//International Conference on Machine Learning. Long Beach, CA, USA: ICML, 2019: 1-14.
[25] ZHANG Y F, YU X L, CUI ZY, et al. Every document owns its structure: inductive text classification via graph neural networks[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Seattle, Washington, United States: ACL, 2020: 334-339.
[26] LI Y J, TARLOW D, BROCKSCHMIDT M, et al. Gated graph sequence neural networks[EB/OL]. arXiv: 1511.05493(2017-09-22) [2022-08-20]. https://arxiv.org/abs/1511.05493.
[27] ROMERO A, CUCURULL G, CASANOVA A, et al. Graph attention networks[EB/OL]. arXiv:1710.10903(2018-02-04) [2022-08-20]. https://arxiv.org/abs/1710.10903.
[28] DING K Z, WANG J L, LI J D, et al. Be more with less: hypergraph attention networks for inductive text classification [C]//The 2020 Conference on Empirical Methods in Natural Language Processing. Online: EMNLP, 2020: 4927-4936.
[29] LIN Y X, MENG Y X, SUN X F, et al. BertGCN: transductive text classification by combining GCN an BERT[C]//Annual Meeting of the Association for Computational Linguistics. Bangkok, Thailand: ACL-IJCNLP, 2021: 1456-1462.
[1] 林哲煌, 李东. 语义引导下自适应拓扑推理图卷积网络的人体动作识别[J]. 广东工业大学学报, 2023, 40(04): 45-52.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
版权所有 © 2017 《广东工业大学学报》编辑部
地址：广州市东风东路729号广东工业大学 邮编：510090
电话：020-37626653,020-37626139,020-37626396
邮箱：xbzrb@gdut.edu.cn
本系统由北京玛格泰克科技发展有限公司设计开发