文章摘要
胡容波* ** ***,张广发* ***,王雅雯* ***,方金云*.基于BERT提示的矿产资源管理规则检测方法研究[J].高技术通讯(中文),2023,33(11):1136~1145
基于BERT提示的矿产资源管理规则检测方法研究
Research on detection method of mineral resources regulatory rules based on BERT with prompts
  
DOI:10. 3772/ j. issn. 1002-0470. 2023. 11. 002
中文关键词: 矿产资源; 管理规则; 文本分类; 基于转换器的双向编码表征(BERT); 提示学习
英文关键词: mineral resources, regulatory rule, text classification, bidirectional encoder representation from transformers (BERT), prompt-based learning
基金项目:
作者单位
胡容波* ** *** (*中国科学院计算技术研究所北京 100190) (**自然资源部信息中心北京 100036) (***中国科学院大学北京 100190) 
张广发* ***  
王雅雯* ***  
方金云*  
摘要点击次数: 633
全文下载次数: 520
中文摘要:
      政策文本中管理规则检测是一个新兴的自然语言处理任务,在政策冲突检测、政策智能检索、事项合规性检查以及政务系统需求工程等方面具有重要应用价值。本文以矿产资源管理规则检测为研究目标,提出基于转换器的双向编码表征(BERT)提示的政策文本管理规则检测方法。该方法通过构建融入管理规则信息、带有[MASK]标记的提示模板,可以充分发挥掩码语言模型的自编码优势,有效激发BERT模型提取与管理规则相关的文本特征,增加模型稳定性;提出基于BERT模型进行管理规则检测的新应用模式,放弃使用[CLS]隐向量而采用[MASK]隐向量进行分类预测;在矿产资源管理规则数据集上的实验结果表明,该方法的准确率、宏平均F1值、加权平均F1值均优于基线方法,在公开数据集上的实验结果也表明了该方法的有效性。
英文摘要:
      Regulatory rules detection in policy text is an emerging natural language processing task, which has important application value for policy conflict detection, policy intelligent retrieval, regulatory compliance inspection, and e-government system requirements engineering. This paper takes the detection of mineral resources regulatory rules as the research goal, and proposes a detection method based on bidirectional encoder representation from transformers (BERT) with prompts. By constructing a prompt template with [MASK], which incorporates regulatory rules information, the proposed method can give full play to the auto-encoding advantages of the mask language model, effectively stimulate BERT model to extract text features related to regulatory rules and increase the stability of the model. A new application mode of regulatory rules detection based on BERT model is proposed, which uses the [MASK] hidden vector instead of the [CLS] hidden vector for classification and prediction. The experimental results on the dataset of mineral resources regulatory rules show that the accuracy, macro-average F1 score and weighted-average F1 score of this method are better than the baseline methods. The experimental results on the public dataset also show the effectiveness of the proposed method.
查看全文   查看/发表评论  下载PDF阅读器
关闭

分享按钮