评估 | 自动摘要评估
程序员文章站
2022-07-01 18:27:03
...
1.rouge评估,英文中文都可以。
中文,都可以使用rouge评估,代码:
# -*- coding: utf-8 -*-
from rouge import Rouge
hypothesis = "我 爱 最美 的 中国"
reference = "我 爱 中国"
rouge = Rouge()
scores = rouge.get_scores(hypothesis, reference)
print("rouge-1:",scores[0]['rouge-1'])
print("rouge-2:",scores[0]['rouge-2'])
print("rouge-l:",scores[0]['rouge-l'])
效果:
rouge-1: {'f': 0.7499999953125, 'p': 0.6, 'r': 1.0}
rouge-2: {'f': 0.33333332888888895, 'p': 0.25, 'r': 0.5}
rouge-l: {'f': 0.7499999953125, 'p': 0.6, 'r': 1.0}
英文:
# -*- coding: utf-8 -*-
from rouge import Rouge
hypothesis = "there is an apple on the desktop"
reference = "there is an apple"
rouge = Rouge()
scores = rouge.get_scores(hypothesis, reference)
print("rouge-1:",scores[0]['rouge-1'])
print("rouge-2:",scores[0]['rouge-2'])
print("rouge-l:",scores[0]['rouge-l'])
效果:
rouge-1: {'f': 0.7272727226446282, 'p': 0.5714285714285714, 'r': 1.0}
rouge-2: {'f': 0.6666666622222223, 'p': 0.5, 'r': 1.0}
rouge-l: {'f': 0.7272727226446282, 'p': 0.5714285714285714, 'r': 1.0}
2.pyrouge评估,只有英文可以
# -*- coding: utf-8 -*-
from pyrouge import Rouge155
r = Rouge155()
r.system_dir = 'system_summaries'
r.model_dir = 'model_summaries'
r.system_filename_pattern = '(\d+).txt' #参考摘要
r.model_filename_pattern = '#ID#_reference.txt'#生成摘要
output = r.convert_and_evaluate()
print(output)
注意命名格式
效果:
区别:后者可以显示更多评估,前者是只有rouge-1、rouge-2、rouge-l三种。