中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
Mapping the evaluation results between quantitative metrics and meta-synthesis from experts' judgements: evidence from the Supply Chain Management and Logistics journals ranking

文献类型:期刊论文

作者Yuan, Lili; Li, Jianping; Li, Ruoyun; Lu, Xiaoli; Wu, Dengsheng
刊名SOFT COMPUTING
出版日期2020
卷号24期号:0页码:6227-6243
DOI10.1007/s00500-019-03837-3
英文摘要

Meta-syntheses from experts' judgements and quantitative metrics are two main forms of evaluation. But they both have limitations. This paper constructs a framework for mapping the evaluation results between quantitative metrics and experts' judgements such that they may be solved. In this way, the weights of metrics in quantitative evaluation are objectively obtained, and the validity of the results can be testified. Weighted average percentile (WAP) is employed to aggregate different experts' judgements into standard WAP scores. The Technique for Order Preference by Similarity to an Ideal Solution (TOPSIS) method is used to map quantitative results into experts' judgements, while WAP scores are equal to the final closeness coefficients generated by the TOPSIS method. However, the closeness coefficients of TOPSIS rely on the weights of quantitative metrics. In this way, the mapping procedure is transformed into an optimization problem, and a genetic algorithm is introduced to search for the best weights. An academic journal ranking in the field of Supply Chain Management and Logistics (SCML) is used to test the validity obtained by mapping results. Four prominent ranking lists from Association of Business Schools, Australian Business Deans Council, German Academic Association for Business Research, and Comite National de la Recherche Scientifique were selected to represent different experts' judgements. Twelve indices including IF, Eigenfactor Score (ES), H-index, Scimago Journal Ranking, and Source Normalized Impact per Paper (SNIP) were chosen for quantitative evaluation. The results reveal that the mapping results possess high validity for the relative error of experts' judgements, the quantitative metrics are 43.4%, and the corresponding best weights are determined in the meantime. Thus, some interesting findings are concluded. First, H-index, Impact Per Publication (IPP), and SNIP play dominant roles in the SCML journal's quality evaluation. Second, all the metrics are positively correlated, although the correlation varies among metrics. For example, ES and NE are perfectly, positively correlated with each other, yet they have the lowest correlation with the other metrics. Metrics such as IF, IFWJ, 5-year IF, and IPP are highly correlated. Third, some highly correlated metrics may perform differently in quality evaluation, such as IPP and 5-year IF. Therefore, when mapping the quantitative metrics and experts' judgements, academic fields should be treated distinctively.

语种英语
源URL[http://ir.casisd.cn/handle/190111/9840]  
专题中国科学院科技战略咨询研究院
通讯作者Wu, Dengsheng
推荐引用方式
GB/T 7714
Yuan, Lili,Li, Jianping,Li, Ruoyun,et al. Mapping the evaluation results between quantitative metrics and meta-synthesis from experts' judgements: evidence from the Supply Chain Management and Logistics journals ranking[J]. SOFT COMPUTING,2020,24(0):6227-6243.
APA Yuan, Lili,Li, Jianping,Li, Ruoyun,Lu, Xiaoli,&Wu, Dengsheng.(2020).Mapping the evaluation results between quantitative metrics and meta-synthesis from experts' judgements: evidence from the Supply Chain Management and Logistics journals ranking.SOFT COMPUTING,24(0),6227-6243.
MLA Yuan, Lili,et al."Mapping the evaluation results between quantitative metrics and meta-synthesis from experts' judgements: evidence from the Supply Chain Management and Logistics journals ranking".SOFT COMPUTING 24.0(2020):6227-6243.

入库方式: OAI收割

来源:科技战略咨询研究院

浏览0
下载0
收藏0
其他版本

除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。