Nowadays, academic evaluation is increasingly dependent on quantitative indicators, which has incited controversy in academia.

Evaluation of scientific research is now increasingly dependent on data. An evaluation system based on quantitative indicators has become mainstream, but people rely too heavily on data rather than their own judgment, noted Jean-François Hérold, a lecturer with the Learning, Teaching, Assessment and Training Laboratory at the University of Aix-Marseille in France. Although these quantitative index systems are carefully designed and their goals are to promote scientific research, too often those who carry out the evaluations do not understand them well, which causes a misuse of indicators.

Jing Linbo, director of the Chinese Evaluation Center for Humanities and Social Sciences at the Chinese Academy of Social Sciences, said that in various evaluation systems, there is a general orientation that prioritizes quantitative evaluation over qualitative evaluation. However, some evaluation systems only adopt qualitative evaluation. Other systems have problems combining the two, and could even negate quantitative evaluation with qualitative evaluation.

We should correctly use quantitative indicators, rather than completely replace them with qualitative reviews, Hérold said. Quantitative indicators can reduce peer review bias and improve quality. It should be the basis for evaluators to make judgments, but it cannot completely replace the evaluators’ judgment. Consideration should be given to the selection and application of indicators on the basis of a broader social, economic, and cultural environment.

Jiang Bo, director of the editorial department of the Journal of Soochow University, identified two major problems with China’s academic evaluation: excessively seeking the certainty of results, and excessively encouraging a competitive culture.

An excessive demand for certainty in results will continue to narrow the scope of academic evaluation and encourage the focus on quantitative methods. Although the intention is to show the accuracy and scientific merit of evaluation, it actually diminishes the connotation of “academic,” making it one-sided and narrow, and thus cannot truly and completely reflect the academic level of research.

The problem with excessively encouraging competition culture is mainly seen in the incentive mechanism for evaluation. The focus is limited to comparisons between individuals. Although competition can provide motivation, if it evolves into internal frictions without paying attention to the overall academic atmosphere, it will run counter to the academic spirit of sharing knowledge and mutual assistance.

Jiang said that it is necessary to overcome this one-sided tendency in evaluation, and shift from focusing only on data including impact factors to comprehensively considering the academic results.

Jiang also suggested shifting priorities from the performance of individuals to designing an overall plan that can promote both individual and team development, so as to fully release the positive effects of the evaluation system.

When evaluating, people should pay attention to local academic results published in their native language, treat them equally as results published in English in international journals, and even encourage scholars to publish academic results locally in their native language, Hérold said.

The data collection and analysis process used in assessment should be open, transparent, and simple, and the assessee should be allowed to check the data and analysis results, Hérold said. It is best to use multiple indicators to evaluate scholars and their achievements from different perspectives, and to review and update evaluation indicators regularly so that they meet the new needs of our social and academic environment.