Journals of humanities and social sciences in China Photo: Yang Lanlan/CSST

The rapid advancements in fields such as big data, open science, and artificial intelligence have created new opportunities for the theoretical development and practice of academic evaluation while introducing new demands. Domestic scholars engaged in academic evaluation research and database construction are actively embracing these changes and exploring the establishment of a more scientifically grounded, independent academic evaluation system.

Status quo

Academic evaluation serves as both a barometer and compass of academic research, and also the “purifier” and “filter” of academic development. Since the 1980s, the primarily quantitative approach to academic evaluation and the focus on journal and institutional assessments have played a role in advancing the scientific, standardized, and international development of philosophy and social sciences in China. However, this evaluation system has also fueled problems such as an undue emphasis on quantity over quality, forms over substance, and international perspectives over local contexts, and has thus been widely criticized by the academic community.

Jing Linbo, director of the Chinese Academy of Social Sciences Evaluation Studies (CASSES) at the Chinese Academy of Social Sciences, reviewed the achievements of his research team over the past decade in areas such as journal and think tank evaluation, public policy assessment, evaluative database construction, and the cultivation of research integrity. Notably, under his leadership, the team has independently developed the AMI journal evaluation index system with Chinese characteristics and created a four-tier peer review expert team, addressing the shortcomings of quantitative evaluation and gaining widespread recognition in the philosophy and social science community. Additionally, the AMI evaluation framework has been extended to think tank and disciplinary evaluation, advancing the construction of China’s independent academic evaluation system. Currently, the team is working to internationalize its evaluation standards, aspiring to make China’s voice heard in international academic evaluation exchanges. 

The academic community represents not only the cornerstone of the academic exchange ecosystem, but also a guarantor of the credibility of the academic evaluation system. According to Jing, the peer expert team established by the CASSES comprises over 30,000 members, including experts of various disciplines, journal editors, young scholars, and doctoral students, playing a significant role in journal evaluation.

Evaluation of journals

As a platform for academic dissemination and exchange, academic journals have always attracted significant attention from all walks of life. However, some scholars believe that the current field of journal evaluation faces several challenges, such as limited evaluation dimensions, convergence of evaluation methods, a narrow range of evaluators, insufficient classification and hierarchical evaluation, slow evaluation times, low data richness, as well as inadequate influence and recognition of the evaluation system. These challenges need to be addressed urgently. At present, the Chinese philosophy and social sciences journal evaluation system, which is composed of the AMI Comprehensive Evaluation Report on Chinese Humanities and Social Sciences Journals, A Guide to the Core Journals of China, “Source Journals for Chinese Social Sciences Citation Index (CSSCI),” and A Report on Chinese Academic Journals Evaluation, is still undergoing refinement.

Regarding the practice of using journal-based evaluation standards for assessing academic outcomes, Shen Guchao, a professor from the School of Information Management at Nanjing University, warned that journal evaluations should not replace academic evaluations. For the development of journals themselves, he emphasized that “ranking journals is not the goal, the evaluation of journals’ influence cannot replace the evaluation of innovation, and formal evaluation cannot be a substitute for content evaluation.” Shen called on the academic community to downplay its emphasis on core journals and return to the goal of serving academic prosperity. 

Intelligent development

Ma Zheng, a research fellow from the National Science Library at the Chinese Academy of Sciences, analyzed the application of AI in academic journals and academic evaluation, as well as its impact on the journal industry. For editors of academic journals, AI can assist with manuscript screening and preliminary evaluation, optimize the review process, aid in paper editing and typesetting, detect academic misconduct, provide personalized recommendations, enhance precision of impact, and support data analysis and decision-making. Regarding the evaluation of academic journals, AI can achieve in-depth literature analysis in terms of innovativeness, frontier relevance, influence, academic norms, political contexts, and other indicators.

When assessing the value of a paper, Wu Junhong, director of the evaluation center of China Academic Journals (CD-ROM) Electronic Magazine House, explained that AI can identify the internal connections and logical chains between cited segments based on the citation content of academic papers, presenting a comprehensive and structured overview of the paper’s value.  

In April this year, China’s first preprint platform for philosophy and social sciences, Philosophy and Social Sciences Preprint, was established and launched by the Informational Center for Social Sciences at Renmin University of China. According to Yang Hongyan, deputy director of the center, the platform facilitates collaboration among authors, journals, and review experts, embodying an evaluation philosophy geared toward the new ecosystem of academic publishing and communication. Looking ahead, there is an urgent need to establish a more comprehensive evaluation system for preprints in philosophy and social sciences.

In the view of Ding Jingda, executive director of the Chinese Academy of Science and Education Evaluation at Hangzhou Dianzi University, the use of multivariate data fusion technology to discover new research methods from emerging topics helps locate valuable research frontiers and lead the innovative development of scholarship.

As a representative of database development agencies, Liu Xin, a researcher from Clarivate, introduced the development and update of the JCR database and its auxiliary role for researchers.