GenAI is reshaping knowledge production in the humanities and social sciences. Photo: TUCHONG

Social science research methods have long evolved in close tandem with technological progress. The recent rise of generative artificial intelligence (GenAI) has once again prompted wide-ranging discussion across academia and beyond. Against this backdrop, a careful examination of how GenAI can be used appropriately in social science research has become especially important. In fact, social scientists have already begun to explore these possibilities, with particular attention to large language models (LLMs) based on transformer architectures. Trained on vast corpora of data, such models are able to understand and process highly complex contextual information. These developments inevitably pose new questions for the discipline: How should social science researchers adapt to the growing integration of LLMs into research practice? Using bias research as a focal case, this article reviews both the potential benefits and the attendant risks that GenAI brings to social science research methods, from two perspectives: GenAI as a research tool and GenAI as a research object.

GenAI as a research tool

The notion of “GenAI as a research tool” refers to its role as an auxiliary instrument that assists researchers in understanding the world and participating in specific stages of the research process. Philip J. Runkel and Joseph E. McGrath (1972) famously classified traditional research methods into four types according to the degree of obstruction imposed by research operations and the universality of behavioral systems: experimental methods and judgment tasks; sampling surveys and formal theories; experimental stimuli and field experiments; and field investigations and computer simulations. These approaches face a well-known triangular dilemma: It is impossible to simultaneously ensure cross-actor generalizability, measurement control and precision, and situational realism. In recent decades, computational social science has sought to address this dilemma by leveraging large-scale data, giving rise to a so-called “fourth paradigm” that is data-driven rather than purely theory-driven. The introduction of GenAI has the potential to further enhance the efficiency of existing research paradigms, thereby accelerating the development of this fourth paradigm.

In the context of bias research, prejudice refers to generalized negative attitudes toward out-groups, closely linked to stereotypical impressions and the behaviors that follow from them. Traditional data collection approaches for studying prejudice have relied primarily on surveys, psychometric scales, and experimental methods. More recently, computational social scientists have begun to draw on online data, experiments, virtual reality technologies, and agent-based modeling to examine phenomena such as political polarization, seeking to better capture contemporary forms of bias influenced by the internet. Building on both the stages of the traditional research paradigm—from theory formulation to hypothesis generation, data collection, and analysis—and the operational processes characteristic of the fourth paradigm, this article explores how GenAI may extend existing methods and expand the analytical toolkit available to bias researchers.

Can GenAI stimulate theoretical inspiration? The answer appears to be cautiously affirmative. LLMs, including the GPT series and Claude, can assist researchers in brainstorming around specific questions, while their capacity to probe and challenge particular arguments may also help expose weaknesses in established theories. As intelligent technologies continue to advance, familiar pathways to theoretical innovation in the social sciences—such as interdisciplinary conceptual borrowing, the identification of phenomena unexplained by existing frameworks, and the construction of new conceptual spaces—may increasingly intersect with the combinatorial, exploratory, and even transformative forms of creativity that computational systems are capable of approximating.

GenAI’s contribution to data collection can be observed in four respects. First, in classic experimental research paradigm, it may help mitigate longstanding concerns about external validity. Second, in ethically sensitive domains, it offers new ways to explore topics such as social bias while reducing potential risks. Third, when combined with agent-based simulation, GenAI can facilitate the investigation of emergent mechanisms in complex social phenomena. Fourth, in large-scale social surveys, it also holds promise for reducing human bias in questionnaire design. At present, GenAI’s advantages in data analysis are most evident in the coding of unstructured data, particularly in its capacity to handle large volumes of information and to achieve more nuanced semantic interpretation.

Drawing on Thomas Kuhn’s theory of scientific paradigms, Jim Gray proposed four paradigm shifts in human cognition: pre-Galilean experimental science, post-Newtonian theoretical science, computational science supported by simulation, and a fourth paradigm centered on data-intensive discovery. The evolution of social science research methods has broadly mirrored this trajectory, progressing from empirical observation and theoretical construction to computational social science and, more recently, to digitally mediated social science deeply integrated with AI. Notably, the proposal of “silicon sampling” social surveys by Anant Agarwal et al. represents a concrete methodological expression of this fourth paradigm.

GenAI as a research object

When treated as a research object, GenAI becomes the focus of inquiry into machine behavior and cognition, including studies of bias, discrimination, and the production of harmful content by LLMs. Even in this mode, the ultimate aim of the social sciences remains understanding human society itself. This article focuses on prompt-based probing methods, which use carefully designed prompts to reveal potential unfairness or bias in a model’s responses to different types of input. Such methods have already been widely adopted in research on the inherent biases of GenAI systems.

Drawing on Robert Sternberg’s theory of successful intelligence and Franc?ois Chollet’s work on AI, the authors propose a basic framework for detecting bias in GenAI through prompt-based analysis. The explanatory power of this theoretical framework rests on the assumption that theories of human intelligence can, to some extent, be meaningfully applied to artificial systems. Creative, analytical, and practical intelligence represent functional distinctions within human cognition, and these map structurally onto GenAI’s core capacities for text generation, logical reasoning, and contextual adaptation. Tasks associated with analytical intelligence, such as analogical reasoning, can be used to detect attribution biases embedded in logical rules; creative intelligence tasks involving text or image generation may expose cultural prototype biases latent in training data; and practical intelligence tasks that emphasize contextualized creation can assess how systems adapt to complex social situations, revealing implicit cultural hierarchies in value prioritization.

Analytical intelligence typically concerns an agent’s logical reasoning abilities, including judgment, discrimination, and problem-solving. Within bias detection frameworks, tasks in this dimension are effective precisely because they force causal structures and attribute associations to become explicit during the cognitive mapping process. Relevant generative tasks include analogical reasoning, machine translation, code generation, and image description.

In analogical reasoning tasks, for example, LLMs are presented with statements containing key concepts and blanks, and are asked to supply terms they deem analogous. Because bias is culturally variable, yet often treated as homogeneous in earlier studies, GenAI offers an opportunity to explore such differences with greater cultural sensitivity, particularly in areas such as machine translation. Image-processing tasks may further expose latent biases, since images must be converted into sequential representations and image–text training data are often less tightly controlled than textual corpora. Even code generation, which appears to embody pure logical rationality, can reproduce social biases embedded in training data.

Creative intelligence involves the capacity to draw inspiration and insights from experience and to generate novel outputs in specific contexts. In bias detection, creative intelligence tasks are effective because they activate mechanisms through which AI systems extract and recombine symbolic elements that function analogously to a form of “subconscious” association. This article focuses on text creation and image generation, noting that because generative models rely heavily on human-produced corpora and employ neural network architectures designed to emulate aspects of human cognition, they inevitably inherit human heuristic biases. Text generation can thus serve as a window into biased associative patterns, while image generation—long used in human-subject research to probe cognitive bias—has revealed widespread representational and performance biases related to gender within GenAI systems.

Practical intelligence refers to the application of knowledge in everyday, context-dependent situations. In prompt-based probing, tasks designed to test practical intelligence place AI systems under situational pressure, compelling them to reveal cultural ranking strategies or conceptual association schemas. Techniques such as role-playing or polarized prompts allow researchers to examine how language use and content vary when LLMs adopt different social roles, shedding light on links between linguistic variation and socially embedded expectations. Polarization tasks, in particular, are well suited to surfacing stereotypes that are otherwise submerged within common sense, gradually rendering implicit biases more visible through sustained human–machine interaction.

As technological capabilities advance and scientific logic continues to permeate research practice, knowledge production has undergone a shift from holistic scholarly traditions toward increasing disciplinary specialization. The emergence of GenAI, however, appears to gesture toward a renewed integration of perspectives in social research. How AI technologies will ultimately reshape knowledge production in the humanities and social sciences—and how issues of talent cultivation and research ethics should be addressed in response—remain open questions for future inquiry.

Lu Yunfeng (professor), Huang Xingyao, and Wu Yufei are from the Department of Sociology at Peking University. Zhou Lyujun is an associate professor from the School of Social Work at China Women’s University. The article has been edited and excerpted from Journal of Guizhou Minzu University (Philosophy and Social Sciences), Issue 3, 2025.