Fried views exploratory research as akin to deploying researchers into an uncharted wilderness. Photo: COURTESY OF EIKO FRIED

Current trends in the social sciences show a strong preference for confirmatory research, while exploratory studies are increasingly pushed to the margins. Balazs Aczel, a professor at the Metascience Laboratory of Eotovs Loránd University in Hungary, noted this pattern at a recent psychology conference, where the vast majority of submissions focused on verification, with few contributions engaging in genuine exploration.

In scientific inquiry, exploratory and confirmatory approaches represent two essential stages of the research process. Rather than competing, they complement each other—qualitative insight and quantitative validation work together to expand the frontiers of knowledge. To explore the definitions, distinctions, and evolving roles of these two approaches—as well as the reasons behind the decline of exploratory research and potential remedies—CSST spoke with Eiko Fried, an associate professor of psychology at Leiden University in the Netherlands. Fried argues that the growing marginalization of exploratory research not only undermines the comprehensiveness and innovation of scientific inquiry but may also pose a latent threat to the robustness of scientific progress.

Exploration and verification complement each other

CSST: Why is exploratory research important, particularly in the social sciences?

Fried: Science entails, somewhat simplified, two steps: First, establishing phenomena—defined as robust, recurring features in the world. And second, explaining these phenomena via theories. Some phenomena can be readily observed without collecting data or performing statistical analyses, such as that humans have the capacity for language. Other phenomena are discovered in data, such as the relationship between smoking and lung cancer, or that depression rates are higher among women.

Exploratory research and its counterpart, confirmatory research, map onto these two steps to some degree. Exploratory research is more involved towards the side of establishing phenomena: it entails looking around in the world carefully and thoughtfully to see whether anything emerges that looks interesting. Confirmatory research is more involved in theory building and explanation, and is commonly defined as taking some sort of theory, generating a hypothesis from it, and seeing whether this hypothesis can be confirmed in data, for example, via experiments. The 2012 experiment at the Large Hadron Collider (LHC) at CERN was confirmatory: the existence of the Higgs Boson was predicted by a theory (the Standard Model of particle physics) and confirmed in an experiment, ultimately leading to the 2013 Nobel Prize.

CSST: What differentiates exploratory research from confirmatory research?

Fried: Exploratory research can generate hypotheses and make interesting discoveries worth pursuing further. Think of it as dropping many researchers into some unexplored wilderness. They walk around, explore the jungle, climb over mountains, and swim through lakes. Some of them come back and say: “there is something interesting in this direction.”

Confirmatory research is better equipped to interrogate these original findings in more detail, and test hypotheses. To follow our analogy, the research community arrives at an initial theory, puts money together, carefully plans their trek over the mountain where supposedly interesting findings are, and carries out a series of adventures to confirm that there is something there, how it looks exactly, and how it can best be explained. If things work out, this previously unexplored part of the scientific landscape behind the mountains then gets added to our scientific map of what is known.

Neglecting exploration leads to questionable practices

CSST: The social sciences lack open-ended exploration for generating hypotheses, and there is a persistent bias that exploratory research is unscientific. What do you think is the root of this bias? Could it be linked to the over-reliance on quantitative empiricism in current social science paradigms? How would you analyze this phenomenon?

Fried: There is some careful and faithful exploration of new spaces in the social sciences, but it is not so common, and I agree that there is a misunderstanding that good science is confirmatory science. In 2021, Daniel Nettle, a professor of social work, education and community wellbeing, from Northumbria University, U.K., made this point very well in the blog post “Theories and models are not the only fruit”, where he argued for the immense importance of exploratory research: “You shouldn’t have to claim to have a fully identified theory (especially if you haven’t actually got one, which I think is true of much psychological research, including plenty that is well worth doing). [...] Biology had hundreds of years of taxonomy and natural history before it had formal phylogenetic models; geology had detailed maps of rocks before it accepted plate tectonics; people had been studying the motion of planets long before Newtonian mechanics, and so on.”

We don’t have a lot of theories I would consider “strong” in psychology. I have written about this lack of clear theories in “Lack of Theory Building and Testing Impedes Progress in The Factor and Network Literature” and “Theories and Models: What They Are, What They Are for, and What They Are About.” Others have written about it, too, and many of us agree that the over-reliance of social sciences on testing and confirmatory work comes from the fact that we do not have a lot of robust theories. Denny Borsboom, a professor of social and behavioral sciences from the University of Amsterdam called this phenomenon “theoretical amnesia” in a widely read 2013 blog post: “It is a sad but, in my view, inescapable conclusion: we don’t have much in the way of scientific theory in psychology. [...] And that’s why psychology is so hyper-ultra-mega empirical.”

CSST: What is shifting researchers away from exploratory research?

Fried: We’ve had a replication crisis in the social sciences (and elsewhere): well-known findings we thought were robust phenomena ended up not being replicable, meaning they were not robust in the first place.

My brief explanation of this crisis is that researchers who were dropped in the wilderness (which is really what most of psychology is at the moment—a lot of things are unknown) came back from over the mountain top. But instead of saying “there is something interesting there,” they came back saying “I had hypothethized based on my theory that there is something there, and I found it! Heureka.” And then they usually gave a TED talk.

In other words, the problem is that researchers were actually doing exploratory work—they collected datasets with many variables and related them all to each other—but they wrote up their papers as confirmatory. This was inappropriate, because they didn’t really have clear hypothesis (let alone a strong theory from which to derive hypotheses) from the start, which introduces false positive findings into the literature: phenomena that are not robust.

CSST: Academic journals’ preference for statistically significant results often drives researchers toward confirmatory studies. Some advocate for journals to embrace exploratory research. Such studies typically lack definitive conclusions. What innovations do you believe are needed in the peer-review standards for exploratory papers? How can we enhance credibility?

Fried: I would slightly rephrase the first sentence: journals’ preference for statistically significant results often drives researchers to write up their papers as confirmatory studies (although they may have been conducted as exploratory in many cases). I am not sure we desperately need dedicated formats for exploratory work, although I would welcome such formats. But I would perhaps rather try to change the perception in the social sciences about exploratory research, which is a crucial step in establishing robust phenomena. This begins by teaching students about the value of doing exploratory work.

Balancing innovation with rigor

CSST: Training in exploratory research is rare; most curricula focus on hypothesis testing. If tasked with redesigning a social science research methods course, how would you balance teaching exploratory and confirmatory approaches?

Fried: Social scientists indeed receive excellent training in testing things. But you know what we get basically no training in at all? The generation of new ideas. Marten Scheffer is an ecologist and mathematical biologist and a professor of aquatic ecology and water quality management at Wageningen University & Research, Netherlands. He refers to this as “the forgotten half of scientific thinking” and advocated in his paper on the topic that generating novel ideas is of crucial importance. Scheffer talks about the importance of stimulating associative thinking, and that scientists need to relearn habits that help with that. He states that, “Indeed, solid scientific skills are needed to weed out right from wrong. However, our current teaching and routines are focused almost exclusively on those skills, whereas the best science tends to come from a balanced mix of rationality and adventurous association.”

So, we should consider teaching students to do this better, and as part of the Young Academy Leiden (YAL), we actually organized a workshop at Leiden University—together with Professor Marten Scheffer—on this topic a few weeks ago, which was very well received.

CSST: Exploratory research must still adhere to scientific rigor, yet traditional standards (e.g., pre-registration, variable control) might constrain openness. How can we balance “avoiding data dredging” with “preserving exploratory freedom”?

Fried: A student recently came up to me after class bemoaning why I had assigned such a “bad” paper to read. They explained that the paper was bad because it was not preregistered. For context, the paper I had assigned was a beautiful exploratory paper that shared all collected data (not just the analyzed variables). The student had misunderstood preregistration as a thing to strive for because it is a good thing in itself, rather than striving for preregistration because it can, in specific circumstances, be a useful tool to further open science values such as transparency.

So, if someone collects a very large data-set, and finds one particular relationship among variables of interest, and writes their paper up in a way that expresses this, and shares all their data, I’m not so concerned. If someone does the same but writes their paper up as confirmatory, and doesn’t share the data (today’s standard in psychology), I am much more concerned.