two types of experimental research

Experimental Research Design — 6 mistakes you should never make!

Since school days’ students perform scientific experiments that provide results that define and prove the laws and theorems in science. These experiments are laid on a strong foundation of experimental research designs.

An experimental research design helps researchers execute their research objectives with more clarity and transparency.

In this article, we will not only discuss the key aspects of experimental research designs but also the issues to avoid and problems to resolve while designing your research study.

Table of Contents

What Is Experimental Research Design?

Experimental research design is a framework of protocols and procedures created to conduct experimental research with a scientific approach using two sets of variables. Herein, the first set of variables acts as a constant, used to measure the differences of the second set. The best example of experimental research methods is quantitative research .

Experimental research helps a researcher gather the necessary data for making better research decisions and determining the facts of a research study.

When Can a Researcher Conduct Experimental Research?

A researcher can conduct experimental research in the following situations —

• When time is an important factor in establishing a relationship between the cause and effect.
• When there is an invariable or never-changing behavior between the cause and effect.
• Finally, when the researcher wishes to understand the importance of the cause and effect.

Importance of Experimental Research Design

To publish significant results, choosing a quality research design forms the foundation to build the research study. Moreover, effective research design helps establish quality decision-making procedures, structures the research to lead to easier data analysis, and addresses the main research question. Therefore, it is essential to cater undivided attention and time to create an experimental research design before beginning the practical experiment.

By creating a research design, a researcher is also giving oneself time to organize the research, set up relevant boundaries for the study, and increase the reliability of the results. Through all these efforts, one could also avoid inconclusive results. If any part of the research design is flawed, it will reflect on the quality of the results derived.

Types of Experimental Research Designs

Based on the methods used to collect data in experimental studies, the experimental research designs are of three primary types:

1. Pre-experimental Research Design

A research study could conduct pre-experimental research design when a group or many groups are under observation after implementing factors of cause and effect of the research. The pre-experimental design will help researchers understand whether further investigation is necessary for the groups under observation.

Pre-experimental research is of three types —

• One-shot Case Study Research Design
• One-group Pretest-posttest Research Design
• Static-group Comparison

2. True Experimental Research Design

A true experimental research design relies on statistical analysis to prove or disprove a researcher’s hypothesis. It is one of the most accurate forms of research because it provides specific scientific evidence. Furthermore, out of all the types of experimental designs, only a true experimental design can establish a cause-effect relationship within a group. However, in a true experiment, a researcher must satisfy these three factors —

• There is a control group that is not subjected to changes and an experimental group that will experience the changed variables
• A variable that can be manipulated by the researcher
• Random distribution of the variables

This type of experimental research is commonly observed in the physical sciences.

3. Quasi-experimental Research Design

The word “Quasi” means similarity. A quasi-experimental design is similar to a true experimental design. However, the difference between the two is the assignment of the control group. In this research design, an independent variable is manipulated, but the participants of a group are not randomly assigned. This type of research design is used in field settings where random assignment is either irrelevant or not required.

The classification of the research subjects, conditions, or groups determines the type of research design to be used.

Advantages of Experimental Research

Experimental research allows you to test your idea in a controlled environment before taking the research to clinical trials. Moreover, it provides the best method to test your theory because of the following advantages:

• Researchers have firm control over variables to obtain results.
• The subject does not impact the effectiveness of experimental research. Anyone can implement it for research purposes.
• The results are specific.
• Post results analysis, research findings from the same dataset can be repurposed for similar research ideas.
• Researchers can identify the cause and effect of the hypothesis and further analyze this relationship to determine in-depth ideas.
• Experimental research makes an ideal starting point. The collected data could be used as a foundation to build new research ideas for further studies.

6 Mistakes to Avoid While Designing Your Research

There is no order to this list, and any one of these issues can seriously compromise the quality of your research. You could refer to the list as a checklist of what to avoid while designing your research.

1. Invalid Theoretical Framework

Usually, researchers miss out on checking if their hypothesis is logical to be tested. If your research design does not have basic assumptions or postulates, then it is fundamentally flawed and you need to rework on your research framework.

2. Inadequate Literature Study

Without a comprehensive research literature review , it is difficult to identify and fill the knowledge and information gaps. Furthermore, you need to clearly state how your research will contribute to the research field, either by adding value to the pertinent literature or challenging previous findings and assumptions.

3. Insufficient or Incorrect Statistical Analysis

Statistical results are one of the most trusted scientific evidence. The ultimate goal of a research experiment is to gain valid and sustainable evidence. Therefore, incorrect statistical analysis could affect the quality of any quantitative research.

4. Undefined Research Problem

This is one of the most basic aspects of research design. The research problem statement must be clear and to do that, you must set the framework for the development of research questions that address the core problems.

5. Research Limitations

Every study has some type of limitations . You should anticipate and incorporate those limitations into your conclusion, as well as the basic research design. Include a statement in your manuscript about any perceived limitations, and how you considered them while designing your experiment and drawing the conclusion.

6. Ethical Implications

The most important yet less talked about topic is the ethical issue. Your research design must include ways to minimize any risk for your participants and also address the research problem or question at hand. If you cannot manage the ethical norms along with your research study, your research objectives and validity could be questioned.

Experimental Research Design Example

In an experimental design, a researcher gathers plant samples and then randomly assigns half the samples to photosynthesize in sunlight and the other half to be kept in a dark box without sunlight, while controlling all the other variables (nutrients, water, soil, etc.)

By comparing their outcomes in biochemical tests, the researcher can confirm that the changes in the plants were due to the sunlight and not the other variables.

Experimental research is often the final form of a study conducted in the research process which is considered to provide conclusive and specific results. But it is not meant for every research. It involves a lot of resources, time, and money and is not easy to conduct, unless a foundation of research is built. Yet it is widely used in research institutes and commercial industries, for its most conclusive results in the scientific approach.

Have you worked on research designs? How was your experience creating an experimental design? What difficulties did you face? Do write to us or comment below and share your insights on experimental research designs!

Frequently Asked Questions

Randomization is important in an experimental research because it ensures unbiased results of the experiment. It also measures the cause-effect relationship on a particular group of interest.

Experimental research design lay the foundation of a research and structures the research to establish quality decision making process.

There are 3 types of experimental research designs. These are pre-experimental research design, true experimental research design, and quasi experimental research design.

The difference between an experimental and a quasi-experimental design are: 1. The assignment of the control group in quasi experimental research is non-random, unlike true experimental design, which is randomly assigned. 2. Experimental research group always has a control group; on the other hand, it may not be always present in quasi experimental research.

Experimental research establishes a cause-effect relationship by testing a theory or hypothesis using experimental groups or control variables. In contrast, descriptive research describes a study or a topic by defining the variables under it and answering the questions related to the same.

good and valuable

Very very good

Good presentation.

Rate this article Cancel Reply

Your email address will not be published.

Enago Academy's Most Popular Articles

• Industry News

Google Releases 2024 Scholar Metrics, Evaluates Impact of Scholarly Articles

Google has released its 2024 Scholar Metrics, assessing scholarly articles from 2019 to 2023. This…

Ensuring Academic Integrity and Transparency in Academic Research: A comprehensive checklist for researchers

Academic integrity is the foundation upon which the credibility and value of scientific findings are…

• Publishing Research
• Reporting Research

How to Optimize Your Research Process: A step-by-step guide

For researchers across disciplines, the path to uncovering novel findings and insights is often filled…

• Trending Now

Breaking Barriers: Sony and Nature unveil “Women in Technology Award”

Sony Group Corporation and the prestigious scientific journal Nature have collaborated to launch the inaugural…

Achieving Research Excellence: Checklist for good research practices

Academia is built on the foundation of trustworthy and high-quality research, supported by the pillars…

Choosing the Right Analytical Approach: Thematic analysis vs. content analysis for…

Comparing Cross Sectional and Longitudinal Studies: 5 steps for choosing the right…

Research Recommendations – Guiding policy-makers for evidence-based decision making

Sign-up to read more

Subscribe for free to get unrestricted access to all our resources on research writing and academic publishing including:

• 2000+ blog articles
• 50+ Webinars
• 10+ Expert podcasts
• 50+ Infographics
• 10+ Checklists
• Research Guides

We hate spam too. We promise to protect your privacy and never spam you.

• AI in Academia
• Promoting Research
• Career Corner
• Diversity and Inclusion
• Infographics
• Expert Video Library
• Other Resources
• Enago Learn
• Upcoming & On-Demand Webinars
• Peer-Review Week 2023
• Open Access Week 2023
• Conference Videos
• Enago Report
• Journal Finder
• Enago Plagiarism & AI Grammar Check
• Editing Services
• Publication Support Services
• Research Impact
• Translation Services
• Publication solutions
• AI-Based Solutions
• Thought Leadership
• Call for Articles
• Call for Speakers
• Author Training
• Edit Profile

I am looking for Editing/ Proofreading services for my manuscript Tentative date of next journal submission:

In your opinion, what is the most effective way to improve integrity in the peer review process?

• Skip to main content
• Skip to primary sidebar
• Skip to footer
• QuestionPro

• Solutions Industries Gaming Automotive Sports and events Education Government Travel & Hospitality Financial Services Healthcare Cannabis Technology Use Case NPS+ Communities Audience Contactless surveys Mobile LivePolls Member Experience GDPR Positive People Science 360 Feedback Surveys
• Resources Blog eBooks Survey Templates Case Studies Training Help center

Home Market Research

Experimental Research: What it is + Types of designs

Any research conducted under scientifically acceptable conditions uses experimental methods. The success of experimental studies hinges on researchers confirming the change of a variable is based solely on the manipulation of the constant variable. The research should establish a notable cause and effect.

What is Experimental Research?

Experimental research is a study conducted with a scientific approach using two sets of variables. The first set acts as a constant, which you use to measure the differences of the second set. Quantitative research methods , for example, are experimental.

If you don’t have enough data to support your decisions, you must first determine the facts. This research gathers the data necessary to help you make better decisions.

You can conduct experimental research in the following situations:

• Time is a vital factor in establishing a relationship between cause and effect.
• Invariable behavior between cause and effect.
• You wish to understand the importance of cause and effect.

Experimental Research Design Types

The classic experimental design definition is: “The methods used to collect data in experimental studies.”

There are three primary types of experimental design:

• Pre-experimental research design
• True experimental research design
• Quasi-experimental research design

The way you classify research subjects based on conditions or groups determines the type of research design  you should use.

0 1. Pre-Experimental Design

A group, or various groups, are kept under observation after implementing cause and effect factors. You’ll conduct this research to understand whether further investigation is necessary for these particular groups.

You can break down pre-experimental research further into three types:

• One-shot Case Study Research Design
• One-group Pretest-posttest Research Design
• Static-group Comparison

0 2. True Experimental Design

It relies on statistical analysis to prove or disprove a hypothesis, making it the most accurate form of research. Of the types of experimental design, only true design can establish a cause-effect relationship within a group. In a true experiment, three factors need to be satisfied:

• There is a Control Group, which won’t be subject to changes, and an Experimental Group, which will experience the changed variables.
• A variable that can be manipulated by the researcher
• Random distribution

This experimental research method commonly occurs in the physical sciences.

0 3. Quasi-Experimental Design

The word “Quasi” indicates similarity. A quasi-experimental design is similar to an experimental one, but it is not the same. The difference between the two is the assignment of a control group. In this research, an independent variable is manipulated, but the participants of a group are not randomly assigned. Quasi-research is used in field settings where random assignment is either irrelevant or not required.

Importance of Experimental Design

Experimental research is a powerful tool for understanding cause-and-effect relationships. It allows us to manipulate variables and observe the effects, which is crucial for understanding how different factors influence the outcome of a study.

But the importance of experimental research goes beyond that. It’s a critical method for many scientific and academic studies. It allows us to test theories, develop new products, and make groundbreaking discoveries.

For example, this research is essential for developing new drugs and medical treatments. Researchers can understand how a new drug works by manipulating dosage and administration variables and identifying potential side effects.

Similarly, experimental research is used in the field of psychology to test theories and understand human behavior. By manipulating variables such as stimuli, researchers can gain insights into how the brain works and identify new treatment options for mental health disorders.

It is also widely used in the field of education. It allows educators to test new teaching methods and identify what works best. By manipulating variables such as class size, teaching style, and curriculum, researchers can understand how students learn and identify new ways to improve educational outcomes.

In addition, experimental research is a powerful tool for businesses and organizations. By manipulating variables such as marketing strategies, product design, and customer service, companies can understand what works best and identify new opportunities for growth.

Advantages of Experimental Research

When talking about this research, we can think of human life. Babies do their own rudimentary experiments (such as putting objects in their mouths) to learn about the world around them, while older children and teens do experiments at school to learn more about science.

Ancient scientists used this research to prove that their hypotheses were correct. For example, Galileo Galilei and Antoine Lavoisier conducted various experiments to discover key concepts in physics and chemistry. The same is true of modern experts, who use this scientific method to see if new drugs are effective, discover treatments for diseases, and create new electronic devices (among others).

It’s vital to test new ideas or theories. Why put time, effort, and funding into something that may not work?

This research allows you to test your idea in a controlled environment before marketing. It also provides the best method to test your theory thanks to the following advantages:

• Researchers have a stronger hold over variables to obtain desired results.
• The subject or industry does not impact the effectiveness of experimental research. Any industry can implement it for research purposes.
• The results are specific.
• After analyzing the results, you can apply your findings to similar ideas or situations.
• You can identify the cause and effect of a hypothesis. Researchers can further analyze this relationship to determine more in-depth ideas.
• Experimental research makes an ideal starting point. The data you collect is a foundation for building more ideas and conducting more action research .

Whether you want to know how the public will react to a new product or if a certain food increases the chance of disease, experimental research is the best place to start. Begin your research by finding subjects using  QuestionPro Audience  and other tools today.

LEARN MORE         FREE TRIAL

MORE LIKE THIS

Types of Organizational Change & Strategies for Business

Aug 2, 2024

Voice of the Customer Programs: What It Is, Implementations

Aug 1, 2024

A Case for Empowerment and Being Bold — Tuesday CX Thoughts

Jul 30, 2024

Typeform vs. Google Forms: Which one is best for my needs?

Other categories.

• Academic Research
• Artificial Intelligence
• Assessments
• Brand Awareness
• Case Studies
• Communities
• Consumer Insights
• Customer effort score
• Customer Engagement
• Customer Experience
• Customer Loyalty
• Customer Research
• Customer Satisfaction
• Employee Benefits
• Employee Engagement
• Employee Retention
• Friday Five
• General Data Protection Regulation
• Insights Hub
• Life@QuestionPro
• Market Research
• Mobile diaries
• Mobile Surveys
• New Features
• Online Communities
• Question Types
• Questionnaire
• QuestionPro Products
• Release Notes
• Research Tools and Apps
• Revenue at Risk
• Survey Templates
• Training Tips
• Tuesday CX Thoughts (TCXT)
• Uncategorized
• What’s Coming Up
• Workforce Intelligence

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

10 Experimental research

Experimental research—often considered to be the ‘gold standard’ in research designs—is one of the most rigorous of all research designs. In this design, one or more independent variables are manipulated by the researcher (as treatments), subjects are randomly assigned to different treatment levels (random assignment), and the results of the treatments on outcomes (dependent variables) are observed. The unique strength of experimental research is its internal validity (causality) due to its ability to link cause and effect through treatment manipulation, while controlling for the spurious effect of extraneous variable.

Experimental research is best suited for explanatory research—rather than for descriptive or exploratory research—where the goal of the study is to examine cause-effect relationships. It also works well for research that involves a relatively limited and well-defined set of independent variables that can either be manipulated or controlled. Experimental research can be conducted in laboratory or field settings. Laboratory experiments , conducted in laboratory (artificial) settings, tend to be high in internal validity, but this comes at the cost of low external validity (generalisability), because the artificial (laboratory) setting in which the study is conducted may not reflect the real world. Field experiments are conducted in field settings such as in a real organisation, and are high in both internal and external validity. But such experiments are relatively rare, because of the difficulties associated with manipulating treatments and controlling for extraneous effects in a field setting.

Experimental research can be grouped into two broad categories: true experimental designs and quasi-experimental designs. Both designs require treatment manipulation, but while true experiments also require random assignment, quasi-experiments do not. Sometimes, we also refer to non-experimental research, which is not really a research design, but an all-inclusive term that includes all types of research that do not employ treatment manipulation or random assignment, such as survey research, observational research, and correlational studies.

Basic concepts

Treatment and control groups. In experimental research, some subjects are administered one or more experimental stimulus called a treatment (the treatment group ) while other subjects are not given such a stimulus (the control group ). The treatment may be considered successful if subjects in the treatment group rate more favourably on outcome variables than control group subjects. Multiple levels of experimental stimulus may be administered, in which case, there may be more than one treatment group. For example, in order to test the effects of a new drug intended to treat a certain medical condition like dementia, if a sample of dementia patients is randomly divided into three groups, with the first group receiving a high dosage of the drug, the second group receiving a low dosage, and the third group receiving a placebo such as a sugar pill (control group), then the first two groups are experimental groups and the third group is a control group. After administering the drug for a period of time, if the condition of the experimental group subjects improved significantly more than the control group subjects, we can say that the drug is effective. We can also compare the conditions of the high and low dosage experimental groups to determine if the high dose is more effective than the low dose.

Treatment manipulation. Treatments are the unique feature of experimental research that sets this design apart from all other research methods. Treatment manipulation helps control for the ‘cause’ in cause-effect relationships. Naturally, the validity of experimental research depends on how well the treatment was manipulated. Treatment manipulation must be checked using pretests and pilot tests prior to the experimental study. Any measurements conducted before the treatment is administered are called pretest measures , while those conducted after the treatment are posttest measures .

Random selection and assignment. Random selection is the process of randomly drawing a sample from a population or a sampling frame. This approach is typically employed in survey research, and ensures that each unit in the population has a positive chance of being selected into the sample. Random assignment, however, is a process of randomly assigning subjects to experimental or control groups. This is a standard practice in true experimental research to ensure that treatment groups are similar (equivalent) to each other and to the control group prior to treatment administration. Random selection is related to sampling, and is therefore more closely related to the external validity (generalisability) of findings. However, random assignment is related to design, and is therefore most related to internal validity. It is possible to have both random selection and random assignment in well-designed experimental research, but quasi-experimental research involves neither random selection nor random assignment.

Threats to internal validity. Although experimental designs are considered more rigorous than other research methods in terms of the internal validity of their inferences (by virtue of their ability to control causes through treatment manipulation), they are not immune to internal validity threats. Some of these threats to internal validity are described below, within the context of a study of the impact of a special remedial math tutoring program for improving the math abilities of high school students.

History threat is the possibility that the observed effects (dependent variables) are caused by extraneous or historical events rather than by the experimental treatment. For instance, students’ post-remedial math score improvement may have been caused by their preparation for a math exam at their school, rather than the remedial math program.

Maturation threat refers to the possibility that observed effects are caused by natural maturation of subjects (e.g., a general improvement in their intellectual ability to understand complex concepts) rather than the experimental treatment.

Testing threat is a threat in pre-post designs where subjects’ posttest responses are conditioned by their pretest responses. For instance, if students remember their answers from the pretest evaluation, they may tend to repeat them in the posttest exam.

Not conducting a pretest can help avoid this threat.

Instrumentation threat , which also occurs in pre-post designs, refers to the possibility that the difference between pretest and posttest scores is not due to the remedial math program, but due to changes in the administered test, such as the posttest having a higher or lower degree of difficulty than the pretest.

Mortality threat refers to the possibility that subjects may be dropping out of the study at differential rates between the treatment and control groups due to a systematic reason, such that the dropouts were mostly students who scored low on the pretest. If the low-performing students drop out, the results of the posttest will be artificially inflated by the preponderance of high-performing students.

Regression threat —also called a regression to the mean—refers to the statistical tendency of a group’s overall performance to regress toward the mean during a posttest rather than in the anticipated direction. For instance, if subjects scored high on a pretest, they will have a tendency to score lower on the posttest (closer to the mean) because their high scores (away from the mean) during the pretest were possibly a statistical aberration. This problem tends to be more prevalent in non-random samples and when the two measures are imperfectly correlated.

Two-group experimental designs

Pretest-posttest control group design . In this design, subjects are randomly assigned to treatment and control groups, subjected to an initial (pretest) measurement of the dependent variables of interest, the treatment group is administered a treatment (representing the independent variable of interest), and the dependent variables measured again (posttest). The notation of this design is shown in Figure 10.1.

Statistical analysis of this design involves a simple analysis of variance (ANOVA) between the treatment and control groups. The pretest-posttest design handles several threats to internal validity, such as maturation, testing, and regression, since these threats can be expected to influence both treatment and control groups in a similar (random) manner. The selection threat is controlled via random assignment. However, additional threats to internal validity may exist. For instance, mortality can be a problem if there are differential dropout rates between the two groups, and the pretest measurement may bias the posttest measurement—especially if the pretest introduces unusual topics or content.

Posttest -only control group design . This design is a simpler version of the pretest-posttest design where pretest measurements are omitted. The design notation is shown in Figure 10.2.

The treatment effect is measured simply as the difference in the posttest scores between the two groups:

The appropriate statistical analysis of this design is also a two-group analysis of variance (ANOVA). The simplicity of this design makes it more attractive than the pretest-posttest design in terms of internal validity. This design controls for maturation, testing, regression, selection, and pretest-posttest interaction, though the mortality threat may continue to exist.

Because the pretest measure is not a measurement of the dependent variable, but rather a covariate, the treatment effect is measured as the difference in the posttest scores between the treatment and control groups as:

Due to the presence of covariates, the right statistical analysis of this design is a two-group analysis of covariance (ANCOVA). This design has all the advantages of posttest-only design, but with internal validity due to the controlling of covariates. Covariance designs can also be extended to pretest-posttest control group design.

Factorial designs

Two-group designs are inadequate if your research requires manipulation of two or more independent variables (treatments). In such cases, you would need four or higher-group designs. Such designs, quite popular in experimental research, are commonly called factorial designs. Each independent variable in this design is called a factor , and each subdivision of a factor is called a level . Factorial designs enable the researcher to examine not only the individual effect of each treatment on the dependent variables (called main effects), but also their joint effect (called interaction effects).

In a factorial design, a main effect is said to exist if the dependent variable shows a significant difference between multiple levels of one factor, at all levels of other factors. No change in the dependent variable across factor levels is the null case (baseline), from which main effects are evaluated. In the above example, you may see a main effect of instructional type, instructional time, or both on learning outcomes. An interaction effect exists when the effect of differences in one factor depends upon the level of a second factor. In our example, if the effect of instructional type on learning outcomes is greater for three hours/week of instructional time than for one and a half hours/week, then we can say that there is an interaction effect between instructional type and instructional time on learning outcomes. Note that the presence of interaction effects dominate and make main effects irrelevant, and it is not meaningful to interpret main effects if interaction effects are significant.

Hybrid experimental designs

Hybrid designs are those that are formed by combining features of more established designs. Three such hybrid designs are randomised bocks design, Solomon four-group design, and switched replications design.

Randomised block design. This is a variation of the posttest-only or pretest-posttest control group design where the subject population can be grouped into relatively homogeneous subgroups (called blocks ) within which the experiment is replicated. For instance, if you want to replicate the same posttest-only design among university students and full-time working professionals (two homogeneous blocks), subjects in both blocks are randomly split between the treatment group (receiving the same treatment) and the control group (see Figure 10.5). The purpose of this design is to reduce the ‘noise’ or variance in data that may be attributable to differences between the blocks so that the actual effect of interest can be detected more accurately.

Solomon four-group design . In this design, the sample is divided into two treatment groups and two control groups. One treatment group and one control group receive the pretest, and the other two groups do not. This design represents a combination of posttest-only and pretest-posttest control group design, and is intended to test for the potential biasing effect of pretest measurement on posttest measures that tends to occur in pretest-posttest designs, but not in posttest-only designs. The design notation is shown in Figure 10.6.

Switched replication design . This is a two-group design implemented in two phases with three waves of measurement. The treatment group in the first phase serves as the control group in the second phase, and the control group in the first phase becomes the treatment group in the second phase, as illustrated in Figure 10.7. In other words, the original design is repeated or replicated temporally with treatment/control roles switched between the two groups. By the end of the study, all participants will have received the treatment either during the first or the second phase. This design is most feasible in organisational contexts where organisational programs (e.g., employee training) are implemented in a phased manner or are repeated at regular intervals.

Quasi-experimental designs

Quasi-experimental designs are almost identical to true experimental designs, but lacking one key ingredient: random assignment. For instance, one entire class section or one organisation is used as the treatment group, while another section of the same class or a different organisation in the same industry is used as the control group. This lack of random assignment potentially results in groups that are non-equivalent, such as one group possessing greater mastery of certain content than the other group, say by virtue of having a better teacher in a previous semester, which introduces the possibility of selection bias . Quasi-experimental designs are therefore inferior to true experimental designs in interval validity due to the presence of a variety of selection related threats such as selection-maturation threat (the treatment and control groups maturing at different rates), selection-history threat (the treatment and control groups being differentially impacted by extraneous or historical events), selection-regression threat (the treatment and control groups regressing toward the mean between pretest and posttest at different rates), selection-instrumentation threat (the treatment and control groups responding differently to the measurement), selection-testing (the treatment and control groups responding differently to the pretest), and selection-mortality (the treatment and control groups demonstrating differential dropout rates). Given these selection threats, it is generally preferable to avoid quasi-experimental designs to the greatest extent possible.

In addition, there are quite a few unique non-equivalent designs without corresponding true experimental design cousins. Some of the more useful of these designs are discussed next.

Regression discontinuity (RD) design . This is a non-equivalent pretest-posttest design where subjects are assigned to the treatment or control group based on a cut-off score on a preprogram measure. For instance, patients who are severely ill may be assigned to a treatment group to test the efficacy of a new drug or treatment protocol and those who are mildly ill are assigned to the control group. In another example, students who are lagging behind on standardised test scores may be selected for a remedial curriculum program intended to improve their performance, while those who score high on such tests are not selected from the remedial program.

Because of the use of a cut-off score, it is possible that the observed results may be a function of the cut-off score rather than the treatment, which introduces a new threat to internal validity. However, using the cut-off score also ensures that limited or costly resources are distributed to people who need them the most, rather than randomly across a population, while simultaneously allowing a quasi-experimental treatment. The control group scores in the RD design do not serve as a benchmark for comparing treatment group scores, given the systematic non-equivalence between the two groups. Rather, if there is no discontinuity between pretest and posttest scores in the control group, but such a discontinuity persists in the treatment group, then this discontinuity is viewed as evidence of the treatment effect.

Proxy pretest design . This design, shown in Figure 10.11, looks very similar to the standard NEGD (pretest-posttest) design, with one critical difference: the pretest score is collected after the treatment is administered. A typical application of this design is when a researcher is brought in to test the efficacy of a program (e.g., an educational program) after the program has already started and pretest data is not available. Under such circumstances, the best option for the researcher is often to use a different prerecorded measure, such as students’ grade point average before the start of the program, as a proxy for pretest data. A variation of the proxy pretest design is to use subjects’ posttest recollection of pretest data, which may be subject to recall bias, but nevertheless may provide a measure of perceived gain or change in the dependent variable.

Separate pretest-posttest samples design . This design is useful if it is not possible to collect pretest and posttest data from the same subjects for some reason. As shown in Figure 10.12, there are four groups in this design, but two groups come from a single non-equivalent group, while the other two groups come from a different non-equivalent group. For instance, say you want to test customer satisfaction with a new online service that is implemented in one city but not in another. In this case, customers in the first city serve as the treatment group and those in the second city constitute the control group. If it is not possible to obtain pretest and posttest measures from the same customers, you can measure customer satisfaction at one point in time, implement the new service program, and measure customer satisfaction (with a different set of customers) after the program is implemented. Customer satisfaction is also measured in the control group at the same times as in the treatment group, but without the new program implementation. The design is not particularly strong, because you cannot examine the changes in any specific customer’s satisfaction score before and after the implementation, but you can only examine average customer satisfaction scores. Despite the lower internal validity, this design may still be a useful way of collecting quasi-experimental data when pretest and posttest data is not available from the same subjects.

An interesting variation of the NEDV design is a pattern-matching NEDV design , which employs multiple outcome variables and a theory that explains how much each variable will be affected by the treatment. The researcher can then examine if the theoretical prediction is matched in actual observations. This pattern-matching technique—based on the degree of correspondence between theoretical and observed patterns—is a powerful way of alleviating internal validity concerns in the original NEDV design.

Perils of experimental research

Experimental research is one of the most difficult of research designs, and should not be taken lightly. This type of research is often best with a multitude of methodological problems. First, though experimental research requires theories for framing hypotheses for testing, much of current experimental research is atheoretical. Without theories, the hypotheses being tested tend to be ad hoc, possibly illogical, and meaningless. Second, many of the measurement instruments used in experimental research are not tested for reliability and validity, and are incomparable across studies. Consequently, results generated using such instruments are also incomparable. Third, often experimental research uses inappropriate research designs, such as irrelevant dependent variables, no interaction effects, no experimental controls, and non-equivalent stimulus across treatment groups. Findings from such studies tend to lack internal validity and are highly suspect. Fourth, the treatments (tasks) used in experimental research may be diverse, incomparable, and inconsistent across studies, and sometimes inappropriate for the subject population. For instance, undergraduate student subjects are often asked to pretend that they are marketing managers and asked to perform a complex budget allocation task in which they have no experience or expertise. The use of such inappropriate tasks, introduces new threats to internal validity (i.e., subject’s performance may be an artefact of the content or difficulty of the task setting), generates findings that are non-interpretable and meaningless, and makes integration of findings across studies impossible.

The design of proper experimental treatments is a very important task in experimental design, because the treatment is the raison d’etre of the experimental method, and must never be rushed or neglected. To design an adequate and appropriate task, researchers should use prevalidated tasks if available, conduct treatment manipulation checks to check for the adequacy of such tasks (by debriefing subjects after performing the assigned task), conduct pilot tests (repeatedly, if necessary), and if in doubt, use tasks that are simple and familiar for the respondent sample rather than tasks that are complex or unfamiliar.

In summary, this chapter introduced key concepts in the experimental design research method and introduced a variety of true experimental and quasi-experimental designs. Although these designs vary widely in internal validity, designs with less internal validity should not be overlooked and may sometimes be useful under specific circumstances and empirical contingencies.

Social Science Research: Principles, Methods and Practices (Revised edition) Copyright © 2019 by Anol Bhattacherjee is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Share This Book

• Experimental Research Designs: Types, Examples & Methods

Experimental research is the most familiar type of research design for individuals in the physical sciences and a host of other fields. This is mainly because experimental research is a classical scientific experiment, similar to those performed in high school science classes.

Imagine taking 2 samples of the same plant and exposing one of them to sunlight, while the other is kept away from sunlight. Let the plant exposed to sunlight be called sample A, while the latter is called sample B.

If after the duration of the research, we find out that sample A grows and sample B dies, even though they are both regularly wetted and given the same treatment. Therefore, we can conclude that sunlight will aid growth in all similar plants.

What is Experimental Research?

Experimental research is a scientific approach to research, where one or more independent variables are manipulated and applied to one or more dependent variables to measure their effect on the latter. The effect of the independent variables on the dependent variables is usually observed and recorded over some time, to aid researchers in drawing a reasonable conclusion regarding the relationship between these 2 variable types.

The experimental research method is widely used in physical and social sciences, psychology, and education. It is based on the comparison between two or more groups with a straightforward logic, which may, however, be difficult to execute.

Mostly related to a laboratory test procedure, experimental research designs involve collecting quantitative data and performing statistical analysis on them during research. Therefore, making it an example of quantitative research method .

What are The Types of Experimental Research Design?

The types of experimental research design are determined by the way the researcher assigns subjects to different conditions and groups. They are of 3 types, namely; pre-experimental, quasi-experimental, and true experimental research.

Pre-experimental Research Design

In pre-experimental research design, either a group or various dependent groups are observed for the effect of the application of an independent variable which is presumed to cause change. It is the simplest form of experimental research design and is treated with no control group.

Although very practical, experimental research is lacking in several areas of the true-experimental criteria. The pre-experimental research design is further divided into three types

• One-shot Case Study Research Design

In this type of experimental study, only one dependent group or variable is considered. The study is carried out after some treatment which was presumed to cause change, making it a posttest study.

• One-group Pretest-posttest Research Design: 

This research design combines both posttest and pretest study by carrying out a test on a single group before the treatment is administered and after the treatment is administered. With the former being administered at the beginning of treatment and later at the end.

• Static-group Comparison: 

In a static-group comparison study, 2 or more groups are placed under observation, where only one of the groups is subjected to some treatment while the other groups are held static. All the groups are post-tested, and the observed differences between the groups are assumed to be a result of the treatment.

Quasi-experimental Research Design

  The word “quasi” means partial, half, or pseudo. Therefore, the quasi-experimental research bearing a resemblance to the true experimental research, but not the same.  In quasi-experiments, the participants are not randomly assigned, and as such, they are used in settings where randomization is difficult or impossible.

 This is very common in educational research, where administrators are unwilling to allow the random selection of students for experimental samples.

Some examples of quasi-experimental research design include; the time series, no equivalent control group design, and the counterbalanced design.

True Experimental Research Design

The true experimental research design relies on statistical analysis to approve or disprove a hypothesis. It is the most accurate type of experimental design and may be carried out with or without a pretest on at least 2 randomly assigned dependent subjects.

The true experimental research design must contain a control group, a variable that can be manipulated by the researcher, and the distribution must be random. The classification of true experimental design include:

• The posttest-only Control Group Design: In this design, subjects are randomly selected and assigned to the 2 groups (control and experimental), and only the experimental group is treated. After close observation, both groups are post-tested, and a conclusion is drawn from the difference between these groups.
• The pretest-posttest Control Group Design: For this control group design, subjects are randomly assigned to the 2 groups, both are presented, but only the experimental group is treated. After close observation, both groups are post-tested to measure the degree of change in each group.
• Solomon four-group Design: This is the combination of the pretest-only and the pretest-posttest control groups. In this case, the randomly selected subjects are placed into 4 groups.

The first two of these groups are tested using the posttest-only method, while the other two are tested using the pretest-posttest method.

Examples of Experimental Research

Experimental research examples are different, depending on the type of experimental research design that is being considered. The most basic example of experimental research is laboratory experiments, which may differ in nature depending on the subject of research.

Administering Exams After The End of Semester

During the semester, students in a class are lectured on particular courses and an exam is administered at the end of the semester. In this case, the students are the subjects or dependent variables while the lectures are the independent variables treated on the subjects.

Only one group of carefully selected subjects are considered in this research, making it a pre-experimental research design example. We will also notice that tests are only carried out at the end of the semester, and not at the beginning.

Further making it easy for us to conclude that it is a one-shot case study research. 

Employee Skill Evaluation

Before employing a job seeker, organizations conduct tests that are used to screen out less qualified candidates from the pool of qualified applicants. This way, organizations can determine an employee’s skill set at the point of employment.

In the course of employment, organizations also carry out employee training to improve employee productivity and generally grow the organization. Further evaluation is carried out at the end of each training to test the impact of the training on employee skills, and test for improvement.

Here, the subject is the employee, while the treatment is the training conducted. This is a pretest-posttest control group experimental research example.

Evaluation of Teaching Method

Let us consider an academic institution that wants to evaluate the teaching method of 2 teachers to determine which is best. Imagine a case whereby the students assigned to each teacher is carefully selected probably due to personal request by parents or due to stubbornness and smartness.

This is a no equivalent group design example because the samples are not equal. By evaluating the effectiveness of each teacher’s teaching method this way, we may conclude after a post-test has been carried out.

However, this may be influenced by factors like the natural sweetness of a student. For example, a very smart student will grab more easily than his or her peers irrespective of the method of teaching.

What are the Characteristics of Experimental Research?  

Experimental research contains dependent, independent and extraneous variables. The dependent variables are the variables being treated or manipulated and are sometimes called the subject of the research.

The independent variables are the experimental treatment being exerted on the dependent variables. Extraneous variables, on the other hand, are other factors affecting the experiment that may also contribute to the change.

The setting is where the experiment is carried out. Many experiments are carried out in the laboratory, where control can be exerted on the extraneous variables, thereby eliminating them.

Other experiments are carried out in a less controllable setting. The choice of setting used in research depends on the nature of the experiment being carried out.

• Multivariable

Experimental research may include multiple independent variables, e.g. time, skills, test scores, etc.

Why Use Experimental Research Design?  

Experimental research design can be majorly used in physical sciences, social sciences, education, and psychology. It is used to make predictions and draw conclusions on a subject matter. 

Some uses of experimental research design are highlighted below.

• Medicine: Experimental research is used to provide the proper treatment for diseases. In most cases, rather than directly using patients as the research subject, researchers take a sample of the bacteria from the patient’s body and are treated with the developed antibacterial

The changes observed during this period are recorded and evaluated to determine its effectiveness. This process can be carried out using different experimental research methods.

• Education: Asides from science subjects like Chemistry and Physics which involves teaching students how to perform experimental research, it can also be used in improving the standard of an academic institution. This includes testing students’ knowledge on different topics, coming up with better teaching methods, and the implementation of other programs that will aid student learning.
• Human Behavior: Social scientists are the ones who mostly use experimental research to test human behaviour. For example, consider 2 people randomly chosen to be the subject of the social interaction research where one person is placed in a room without human interaction for 1 year.

The other person is placed in a room with a few other people, enjoying human interaction. There will be a difference in their behaviour at the end of the experiment.

• UI/UX: During the product development phase, one of the major aims of the product team is to create a great user experience with the product. Therefore, before launching the final product design, potential are brought in to interact with the product.

For example, when finding it difficult to choose how to position a button or feature on the app interface, a random sample of product testers are allowed to test the 2 samples and how the button positioning influences the user interaction is recorded.

What are the Disadvantages of Experimental Research?  

• It is highly prone to human error due to its dependency on variable control which may not be properly implemented. These errors could eliminate the validity of the experiment and the research being conducted.
• Exerting control of extraneous variables may create unrealistic situations. Eliminating real-life variables will result in inaccurate conclusions. This may also result in researchers controlling the variables to suit his or her personal preferences.
• It is a time-consuming process. So much time is spent on testing dependent variables and waiting for the effect of the manipulation of dependent variables to manifest.
• It is expensive.
• It is very risky and may have ethical complications that cannot be ignored. This is common in medical research, where failed trials may lead to a patient’s death or a deteriorating health condition.
• Experimental research results are not descriptive.
• Response bias can also be supplied by the subject of the conversation.
• Human responses in experimental research can be difficult to measure.

What are the Data Collection Methods in Experimental Research?  

Data collection methods in experimental research are the different ways in which data can be collected for experimental research. They are used in different cases, depending on the type of research being carried out.

1. Observational Study

This type of study is carried out over a long period. It measures and observes the variables of interest without changing existing conditions.

When researching the effect of social interaction on human behavior, the subjects who are placed in 2 different environments are observed throughout the research. No matter the kind of absurd behavior that is exhibited by the subject during this period, its condition will not be changed.

This may be a very risky thing to do in medical cases because it may lead to death or worse medical conditions.

2. Simulations

This procedure uses mathematical, physical, or computer models to replicate a real-life process or situation. It is frequently used when the actual situation is too expensive, dangerous, or impractical to replicate in real life.

This method is commonly used in engineering and operational research for learning purposes and sometimes as a tool to estimate possible outcomes of real research. Some common situation software are Simulink, MATLAB, and Simul8.

Not all kinds of experimental research can be carried out using simulation as a data collection tool . It is very impractical for a lot of laboratory-based research that involves chemical processes.

A survey is a tool used to gather relevant data about the characteristics of a population and is one of the most common data collection tools. A survey consists of a group of questions prepared by the researcher, to be answered by the research subject.

Surveys can be shared with the respondents both physically and electronically. When collecting data through surveys, the kind of data collected depends on the respondent, and researchers have limited control over it.

Formplus is the best tool for collecting experimental data using survey s. It has relevant features that will aid the data collection process and can also be used in other aspects of experimental research.

Differences between Experimental and Non-Experimental Research 

1. In experimental research, the researcher can control and manipulate the environment of the research, including the predictor variable which can be changed. On the other hand, non-experimental research cannot be controlled or manipulated by the researcher at will.

This is because it takes place in a real-life setting, where extraneous variables cannot be eliminated. Therefore, it is more difficult to conclude non-experimental studies, even though they are much more flexible and allow for a greater range of study fields.

2. The relationship between cause and effect cannot be established in non-experimental research, while it can be established in experimental research. This may be because many extraneous variables also influence the changes in the research subject, making it difficult to point at a particular variable as the cause of a particular change

3. Independent variables are not introduced, withdrawn, or manipulated in non-experimental designs, but the same may not be said about experimental research.

Experimental Research vs. Alternatives and When to Use Them

1. experimental research vs causal comparative.

Experimental research enables you to control variables and identify how the independent variable affects the dependent variable. Causal-comparative find out the cause-and-effect relationship between the variables by comparing already existing groups that are affected differently by the independent variable.

For example, in an experiment to see how K-12 education affects children and teenager development. An experimental research would split the children into groups, some would get formal K-12 education, while others won’t. This is not ethically right because every child has the right to education. So, what we do instead would be to compare already existing groups of children who are getting formal education with those who due to some circumstances can not.

Pros and Cons of Experimental vs Causal-Comparative Research

• Causal-Comparative:   Strengths:  More realistic than experiments, can be conducted in real-world settings.  Weaknesses:  Establishing causality can be weaker due to the lack of manipulation.

2. Experimental Research vs Correlational Research

When experimenting, you are trying to establish a cause-and-effect relationship between different variables. For example, you are trying to establish the effect of heat on water, the temperature keeps changing (independent variable) and you see how it affects the water (dependent variable).

For correlational research, you are not necessarily interested in the why or the cause-and-effect relationship between the variables, you are focusing on the relationship. Using the same water and temperature example, you are only interested in the fact that they change, you are not investigating which of the variables or other variables causes them to change.

Pros and Cons of Experimental vs Correlational Research

3. experimental research vs descriptive research.

With experimental research, you alter the independent variable to see how it affects the dependent variable, but with descriptive research you are simply studying the characteristics of the variable you are studying.

So, in an experiment to see how blown glass reacts to temperature, experimental research would keep altering the temperature to varying levels of high and low to see how it affects the dependent variable (glass). But descriptive research would investigate the glass properties.

Pros and Cons of Experimental vs Descriptive Research

4. experimental research vs action research.

Experimental research tests for causal relationships by focusing on one independent variable vs the dependent variable and keeps other variables constant. So, you are testing hypotheses and using the information from the research to contribute to knowledge.

However, with action research, you are using a real-world setting which means you are not controlling variables. You are also performing the research to solve actual problems and improve already established practices.

For example, if you are testing for how long commutes affect workers’ productivity. With experimental research, you would vary the length of commute to see how the time affects work. But with action research, you would account for other factors such as weather, commute route, nutrition, etc. Also, experimental research helps know the relationship between commute time and productivity, while action research helps you look for ways to improve productivity

Pros and Cons of Experimental vs Action Research

Conclusion  .

Experimental research designs are often considered to be the standard in research designs. This is partly due to the common misconception that research is equivalent to scientific experiments—a component of experimental research design.

In this research design, one or more subjects or dependent variables are randomly assigned to different treatments (i.e. independent variables manipulated by the researcher) and the results are observed to conclude. One of the uniqueness of experimental research is in its ability to control the effect of extraneous variables.

Experimental research is suitable for research whose goal is to examine cause-effect relationships, e.g. explanatory research. It can be conducted in the laboratory or field settings, depending on the aim of the research that is being carried out. 

Connect to Formplus, Get Started Now - It's Free!

• examples of experimental research
• experimental research methods
• types of experimental research
• busayo.longe

You may also like:

Experimental Vs Non-Experimental Research: 15 Key Differences

Differences between experimental and non experimental research on definitions, types, examples, data collection tools, uses, advantages etc.

What is Experimenter Bias? Definition, Types & Mitigation

In this article, we will look into the concept of experimental bias and how it can be identified in your research

Response vs Explanatory Variables: Definition & Examples

In this article, we’ll be comparing the two types of variables, what they both mean and see some of their real-life applications in research

Simpson’s Paradox & How to Avoid it in Experimental Research

In this article, we are going to look at Simpson’s Paradox from its historical point and later, we’ll consider its effect in...

Formplus - For Seamless Data Collection

Collect data the right way with a versatile data collection tool. try formplus and transform your work productivity today..

Experimental Design: Types, Examples & Methods

Saul Mcleod, PhD

Editor-in-Chief for Simply Psychology

BSc (Hons) Psychology, MRes, PhD, University of Manchester

Saul Mcleod, PhD., is a qualified psychology teacher with over 18 years of experience in further and higher education. He has been published in peer-reviewed journals, including the Journal of Clinical Psychology.

Learn about our Editorial Process

Olivia Guy-Evans, MSc

Associate Editor for Simply Psychology

BSc (Hons) Psychology, MSc Psychology of Education

Olivia Guy-Evans is a writer and associate editor for Simply Psychology. She has previously worked in healthcare and educational sectors.

On This Page:

Experimental design refers to how participants are allocated to different groups in an experiment. Types of design include repeated measures, independent groups, and matched pairs designs.

Probably the most common way to design an experiment in psychology is to divide the participants into two groups, the experimental group and the control group, and then introduce a change to the experimental group, not the control group.

The researcher must decide how he/she will allocate their sample to the different experimental groups.  For example, if there are 10 participants, will all 10 participants participate in both groups (e.g., repeated measures), or will the participants be split in half and take part in only one group each?

Three types of experimental designs are commonly used:

1. Independent Measures

Independent measures design, also known as between-groups , is an experimental design where different participants are used in each condition of the independent variable.  This means that each condition of the experiment includes a different group of participants.

This should be done by random allocation, ensuring that each participant has an equal chance of being assigned to one group.

Independent measures involve using two separate groups of participants, one in each condition. For example:

• Con : More people are needed than with the repeated measures design (i.e., more time-consuming).
• Pro : Avoids order effects (such as practice or fatigue) as people participate in one condition only.  If a person is involved in several conditions, they may become bored, tired, and fed up by the time they come to the second condition or become wise to the requirements of the experiment!
• Con : Differences between participants in the groups may affect results, for example, variations in age, gender, or social background.  These differences are known as participant variables (i.e., a type of extraneous variable ).
• Control : After the participants have been recruited, they should be randomly assigned to their groups. This should ensure the groups are similar, on average (reducing participant variables).

2. Repeated Measures Design

Repeated Measures design is an experimental design where the same participants participate in each independent variable condition.  This means that each experiment condition includes the same group of participants.

Repeated Measures design is also known as within-groups or within-subjects design .

• Pro : As the same participants are used in each condition, participant variables (i.e., individual differences) are reduced.
• Con : There may be order effects. Order effects refer to the order of the conditions affecting the participants’ behavior.  Performance in the second condition may be better because the participants know what to do (i.e., practice effect).  Or their performance might be worse in the second condition because they are tired (i.e., fatigue effect). This limitation can be controlled using counterbalancing.
• Pro : Fewer people are needed as they participate in all conditions (i.e., saves time).
• Control : To combat order effects, the researcher counter-balances the order of the conditions for the participants.  Alternating the order in which participants perform in different conditions of an experiment.

Counterbalancing

Suppose we used a repeated measures design in which all of the participants first learned words in “loud noise” and then learned them in “no noise.”

We expect the participants to learn better in “no noise” because of order effects, such as practice. However, a researcher can control for order effects using counterbalancing.

The sample would be split into two groups: experimental (A) and control (B).  For example, group 1 does ‘A’ then ‘B,’ and group 2 does ‘B’ then ‘A.’ This is to eliminate order effects.

Although order effects occur for each participant, they balance each other out in the results because they occur equally in both groups.

3. Matched Pairs Design

A matched pairs design is an experimental design where pairs of participants are matched in terms of key variables, such as age or socioeconomic status. One member of each pair is then placed into the experimental group and the other member into the control group .

One member of each matched pair must be randomly assigned to the experimental group and the other to the control group.

• Con : If one participant drops out, you lose 2 PPs’ data.
• Pro : Reduces participant variables because the researcher has tried to pair up the participants so that each condition has people with similar abilities and characteristics.
• Con : Very time-consuming trying to find closely matched pairs.
• Pro : It avoids order effects, so counterbalancing is not necessary.
• Con : Impossible to match people exactly unless they are identical twins!
• Control : Members of each pair should be randomly assigned to conditions. However, this does not solve all these problems.

Experimental design refers to how participants are allocated to an experiment’s different conditions (or IV levels). There are three types:

1. Independent measures / between-groups : Different participants are used in each condition of the independent variable.

2. Repeated measures /within groups : The same participants take part in each condition of the independent variable.

3. Matched pairs : Each condition uses different participants, but they are matched in terms of important characteristics, e.g., gender, age, intelligence, etc.

Learning Check

Read about each of the experiments below. For each experiment, identify (1) which experimental design was used; and (2) why the researcher might have used that design.

1 . To compare the effectiveness of two different types of therapy for depression, depressed patients were assigned to receive either cognitive therapy or behavior therapy for a 12-week period.

The researchers attempted to ensure that the patients in the two groups had similar severity of depressed symptoms by administering a standardized test of depression to each participant, then pairing them according to the severity of their symptoms.

2 . To assess the difference in reading comprehension between 7 and 9-year-olds, a researcher recruited each group from a local primary school. They were given the same passage of text to read and then asked a series of questions to assess their understanding.

3 . To assess the effectiveness of two different ways of teaching reading, a group of 5-year-olds was recruited from a primary school. Their level of reading ability was assessed, and then they were taught using scheme one for 20 weeks.

At the end of this period, their reading was reassessed, and a reading improvement score was calculated. They were then taught using scheme two for a further 20 weeks, and another reading improvement score for this period was calculated. The reading improvement scores for each child were then compared.

4 . To assess the effect of the organization on recall, a researcher randomly assigned student volunteers to two conditions.

Condition one attempted to recall a list of words that were organized into meaningful categories; condition two attempted to recall the same words, randomly grouped on the page.

Experiment Terminology

Ecological validity.

The degree to which an investigation represents real-life experiences.

Experimenter effects

These are the ways that the experimenter can accidentally influence the participant through their appearance or behavior.

Demand characteristics

The clues in an experiment lead the participants to think they know what the researcher is looking for (e.g., the experimenter’s body language).

Independent variable (IV)

The variable the experimenter manipulates (i.e., changes) is assumed to have a direct effect on the dependent variable.

Dependent variable (DV)

Variable the experimenter measures. This is the outcome (i.e., the result) of a study.

Extraneous variables (EV)

All variables which are not independent variables but could affect the results (DV) of the experiment. Extraneous variables should be controlled where possible.

Confounding variables

Variable(s) that have affected the results (DV), apart from the IV. A confounding variable could be an extraneous variable that has not been controlled.

Random Allocation

Randomly allocating participants to independent variable conditions means that all participants should have an equal chance of taking part in each condition.

The principle of random allocation is to avoid bias in how the experiment is carried out and limit the effects of participant variables.

Order effects

Changes in participants’ performance due to their repeating the same or similar test more than once. Examples of order effects include:

(i) practice effect: an improvement in performance on a task due to repetition, for example, because of familiarity with the task;

(ii) fatigue effect: a decrease in performance of a task due to repetition, for example, because of boredom or tiredness.

Exploring Experimental Research: Methodologies, Designs, and Applications Across Disciplines

• SSRN Electronic Journal

• The National University of Cheasim Kamchaymear

Discover the world's research

• 25+ million members
• 160+ million publication pages
• 2.3+ billion citations

• COMPUT COMMUN REV

• Debbie Rohwer

• Sorakrich Maneewan
• Ravinder Koul
• Int J Contemp Hospit Manag

• J EXP ANAL BEHAV
• Alan E. Kazdin
• Jimmie Leppink
• Keith Morrison
• Louis Cohen
• Lawrence Manion
• ACCOUNT ORG SOC
• Wim A. Van der Stede
• Recruit researchers
• Join for free
• Login Email Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google Welcome back! Please log in. Email · Hint Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google No account? Sign up

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• Types of Research Designs Compared | Guide & Examples

Types of Research Designs Compared | Guide & Examples

Published on June 20, 2019 by Shona McCombes . Revised on June 22, 2023.

When you start planning a research project, developing research questions and creating a  research design , you will have to make various decisions about the type of research you want to do.

There are many ways to categorize different types of research. The words you use to describe your research depend on your discipline and field. In general, though, the form your research design takes will be shaped by:

• The type of knowledge you aim to produce
• The type of data you will collect and analyze
• The sampling methods , timescale and location of the research

This article takes a look at some common distinctions made between different types of research and outlines the key differences between them.

Table of contents

Types of research aims, types of research data, types of sampling, timescale, and location, other interesting articles.

The first thing to consider is what kind of knowledge your research aims to contribute.

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

• Academic style
• Vague sentences
• Style consistency

See an example

The next thing to consider is what type of data you will collect. Each kind of data is associated with a range of specific research methods and procedures.

Finally, you have to consider three closely related questions: how will you select the subjects or participants of the research? When and how often will you collect data from your subjects? And where will the research take place?

Keep in mind that the methods that you choose bring with them different risk factors and types of research bias . Biases aren’t completely avoidable, but can heavily impact the validity and reliability of your findings if left unchecked.

Choosing between all these different research types is part of the process of creating your research design , which determines exactly how your research will be conducted. But the type of research is only the first step: next, you have to make more concrete decisions about your research methods and the details of the study.

Read more about creating a research design

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Normal distribution
• Degrees of freedom
• Null hypothesis
• Discourse analysis
• Control groups
• Mixed methods research
• Non-probability sampling
• Quantitative research
• Ecological validity

Research bias

• Rosenthal effect
• Implicit bias
• Cognitive bias
• Selection bias
• Negativity bias
• Status quo bias

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, June 22). Types of Research Designs Compared | Guide & Examples. Scribbr. Retrieved July 30, 2024, from https://www.scribbr.com/methodology/types-of-research/

Is this article helpful?

Shona McCombes

Other students also liked, what is a research design | types, guide & examples, qualitative vs. quantitative research | differences, examples & methods, what is a research methodology | steps & tips, get unlimited documents corrected.

✔ Free APA citation check included ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

Child Care and Early Education Research Connections

Experiments and quasi-experiments.

This page includes an explanation of the types, key components, validity, ethics, and advantages and disadvantages of experimental design.

An experiment is a study in which the researcher manipulates the level of some independent variable and then measures the outcome. Experiments are powerful techniques for evaluating cause-and-effect relationships. Many researchers consider experiments the "gold standard" against which all other research designs should be judged. Experiments are conducted both in the laboratory and in real life situations.

Types of Experimental Design

There are two basic types of research design:

• True experiments
• Quasi-experiments

The purpose of both is to examine the cause of certain phenomena.

True experiments, in which all the important factors that might affect the phenomena of interest are completely controlled, are the preferred design. Often, however, it is not possible or practical to control all the key factors, so it becomes necessary to implement a quasi-experimental research design.

Similarities between true and quasi-experiments:

• Study participants are subjected to some type of treatment or condition
• Some outcome of interest is measured
• The researchers test whether differences in this outcome are related to the treatment

Differences between true experiments and quasi-experiments:

• In a true experiment, participants are randomly assigned to either the treatment or the control group, whereas they are not assigned randomly in a quasi-experiment
• In a quasi-experiment, the control and treatment groups differ not only in terms of the experimental treatment they receive, but also in other, often unknown or unknowable, ways. Thus, the researcher must try to statistically control for as many of these differences as possible
• Because control is lacking in quasi-experiments, there may be several "rival hypotheses" competing with the experimental manipulation as explanations for observed results

Key Components of Experimental Research Design

The manipulation of predictor variables.

In an experiment, the researcher manipulates the factor that is hypothesized to affect the outcome of interest. The factor that is being manipulated is typically referred to as the treatment or intervention. The researcher may manipulate whether research subjects receive a treatment (e.g., antidepressant medicine: yes or no) and the level of treatment (e.g., 50 mg, 75 mg, 100 mg, and 125 mg).

Suppose, for example, a group of researchers was interested in the causes of maternal employment. They might hypothesize that the provision of government-subsidized child care would promote such employment. They could then design an experiment in which some subjects would be provided the option of government-funded child care subsidies and others would not. The researchers might also manipulate the value of the child care subsidies in order to determine if higher subsidy values might result in different levels of maternal employment.

Random Assignment

• Study participants are randomly assigned to different treatment groups
• All participants have the same chance of being in a given condition
• Participants are assigned to either the group that receives the treatment, known as the "experimental group" or "treatment group," or to the group which does not receive the treatment, referred to as the "control group"
• Random assignment neutralizes factors other than the independent and dependent variables, making it possible to directly infer cause and effect

Random Sampling

Traditionally, experimental researchers have used convenience sampling to select study participants. However, as research methods have become more rigorous, and the problems with generalizing from a convenience sample to the larger population have become more apparent, experimental researchers are increasingly turning to random sampling. In experimental policy research studies, participants are often randomly selected from program administrative databases and randomly assigned to the control or treatment groups.

Validity of Results

The two types of validity of experiments are internal and external. It is often difficult to achieve both in social science research experiments.

Internal Validity

• When an experiment is internally valid, we are certain that the independent variable (e.g., child care subsidies) caused the outcome of the study (e.g., maternal employment)
• When subjects are randomly assigned to treatment or control groups, we can assume that the independent variable caused the observed outcomes because the two groups should not have differed from one another at the start of the experiment
• For example, take the child care subsidy example above. Since research subjects were randomly assigned to the treatment (child care subsidies available) and control (no child care subsidies available) groups, the two groups should not have differed at the outset of the study. If, after the intervention, mothers in the treatment group were more likely to be working, we can assume that the availability of child care subsidies promoted maternal employment

One potential threat to internal validity in experiments occurs when participants either drop out of the study or refuse to participate in the study. If particular types of individuals drop out or refuse to participate more often than individuals with other characteristics, this is called differential attrition. For example, suppose an experiment was conducted to assess the effects of a new reading curriculum. If the new curriculum was so tough that many of the slowest readers dropped out of school, the school with the new curriculum would experience an increase in the average reading scores. The reason they experienced an increase in reading scores, however, is because the worst readers left the school, not because the new curriculum improved students' reading skills.

External Validity

• External validity is also of particular concern in social science experiments
• It can be very difficult to generalize experimental results to groups that were not included in the study
• Studies that randomly select participants from the most diverse and representative populations are more likely to have external validity
• The use of random sampling techniques makes it easier to generalize the results of studies to other groups

For example, a research study shows that a new curriculum improved reading comprehension of third-grade children in Iowa. To assess the study's external validity, you would ask whether this new curriculum would also be effective with third graders in New York or with children in other elementary grades.

Glossary terms related to validity:

• internal validity
• external validity
• differential attrition

It is particularly important in experimental research to follow ethical guidelines. Protecting the health and safety of research subjects is imperative. In order to assure subject safety, all researchers should have their project reviewed by the Institutional Review Boards (IRBS). The  National Institutes of Health  supplies strict guidelines for project approval. Many of these guidelines are based on the  Belmont Report  (pdf).

The basic ethical principles:

• Respect for persons  -- requires that research subjects are not coerced into participating in a study and requires the protection of research subjects who have diminished autonomy
• Beneficence  -- requires that experiments do not harm research subjects, and that researchers minimize the risks for subjects while maximizing the benefits for them
• Justice  -- requires that all forms of differential treatment among research subjects be justified

Advantages and Disadvantages of Experimental Design

The environment in which the research takes place can often be carefully controlled. Consequently, it is easier to estimate the true effect of the variable of interest on the outcome of interest.

Disadvantages

It is often difficult to assure the external validity of the experiment, due to the frequently nonrandom selection processes and the artificial nature of the experimental context.

• Skip to secondary menu
• Skip to main content
• Skip to primary sidebar

Statistics By Jim

Making statistics intuitive

Experimental Design: Definition and Types

By Jim Frost 3 Comments

What is Experimental Design?

An experimental design is a detailed plan for collecting and using data to identify causal relationships. Through careful planning, the design of experiments allows your data collection efforts to have a reasonable chance of detecting effects and testing hypotheses that answer your research questions.

An experiment is a data collection procedure that occurs in controlled conditions to identify and understand causal relationships between variables. Researchers can use many potential designs. The ultimate choice depends on their research question, resources, goals, and constraints. In some fields of study, researchers refer to experimental design as the design of experiments (DOE). Both terms are synonymous.

Ultimately, the design of experiments helps ensure that your procedures and data will evaluate your research question effectively. Without an experimental design, you might waste your efforts in a process that, for many potential reasons, can’t answer your research question. In short, it helps you trust your results.

Learn more about Independent and Dependent Variables .

Design of Experiments: Goals & Settings

Experiments occur in many settings, ranging from psychology, social sciences, medicine, physics, engineering, and industrial and service sectors. Typically, experimental goals are to discover a previously unknown effect , confirm a known effect, or test a hypothesis.

Effects represent causal relationships between variables. For example, in a medical experiment, does the new medicine cause an improvement in health outcomes? If so, the medicine has a causal effect on the outcome.

An experimental design’s focus depends on the subject area and can include the following goals:

• Understanding the relationships between variables.
• Identifying the variables that have the largest impact on the outcomes.
• Finding the input variable settings that produce an optimal result.

For example, psychologists have conducted experiments to understand how conformity affects decision-making. Sociologists have performed experiments to determine whether ethnicity affects the public reaction to staged bike thefts. These experiments map out the causal relationships between variables, and their primary goal is to understand the role of various factors.

Conversely, in a manufacturing environment, the researchers might use an experimental design to find the factors that most effectively improve their product’s strength, identify the optimal manufacturing settings, and do all that while accounting for various constraints. In short, a manufacturer’s goal is often to use experiments to improve their products cost-effectively.

In a medical experiment, the goal might be to quantify the medicine’s effect and find the optimum dosage.

Developing an Experimental Design

Developing an experimental design involves planning that maximizes the potential to collect data that is both trustworthy and able to detect causal relationships. Specifically, these studies aim to see effects when they exist in the population the researchers are studying, preferentially favor causal effects, isolate each factor’s true effect from potential confounders, and produce conclusions that you can generalize to the real world.

To accomplish these goals, experimental designs carefully manage data validity and reliability , and internal and external experimental validity. When your experiment is valid and reliable, you can expect your procedures and data to produce trustworthy results.

An excellent experimental design involves the following:

• Lots of preplanning.
• Developing experimental treatments.
• Determining how to assign subjects to treatment groups.

The remainder of this article focuses on how experimental designs incorporate these essential items to accomplish their research goals.

Learn more about Data Reliability vs. Validity and Internal and External Experimental Validity .

Preplanning, Defining, and Operationalizing for Design of Experiments

This phase of the design of experiments helps you identify critical variables, know how to measure them while ensuring reliability and validity, and understand the relationships between them. The review can also help you find ways to reduce sources of variability, which increases your ability to detect treatment effects. Notably, the literature review allows you to learn how similar studies designed their experiments and the challenges they faced.

Operationalizing a study involves taking your research question, using the background information you gathered, and formulating an actionable plan.

This process should produce a specific and testable hypothesis using data that you can reasonably collect given the resources available to the experiment.

• Null hypothesis : The jumping exercise intervention does not affect bone density.
• Alternative hypothesis : The jumping exercise intervention affects bone density.

To learn more about this early phase, read Five Steps for Conducting Scientific Studies with Statistical Analyses .

Formulating Treatments in Experimental Designs

In an experimental design, treatments are variables that the researchers control. They are the primary independent variables of interest. Researchers administer the treatment to the subjects or items in the experiment and want to know whether it causes changes in the outcome.

As the name implies, a treatment can be medical in nature, such as a new medicine or vaccine. But it’s a general term that applies to other things such as training programs, manufacturing settings, teaching methods, and types of fertilizers. I helped run an experiment where the treatment was a jumping exercise intervention that we hoped would increase bone density. All these treatment examples are things that potentially influence a measurable outcome.

Even when you know your treatment generally, you must carefully consider the amount. How large of a dose? If you’re comparing three different temperatures in a manufacturing process, how far apart are they? For my bone mineral density study, we had to determine how frequently the exercise sessions would occur and how long each lasted.

How you define the treatments in the design of experiments can affect your findings and the generalizability of your results.

Assigning Subjects to Experimental Groups

A crucial decision for all experimental designs is determining how researchers assign subjects to the experimental conditions—the treatment and control groups. The control group is often, but not always, the lack of a treatment. It serves as a basis for comparison by showing outcomes for subjects who don’t receive a treatment. Learn more about Control Groups .

How your experimental design assigns subjects to the groups affects how confident you can be that the findings represent true causal effects rather than mere correlation caused by confounders. Indeed, the assignment method influences how you control for confounding variables. This is the difference between correlation and causation .

Imagine a study finds that vitamin consumption correlates with better health outcomes. As a researcher, you want to be able to say that vitamin consumption causes the improvements. However, with the wrong experimental design, you might only be able to say there is an association. A confounder, and not the vitamins, might actually cause the health benefits.

Let’s explore some of the ways to assign subjects in design of experiments.

Completely Randomized Designs

A completely randomized experimental design randomly assigns all subjects to the treatment and control groups. You simply take each participant and use a random process to determine their group assignment. You can flip coins, roll a die, or use a computer. Randomized experiments must be prospective studies because they need to be able to control group assignment.

Random assignment in the design of experiments helps ensure that the groups are roughly equivalent at the beginning of the study. This equivalence at the start increases your confidence that any differences you see at the end were caused by the treatments. The randomization tends to equalize confounders between the experimental groups and, thereby, cancels out their effects, leaving only the treatment effects.

For example, in a vitamin study, the researchers can randomly assign participants to either the control or vitamin group. Because the groups are approximately equal when the experiment starts, if the health outcomes are different at the end of the study, the researchers can be confident that the vitamins caused those improvements.

Statisticians consider randomized experimental designs to be the best for identifying causal relationships.

If you can’t randomly assign subjects but want to draw causal conclusions about an intervention, consider using a quasi-experimental design .

Learn more about Randomized Controlled Trials and Random Assignment in Experiments .

Randomized Block Designs

Nuisance factors are variables that can affect the outcome, but they are not the researcher’s primary interest. Unfortunately, they can hide or distort the treatment results. When experimenters know about specific nuisance factors, they can use a randomized block design to minimize their impact.

This experimental design takes subjects with a shared “nuisance” characteristic and groups them into blocks. The participants in each block are then randomly assigned to the experimental groups. This process allows the experiment to control for known nuisance factors.

Blocking in the design of experiments reduces the impact of nuisance factors on experimental error. The analysis assesses the effects of the treatment within each block, which removes the variability between blocks. The result is that blocked experimental designs can reduce the impact of nuisance variables, increasing the ability to detect treatment effects accurately.

Suppose you’re testing various teaching methods. Because grade level likely affects educational outcomes, you might use grade level as a blocking factor. To use a randomized block design for this scenario, divide the participants by grade level and then randomly assign the members of each grade level to the experimental groups.

A standard guideline for an experimental design is to “Block what you can, randomize what you cannot.” Use blocking for a few primary nuisance factors. Then use random assignment to distribute the unblocked nuisance factors equally between the experimental conditions.

You can also use covariates to control nuisance factors. Learn about Covariates: Definition and Uses .

Observational Studies

In some experimental designs, randomly assigning subjects to the experimental conditions is impossible or unethical. The researchers simply can’t assign participants to the experimental groups. However, they can observe them in their natural groupings, measure the essential variables, and look for correlations. These observational studies are also known as quasi-experimental designs. Retrospective studies must be observational in nature because they look back at past events.

Imagine you’re studying the effects of depression on an activity. Clearly, you can’t randomly assign participants to the depression and control groups. But you can observe participants with and without depression and see how their task performance differs.

Observational studies let you perform research when you can’t control the treatment. However, quasi-experimental designs increase the problem of confounding variables. For this design of experiments, correlation does not necessarily imply causation. While special procedures can help control confounders in an observational study, you’re ultimately less confident that the results represent causal findings.

Learn more about Observational Studies .

For a good comparison, learn about the differences and tradeoffs between Observational Studies and Randomized Experiments .

Between-Subjects vs. Within-Subjects Experimental Designs

When you think of the design of experiments, you probably picture a treatment and control group. Researchers assign participants to only one of these groups, so each group contains entirely different subjects than the other groups. Analysts compare the groups at the end of the experiment. Statisticians refer to this method as a between-subjects, or independent measures, experimental design.

In a between-subjects design , you can have more than one treatment group, but each subject is exposed to only one condition, the control group or one of the treatment groups.

A potential downside to this approach is that differences between groups at the beginning can affect the results at the end. As you’ve read earlier, random assignment can reduce those differences, but it is imperfect. There will always be some variability between the groups.

In a  within-subjects experimental design , also known as repeated measures, subjects experience all treatment conditions and are measured for each. Each subject acts as their own control, which reduces variability and increases the statistical power to detect effects.

In this experimental design, you minimize pre-existing differences between the experimental conditions because they all contain the same subjects. However, the order of treatments can affect the results. Beware of practice and fatigue effects. Learn more about Repeated Measures Designs .

Design of Experiments Examples

For example, a bone density study has three experimental groups—a control group, a stretching exercise group, and a jumping exercise group.

In a between-subjects experimental design, scientists randomly assign each participant to one of the three groups.

In a within-subjects design, all subjects experience the three conditions sequentially while the researchers measure bone density repeatedly. The procedure can switch the order of treatments for the participants to help reduce order effects.

Matched Pairs Experimental Design

A matched pairs experimental design is a between-subjects study that uses pairs of similar subjects. Researchers use this approach to reduce pre-existing differences between experimental groups. It’s yet another design of experiments method for reducing sources of variability.

Researchers identify variables likely to affect the outcome, such as demographics. When they pick a subject with a set of characteristics, they try to locate another participant with similar attributes to create a matched pair. Scientists randomly assign one member of a pair to the treatment group and the other to the control group.

On the plus side, this process creates two similar groups, and it doesn’t create treatment order effects. While matched pairs do not produce the perfectly matched groups of a within-subjects design (which uses the same subjects in all conditions), it aims to reduce variability between groups relative to a between-subjects study.

On the downside, finding matched pairs is very time-consuming. Additionally, if one member of a matched pair drops out, the other subject must leave the study too.

Learn more about Matched Pairs Design: Uses & Examples .

Another consideration is whether you’ll use a cross-sectional design (one point in time) or use a longitudinal study to track changes over time .

A case study is a research method that often serves as a precursor to a more rigorous experimental design by identifying research questions, variables, and hypotheses to test. Learn more about What is a Case Study? Definition & Examples .

In conclusion, the design of experiments is extremely sensitive to subject area concerns and the time and resources available to the researchers. Developing a suitable experimental design requires balancing a multitude of considerations. A successful design is necessary to obtain trustworthy answers to your research question and to have a reasonable chance of detecting treatment effects when they exist.

Share this:

Reader Interactions

March 23, 2024 at 2:35 pm

Dear Jim You wrote a superb document, I will use it in my Buistatistics course, along with your three books. Thank you very much! Miguel

March 23, 2024 at 5:43 pm

Thanks so much, Miguel! Glad this post was helpful and I trust the books will be as well.

April 10, 2023 at 4:36 am

What are the purpose and uses of experimental research design?

Comments and Questions Cancel reply

Our websites may use cookies to personalize and enhance your experience. By continuing without changing your cookie settings, you agree to this collection. For more information, please see our University Websites Privacy Notice .

Neag School of Education

Educational Research Basics by Del Siegle

Experimental research.

The major feature that distinguishes experimental research from other types of research is that the researcher manipulates the independent variable.  There are a number of experimental group designs in experimental research. Some of these qualify as experimental research, others do not.

• In true experimental research , the researcher not only manipulates the independent variable, he or she also randomly assigned individuals to the various treatment categories (i.e., control and treatment).
• In quasi experimental research , the researcher does not randomly assign subjects to treatment and control groups. In other words, the treatment is not distributed among participants randomly. In some cases, a researcher may randomly assigns one whole group to treatment and one whole group to control. In this case, quasi-experimental research involves using intact groups in an experiment, rather than assigning individuals at random to research conditions. (some researchers define this latter situation differently. For our course, we will allow this definition).
• In causal comparative ( ex post facto ) research, the groups are already formed. It does not meet the standards of an experiment because the independent variable in not manipulated.

The statistics by themselves have no meaning. They only take on meaning within the design of your study. If we just examine stats, bread can be deadly . The term validity is used three ways in research…

• I n the sampling unit, we learn about external validity (generalizability).
• I n the survey unit, we learn about instrument validity .
• In this unit, we learn about internal validity and external validity . Internal validity means that the differences that we were found between groups on the dependent variable in an experiment were directly related to what the researcher did to the independent variable, and not due to some other unintended variable (confounding variable). Simply stated, the question addressed by internal validity is “Was the study done well?” Once the researcher is satisfied that the study was done well and the independent variable caused the dependent variable (internal validity), then the research examines external validity (under what conditions [ecological] and with whom [population] can these results be replicated [Will I get the same results with a different group of people or under different circumstances?]). If a study is not internally valid, then considering external validity is a moot point (If the independent did not cause the dependent, then there is no point in applying the results [generalizing the results] to other situations.). Interestingly, as one tightens a study to control for treats to internal validity, one decreases the generalizability of the study (to whom and under what conditions one can generalize the results).

There are several common threats to internal validity in experimental research. They are described in our text.  I have review each below (this material is also included in the  PowerPoint Presentation on Experimental Research for this unit):

• Subject Characteristics (Selection Bias/Differential Selection) — The groups may have been different from the start. If you were testing instructional strategies to improve reading and one group enjoyed reading more than the other group, they may improve more in their reading because they enjoy it, rather than the instructional strategy you used.
• Loss of Subjects ( Mortality ) — All of the high or low scoring subject may have dropped out or were missing from one of the groups. If we collected posttest data on a day when the honor society was on field trip at the treatment school, the mean for the treatment group would probably be much lower than it really should have been.
• Location — Perhaps one group was at a disadvantage because of their location.  The city may have been demolishing a building next to one of the schools in our study and there are constant distractions which interferes with our treatment.
• Instrumentation Instrument Decay — The testing instruments may not be scores similarly. Perhaps the person grading the posttest is fatigued and pays less attention to the last set of papers reviewed. It may be that those papers are from one of our groups and will received different scores than the earlier group’s papers
• Data Collector Characteristics — The subjects of one group may react differently to the data collector than the other group. A male interviewing males and females about their attitudes toward a type of math instruction may not receive the same responses from females as a female interviewing females would.
• Data Collector Bias — The person collecting data my favors one group, or some characteristic some subject possess, over another. A principal who favors strict classroom management may rate students’ attention under different teaching conditions with a bias toward one of the teaching conditions.
• Testing — The act of taking a pretest or posttest may influence the results of the experiment. Suppose we were conducting a unit to increase student sensitivity to prejudice. As a pretest we have the control and treatment groups watch Shindler’s List and write a reaction essay. The pretest may have actually increased both groups’ sensitivity and we find that our treatment groups didn’t score any higher on a posttest given later than the control group did. If we hadn’t given the pretest, we might have seen differences in the groups at the end of the study.
• History — Something may happen at one site during our study that influences the results. Perhaps a classmate dies in a car accident at the control site for a study teaching children bike safety. The control group may actually demonstrate more concern about bike safety than the treatment group.
• Maturation –There may be natural changes in the subjects that can account for the changes found in a study. A critical thinking unit may appear more effective if it taught during a time when children are developing abstract reasoning.
• Hawthorne Effect — The subjects may respond differently just because they are being studied. The name comes from a classic study in which researchers were studying the effect of lighting on worker productivity. As the intensity of the factor lights increased, so did the work productivity. One researcher suggested that they reverse the treatment and lower the lights. The productivity of the workers continued to increase. It appears that being observed by the researchers was increasing productivity, not the intensity of the lights.
• John Henry Effect — One group may view that it is competition with the other group and may work harder than than they would under normal circumstances. This generally is applied to the control group “taking on” the treatment group. The terms refers to the classic story of John Henry laying railroad track.
• Resentful Demoralization of the Control Group — The control group may become discouraged because it is not receiving the special attention that is given to the treatment group. They may perform lower than usual because of this.
• Regression ( Statistical Regression) — A class that scores particularly low can be expected to score slightly higher just by chance. Likewise, a class that scores particularly high, will have a tendency to score slightly lower by chance. The change in these scores may have nothing to do with the treatment.
• Implementation –The treatment may not be implemented as intended. A study where teachers are asked to use student modeling techniques may not show positive results, not because modeling techniques don’t work, but because the teacher didn’t implement them or didn’t implement them as they were designed.
• Compensatory Equalization of Treatmen t — Someone may feel sorry for the control group because they are not receiving much attention and give them special treatment. For example, a researcher could be studying the effect of laptop computers on students’ attitudes toward math. The teacher feels sorry for the class that doesn’t have computers and sponsors a popcorn party during math class. The control group begins to develop a more positive attitude about mathematics.
• Experimental Treatment Diffusion — Sometimes the control group actually implements the treatment. If two different techniques are being tested in two different third grades in the same building, the teachers may share what they are doing. Unconsciously, the control may use of the techniques she or he learned from the treatment teacher.

When planning a study, it is important to consider the threats to interval validity as we finalize the study design. After we complete our study, we should reconsider each of the threats to internal validity as we review our data and draw conclusions.

Del Siegle, Ph.D. Neag School of Education – University of Connecticut [email protected] www.delsiegle.com

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• J Athl Train
• v.45(1); Jan-Feb 2010

Study/Experimental/Research Design: Much More Than Statistics

Kenneth l. knight.

Brigham Young University, Provo, UT

The purpose of study, experimental, or research design in scientific manuscripts has changed significantly over the years. It has evolved from an explanation of the design of the experiment (ie, data gathering or acquisition) to an explanation of the statistical analysis. This practice makes “Methods” sections hard to read and understand.

To clarify the difference between study design and statistical analysis, to show the advantages of a properly written study design on article comprehension, and to encourage authors to correctly describe study designs.

Description:

The role of study design is explored from the introduction of the concept by Fisher through modern-day scientists and the AMA Manual of Style . At one time, when experiments were simpler, the study design and statistical design were identical or very similar. With the complex research that is common today, which often includes manipulating variables to create new variables and the multiple (and different) analyses of a single data set, data collection is very different than statistical design. Thus, both a study design and a statistical design are necessary.

Advantages:

Scientific manuscripts will be much easier to read and comprehend. A proper experimental design serves as a road map to the study methods, helping readers to understand more clearly how the data were obtained and, therefore, assisting them in properly analyzing the results.

Study, experimental, or research design is the backbone of good research. It directs the experiment by orchestrating data collection, defines the statistical analysis of the resultant data, and guides the interpretation of the results. When properly described in the written report of the experiment, it serves as a road map to readers, 1 helping them negotiate the “Methods” section, and, thus, it improves the clarity of communication between authors and readers.

A growing trend is to equate study design with only the statistical analysis of the data. The design statement typically is placed at the end of the “Methods” section as a subsection called “Experimental Design” or as part of a subsection called “Data Analysis.” This placement, however, equates experimental design and statistical analysis, minimizing the effect of experimental design on the planning and reporting of an experiment. This linkage is inappropriate, because some of the elements of the study design that should be described at the beginning of the “Methods” section are instead placed in the “Statistical Analysis” section or, worse, are absent from the manuscript entirely.

Have you ever interrupted your reading of the “Methods” to sketch out the variables in the margins of the paper as you attempt to understand how they all fit together? Or have you jumped back and forth from the early paragraphs of the “Methods” section to the “Statistics” section to try to understand which variables were collected and when? These efforts would be unnecessary if a road map at the beginning of the “Methods” section outlined how the independent variables were related, which dependent variables were measured, and when they were measured. When they were measured is especially important if the variables used in the statistical analysis were a subset of the measured variables or were computed from measured variables (such as change scores).

The purpose of this Communications article is to clarify the purpose and placement of study design elements in an experimental manuscript. Adopting these ideas may improve your science and surely will enhance the communication of that science. These ideas will make experimental manuscripts easier to read and understand and, therefore, will allow them to become part of readers' clinical decision making.

WHAT IS A STUDY (OR EXPERIMENTAL OR RESEARCH) DESIGN?

The terms study design, experimental design, and research design are often thought to be synonymous and are sometimes used interchangeably in a single paper. Avoid doing so. Use the term that is preferred by the style manual of the journal for which you are writing. Study design is the preferred term in the AMA Manual of Style , 2 so I will use it here.

A study design is the architecture of an experimental study 3 and a description of how the study was conducted, 4 including all elements of how the data were obtained. 5 The study design should be the first subsection of the “Methods” section in an experimental manuscript (see the Table ). “Statistical Design” or, preferably, “Statistical Analysis” or “Data Analysis” should be the last subsection of the “Methods” section.

Table. Elements of a “Methods” Section

The “Study Design” subsection describes how the variables and participants interacted. It begins with a general statement of how the study was conducted (eg, crossover trials, parallel, or observational study). 2 The second element, which usually begins with the second sentence, details the number of independent variables or factors, the levels of each variable, and their names. A shorthand way of doing so is with a statement such as “A 2 × 4 × 8 factorial guided data collection.” This tells us that there were 3 independent variables (factors), with 2 levels of the first factor, 4 levels of the second factor, and 8 levels of the third factor. Following is a sentence that names the levels of each factor: for example, “The independent variables were sex (male or female), training program (eg, walking, running, weight lifting, or plyometrics), and time (2, 4, 6, 8, 10, 15, 20, or 30 weeks).” Such an approach clearly outlines for readers how the various procedures fit into the overall structure and, therefore, enhances their understanding of how the data were collected. Thus, the design statement is a road map of the methods.

The dependent (or measurement or outcome) variables are then named. Details of how they were measured are not given at this point in the manuscript but are explained later in the “Instruments” and “Procedures” subsections.

Next is a paragraph detailing who the participants were and how they were selected, placed into groups, and assigned to a particular treatment order, if the experiment was a repeated-measures design. And although not a part of the design per se, a statement about obtaining written informed consent from participants and institutional review board approval is usually included in this subsection.

The nuts and bolts of the “Methods” section follow, including such things as equipment, materials, protocols, etc. These are beyond the scope of this commentary, however, and so will not be discussed.

The last part of the “Methods” section and last part of the “Study Design” section is the “Data Analysis” subsection. It begins with an explanation of any data manipulation, such as how data were combined or how new variables (eg, ratios or differences between collected variables) were calculated. Next, readers are told of the statistical measures used to analyze the data, such as a mixed 2 × 4 × 8 analysis of variance (ANOVA) with 2 between-groups factors (sex and training program) and 1 within-groups factor (time of measurement). Researchers should state and reference the statistical package and procedure(s) within the package used to compute the statistics. (Various statistical packages perform analyses slightly differently, so it is important to know the package and specific procedure used.) This detail allows readers to judge the appropriateness of the statistical measures and the conclusions drawn from the data.

STATISTICAL DESIGN VERSUS STATISTICAL ANALYSIS

Avoid using the term statistical design . Statistical methods are only part of the overall design. The term gives too much emphasis to the statistics, which are important, but only one of many tools used in interpreting data and only part of the study design:

The most important issues in biostatistics are not expressed with statistical procedures. The issues are inherently scientific, rather than purely statistical, and relate to the architectural design of the research, not the numbers with which the data are cited and interpreted. 6

Stated another way, “The justification for the analysis lies not in the data collected but in the manner in which the data were collected.” 3 “Without the solid foundation of a good design, the edifice of statistical analysis is unsafe.” 7 (pp4–5)

The intertwining of study design and statistical analysis may have been caused (unintentionally) by R.A. Fisher, “… a genius who almost single-handedly created the foundations for modern statistical science.” 8 Most research did not involve statistics until Fisher invented the concepts and procedures of ANOVA (in 1921) 9 , 10 and experimental design (in 1935). 11 His books became standard references for scientists in many disciplines. As a result, many ANOVA books were titled Experimental Design (see, for example, Edwards 12 ), and ANOVA courses taught in psychology and education departments included the words experimental design in their course titles.

Before the widespread use of computers to analyze data, designs were much simpler, and often there was little difference between study design and statistical analysis. So combining the 2 elements did not cause serious problems. This is no longer true, however, for 3 reasons: (1) Research studies are becoming more complex, with multiple independent and dependent variables. The procedures sections of these complex studies can be difficult to understand if your only reference point is the statistical analysis and design. (2) Dependent variables are frequently measured at different times. (3) How the data were collected is often not directly correlated with the statistical design.

For example, assume the goal is to determine the strength gain in novice and experienced athletes as a result of 3 strength training programs. Rate of change in strength is not a measurable variable; rather, it is calculated from strength measurements taken at various time intervals during the training. So the study design would be a 2 × 2 × 3 factorial with independent variables of time (pretest or posttest), experience (novice or advanced), and training (isokinetic, isotonic, or isometric) and a dependent variable of strength. The statistical design , however, would be a 2 × 3 factorial with independent variables of experience (novice or advanced) and training (isokinetic, isotonic, or isometric) and a dependent variable of strength gain. Note that data were collected according to a 3-factor design but were analyzed according to a 2-factor design and that the dependent variables were different. So a single design statement, usually a statistical design statement, would not communicate which data were collected or how. Readers would be left to figure out on their own how the data were collected.

MULTIVARIATE RESEARCH AND THE NEED FOR STUDY DESIGNS

With the advent of electronic data gathering and computerized data handling and analysis, research projects have increased in complexity. Many projects involve multiple dependent variables measured at different times, and, therefore, multiple design statements may be needed for both data collection and statistical analysis. Consider, for example, a study of the effects of heat and cold on neural inhibition. The variables of H max and M max are measured 3 times each: before, immediately after, and 30 minutes after a 20-minute treatment with heat or cold. Muscle temperature might be measured each minute before, during, and after the treatment. Although the minute-by-minute data are important for graphing temperature fluctuations during the procedure, only 3 temperatures (time 0, time 20, and time 50) are used for statistical analysis. A single dependent variable H max :M max ratio is computed to illustrate neural inhibition. Again, a single statistical design statement would tell little about how the data were obtained. And in this example, separate design statements would be needed for temperature measurement and H max :M max measurements.

As stated earlier, drawing conclusions from the data depends more on how the data were measured than on how they were analyzed. 3 , 6 , 7 , 13 So a single study design statement (or multiple such statements) at the beginning of the “Methods” section acts as a road map to the study and, thus, increases scientists' and readers' comprehension of how the experiment was conducted (ie, how the data were collected). Appropriate study design statements also increase the accuracy of conclusions drawn from the study.

CONCLUSIONS

The goal of scientific writing, or any writing, for that matter, is to communicate information. Including 2 design statements or subsections in scientific papers—one to explain how the data were collected and another to explain how they were statistically analyzed—will improve the clarity of communication and bring praise from readers. To summarize:

• Purge from your thoughts and vocabulary the idea that experimental design and statistical design are synonymous.
• Study or experimental design plays a much broader role than simply defining and directing the statistical analysis of an experiment.
• A properly written study design serves as a road map to the “Methods” section of an experiment and, therefore, improves communication with the reader.
• Study design should include a description of the type of design used, each factor (and each level) involved in the experiment, and the time at which each measurement was made.
• Clarify when the variables involved in data collection and data analysis are different, such as when data analysis involves only a subset of a collected variable or a resultant variable from the mathematical manipulation of 2 or more collected variables.

Acknowledgments

Thanks to Thomas A. Cappaert, PhD, ATC, CSCS, CSE, for suggesting the link between R.A. Fisher and the melding of the concepts of research design and statistics.

Summer is here, and so is the sale. Get a yearly plan with up to 65% off today! 🌴🌞

• Form Builder
• Survey Maker
• AI Form Generator
• AI Survey Tool
• AI Quiz Maker
• Store Builder
• WordPress Plugin

HubSpot CRM

Google Sheets

Google Analytics

Microsoft Excel

• Popular Forms
• Job Application Form Template
• Rental Application Form Template
• Hotel Accommodation Form Template
• Online Registration Form Template
• Employment Application Form Template
• Application Forms
• Booking Forms
• Consent Forms
• Contact Forms
• Donation Forms
• Customer Satisfaction Surveys
• Employee Satisfaction Surveys
• Evaluation Surveys
• Feedback Surveys
• Market Research Surveys
• Personality Quiz Template
• Geography Quiz Template
• Math Quiz Template
• Science Quiz Template
• Vocabulary Quiz Template

Try without registration Quick Start

Read engaging stories, how-to guides, learn about forms.app features.

Inspirational ready-to-use templates for getting started fast and powerful.

Spot-on guides on how to use forms.app and make the most out of it.

See the technical measures we take and learn how we keep your data safe and secure.

• Integrations
• Help Center
• Sign In Sign Up Free
• What is experimental research: Definition, types & examples

Defne Çobanoğlu

Life and its secrets can only be proven right or wrong with experimentation. You can speculate and theorize all you wish, but as William Blake once said, “ The true method of knowledge is experiment. ”

It may be a long process and time-consuming, but it is rewarding like no other. And there are multiple ways and methods of experimentation that can help shed light on matters. In this article, we explained the definition, types of experimental research, and some experimental research examples . Let us get started with the definition!

• What is experimental research?

Experimental research is the process of carrying out a study conducted with a scientific approach using two or more variables. In other words, it is when you gather two or more variables and compare and test them in controlled environments. 

With experimental research, researchers can also collect detailed information about the participants by doing pre-tests and post-tests to learn even more information about the process. With the result of this type of study, the researcher can make conscious decisions. 

The more control the researcher has over the internal and extraneous variables, the better it is for the results. There may be different circumstances when a balanced experiment is not possible to conduct. That is why are are different research designs to accommodate the needs of researchers.

• 3 Types of experimental research designs

There is more than one dividing point in experimental research designs that differentiates them from one another. These differences are about whether or not there are pre-tests or post-tests done and how the participants are divided into groups. These differences decide which experimental research design is used.

Types of experimental research designs

1 - Pre-experimental design

This is the most basic method of experimental study. The researcher doing pre-experimental research evaluates a group of dependent variables after changing the independent variables . The results of this scientific method are not satisfactory, and future studies are planned accordingly. The pre-experimental research can be divided into three types:

A. One shot case study research design

Only one variable is considered in this one-shot case study design. This research method is conducted in the post-test part of a study, and the aim is to observe the changes in the effect of the independent variable.

B. One group pre-test post-test research design

In this type of research, a single group is given a pre-test before a study is conducted and a post-test after the study is conducted. The aim of this one-group pre-test post-test research design is to combine and compare the data collected during these tests. 

C. Static-group comparison

In a static group comparison, 2 or more groups are included in a study where only a group of participants is subjected to a new treatment and the other group of participants is held static. After the study is done, both groups do a post-test evaluation, and the changes are seen as results.

2 - Quasi-experimental design

This research type is quite similar to the experimental design; however, it changes in a few aspects. Quasi-experimental research is done when experimentation is needed for accurate data, but it is not possible to do one because of some limitations. Because you can not deliberately deprive someone of medical treatment or give someone harm, some experiments are ethically impossible. In this experimentation method, the researcher can only manipulate some variables. There are three types of quasi-experimental design:

A. Nonequivalent group designs

A nonequivalent group design is used when participants can not be divided equally and randomly for ethical reasons. Because of this, different variables will be more than one, unlike true experimental research.

B. Regression discontinuity

In this type of research design, the researcher does not divide a group into two to make a study, instead, they make use of a natural threshold or pre-existing dividing point. Only participants below or above the threshold get the treatment, and as the divide is minimal, the difference would be minimal as well.

C. Natural Experiments

In natural experiments, random or irregular assignment of patients makes up control and study groups. And they exist in natural scenarios. Because of this reason, they do not qualify as true experiments as they are based on observation.

3 - True experimental design

In true experimental research, the variables, groups, and settings should be identical to the textbook definition. Grouping of the participant are divided randomly, and controlled variables are chosen carefully. Every aspect of a true experiment should be carefully designed and acted out. And only the results of a true experiment can really be fully accurate . A true experimental design can be divided into 3 parts:

A. Post-test only control group design

In this experimental design, the participants are divided into two groups randomly. They are called experimental and control groups. Only the experimental group gets the treatment, while the other one does not. After the experiment and observation, both groups are given a post-test, and a conclusion is drawn from the results.

B. Pre-test post-test control group

In this method, the participants are divided into two groups once again. Also, only the experimental group gets the treatment. And this time, they are given both pre-tests and post-tests with multiple research methods. Thanks to these multiple tests, the researchers can make sure the changes in the experimental group are directly related to the treatment.

C. Solomon four-group design

This is the most comprehensive method of experimentation. The participants are randomly divided into 4 groups. These four groups include all possible permutations by including both control and non-control groups and post-test or pre-test and post-test control groups. This method enhances the quality of the data.

• Advantages and disadvantages of experimental research

Just as with any other study, experimental research also has its positive and negative sides. It is up to the researchers to be mindful of these facts before starting their studies. Let us see some advantages and disadvantages of experimental research:

Advantages of experimental research:

• All the variables are in the researchers’ control, and that means the researcher can influence the experiment according to the research question’s requirements.
• As you can easily control the variables in the experiment, you can specify the results as much as possible.
• The results of the study identify a cause-and-effect relation .
• The results can be as specific as the researcher wants.
• The result of an experimental design opens the doors for future related studies.

Disadvantages of experimental research:

• Completing an experiment may take years and even decades, so the results will not be as immediate as some of the other research types.
• As it involves many steps, participants, and researchers, it may be too expensive for some groups.
• The possibility of researchers making mistakes and having a bias is high. It is important to stay impartial
• Human behavior and responses can be difficult to measure unless it is specifically experimental research in psychology.
• Examples of experimental research

When one does experimental research, that experiment can be about anything. As the variables and environments can be controlled by the researcher, it is possible to have experiments about pretty much any subject. It is especially crucial that it gives critical insight into the cause-and-effect relationships of various elements. Now let us see some important examples of experimental research:

An example of experimental research in science:

When scientists make new medicines or come up with a new type of treatment, they have to test those thoroughly to make sure the results will be unanimous and effective for every individual. In order to make sure of this, they can test the medicine on different people or creatures in different dosages and in different frequencies. They can double-check all the results and have crystal clear results.

An example of experimental research in marketing:

The ideal goal of a marketing product, advertisement, or campaign is to attract attention and create positive emotions in the target audience. Marketers can focus on different elements in different campaigns, change the packaging/outline, and have a different approach. Only then can they be sure about the effectiveness of their approaches. Some methods they can work with are A/B testing, online surveys , or focus groups .

• Frequently asked questions about experimental research

Is experimental research qualitative or quantitative?

Experimental research can be both qualitative and quantitative according to the nature of the study. Experimental research is quantitative when it provides numerical and provable data. The experiment is qualitative when it provides researchers with participants' experiences, attitudes, or the context in which the experiment is conducted.

What is the difference between quasi-experimental research and experimental research?

In true experimental research, the participants are divided into groups randomly and evenly so as to have an equal distinction. However, in quasi-experimental research, the participants can not be divided equally for ethical or practical reasons. They are chosen non-randomly or by using a pre-existing threshold.

• Wrapping it up

The experimentation process can be long and time-consuming but highly rewarding as it provides valuable as well as both qualitative and quantitative data. It is a valuable part of research methods and gives insight into the subjects to let people make conscious decisions.

In this article, we have gathered experimental research definition, experimental research types, examples, and pros & cons to work as a guide for your next study. You can also make a successful experiment using pre-test and post-test methods and analyze the findings. For further information on different research types and for all your research information, do not forget to visit our other articles!

Defne is a content writer at forms.app. She is also a translator specializing in literary translation. Defne loves reading, writing, and translating professionally and as a hobby. Her expertise lies in survey research, research methodologies, content writing, and translation.

• Form Features
• Data Collection

Table of Contents

Related posts.

Patient satisfaction surveys: 15+ Question examples & free template

Işılay Kırbaş

20 Excellent persona interview questions (+Free example)

Şeyma Beyazçiçek

The 5 Shopify alternatives to start an e-commerce business

• Foundations
• Write Paper

Search form

• Experiments
• Anthropology
• Self-Esteem
• Social Anxiety

Experimental Research

Experimental research is commonly used in sciences such as sociology and psychology, physics, chemistry, biology and medicine etc.

This article is a part of the guide:

• Pretest-Posttest
• Third Variable
• Research Bias
• Independent Variable
• Between Subjects

Browse Full Outline

• 1 Experimental Research
• 2.1 Independent Variable
• 2.2 Dependent Variable
• 2.3 Controlled Variables
• 2.4 Third Variable
• 3.1 Control Group
• 3.2 Research Bias
• 3.3.1 Placebo Effect
• 3.3.2 Double Blind Method
• 4.1 Randomized Controlled Trials
• 4.2 Pretest-Posttest
• 4.3 Solomon Four Group
• 4.4 Between Subjects
• 4.5 Within Subject
• 4.6 Repeated Measures
• 4.7 Counterbalanced Measures
• 4.8 Matched Subjects

It is a collection of research designs which use manipulation and controlled testing to understand causal processes. Generally, one or more variables are manipulated to determine their effect on a dependent variable.

The experimental method is a systematic and scientific approach to research in which the researcher manipulates one or more variables, and controls and measures any change in other variables.

Experimental Research is often used where:

• There is time priority in a causal relationship ( cause precedes effect )
• There is consistency in a causal relationship (a cause will always lead to the same effect)
• The magnitude of the correlation is great.

(Reference: en.wikipedia.org)

The word experimental research has a range of definitions. In the strict sense, experimental research is what we call a true experiment .

This is an experiment where the researcher manipulates one variable, and control/randomizes the rest of the variables. It has a control group , the subjects have been randomly assigned between the groups, and the researcher only tests one effect at a time. It is also important to know what variable(s) you want to test and measure.

A very wide definition of experimental research, or a quasi experiment , is research where the scientist actively influences something to observe the consequences. Most experiments tend to fall in between the strict and the wide definition.

A rule of thumb is that physical sciences, such as physics, chemistry and geology tend to define experiments more narrowly than social sciences, such as sociology and psychology, which conduct experiments closer to the wider definition.

Aims of Experimental Research

Experiments are conducted to be able to predict phenomenons. Typically, an experiment is constructed to be able to explain some kind of causation . Experimental research is important to society - it helps us to improve our everyday lives.

Identifying the Research Problem

After deciding the topic of interest, the researcher tries to define the research problem . This helps the researcher to focus on a more narrow research area to be able to study it appropriately.  Defining the research problem helps you to formulate a  research hypothesis , which is tested against the  null hypothesis .

The research problem is often operationalizationed , to define how to measure the research problem. The results will depend on the exact measurements that the researcher chooses and may be operationalized differently in another study to test the main conclusions of the study.

An ad hoc analysis is a hypothesis invented after testing is done, to try to explain why the contrary evidence. A poor ad hoc analysis may be seen as the researcher's inability to accept that his/her hypothesis is wrong, while a great ad hoc analysis may lead to more testing and possibly a significant discovery.

Constructing the Experiment

There are various aspects to remember when constructing an experiment. Planning ahead ensures that the experiment is carried out properly and that the results reflect the real world, in the best possible way.

Sampling Groups to Study

Sampling groups correctly is especially important when we have more than one condition in the experiment. One sample group often serves as a control group , whilst others are tested under the experimental conditions.

Deciding the sample groups can be done in using many different sampling techniques. Population sampling may chosen by a number of methods, such as randomization , "quasi-randomization" and pairing.

Reducing sampling errors is vital for getting valid results from experiments. Researchers often adjust the sample size to minimize chances of random errors .

Here are some common sampling techniques :

• probability sampling
• non-probability sampling
• simple random sampling
• convenience sampling
• stratified sampling
• systematic sampling
• cluster sampling
• sequential sampling
• disproportional sampling
• judgmental sampling
• snowball sampling
• quota sampling

Creating the Design

The research design is chosen based on a range of factors. Important factors when choosing the design are feasibility, time, cost, ethics, measurement problems and what you would like to test. The design of the experiment is critical for the validity of the results.

Typical Designs and Features in Experimental Design

• Pretest-Posttest Design Check whether the groups are different before the manipulation starts and the effect of the manipulation. Pretests sometimes influence the effect.
• Control Group Control groups are designed to measure research bias and measurement effects, such as the Hawthorne Effect or the Placebo Effect . A control group is a group not receiving the same manipulation as the experimental group. Experiments frequently have 2 conditions, but rarely more than 3 conditions at the same time.
• Randomized Controlled Trials Randomized Sampling, comparison between an Experimental Group and a Control Group and strict control/randomization of all other variables
• Solomon Four-Group Design With two control groups and two experimental groups. Half the groups have a pretest and half do not have a pretest. This to test both the effect itself and the effect of the pretest.
• Between Subjects Design Grouping Participants to Different Conditions
• Within Subject Design Participants Take Part in the Different Conditions - See also: Repeated Measures Design
• Counterbalanced Measures Design Testing the effect of the order of treatments when no control group is available/ethical
• Matched Subjects Design Matching Participants to Create Similar Experimental- and Control-Groups
• Double-Blind Experiment Neither the researcher, nor the participants, know which is the control group. The results can be affected if the researcher or participants know this.
• Bayesian Probability Using bayesian probability to "interact" with participants is a more "advanced" experimental design. It can be used for settings were there are many variables which are hard to isolate. The researcher starts with a set of initial beliefs, and tries to adjust them to how participants have responded

Pilot Study

It may be wise to first conduct a pilot-study or two before you do the real experiment. This ensures that the experiment measures what it should, and that everything is set up right.

Minor errors, which could potentially destroy the experiment, are often found during this process. With a pilot study, you can get information about errors and problems, and improve the design, before putting a lot of effort into the real experiment.

If the experiments involve humans, a common strategy is to first have a pilot study with someone involved in the research, but not too closely, and then arrange a pilot with a person who resembles the subject(s) . Those two different pilots are likely to give the researcher good information about any problems in the experiment.

Conducting the Experiment

An experiment is typically carried out by manipulating a variable, called the independent variable , affecting the experimental group. The effect that the researcher is interested in, the dependent variable(s) , is measured.

Identifying and controlling non-experimental factors which the researcher does not want to influence the effects, is crucial to drawing a valid conclusion. This is often done by controlling variables , if possible, or randomizing variables to minimize effects that can be traced back to third variables . Researchers only want to measure the effect of the independent variable(s) when conducting an experiment , allowing them to conclude that this was the reason for the effect.

Analysis and Conclusions

In quantitative research , the amount of data measured can be enormous. Data not prepared to be analyzed is called "raw data". The raw data is often summarized as something called "output data", which typically consists of one line per subject (or item). A cell of the output data is, for example, an average of an effect in many trials for a subject. The output data is used for statistical analysis, e.g. significance tests, to see if there really is an effect.

The aim of an analysis is to draw a conclusion , together with other observations. The researcher might generalize the results to a wider phenomenon, if there is no indication of confounding variables "polluting" the results.

If the researcher suspects that the effect stems from a different variable than the independent variable, further investigation is needed to gauge the validity of the results. An experiment is often conducted because the scientist wants to know if the independent variable is having any effect upon the dependent variable. Variables correlating are not proof that there is causation .

Experiments are more often of quantitative nature than qualitative nature, although it happens.

Examples of Experiments

This website contains many examples of experiments. Some are not true experiments , but involve some kind of manipulation to investigate a phenomenon. Others fulfill most or all criteria of true experiments.

Here are some examples of scientific experiments:

Social Psychology

• Stanley Milgram Experiment - Will people obey orders, even if clearly dangerous?
• Asch Experiment - Will people conform to group behavior?
• Stanford Prison Experiment - How do people react to roles? Will you behave differently?
• Good Samaritan Experiment - Would You Help a Stranger? - Explaining Helping Behavior
• Law Of Segregation - The Mendel Pea Plant Experiment
• Transforming Principle - Griffith's Experiment about Genetics
• Ben Franklin Kite Experiment - Struck by Lightning
• J J Thomson Cathode Ray Experiment
• Psychology 101
• Flags and Countries
• Capitals and Countries

Oskar Blakstad (Jul 10, 2008). Experimental Research. Retrieved Aug 04, 2024 from Explorable.com: https://explorable.com/experimental-research

You Are Allowed To Copy The Text

The text in this article is licensed under the Creative Commons-License Attribution 4.0 International (CC BY 4.0) .

This means you're free to copy, share and adapt any parts (or all) of the text in the article, as long as you give appropriate credit and provide a link/reference to this page.

That is it. You don't need our permission to copy the article; just include a link/reference back to this page. You can use it freely (with some kind of link), and we're also okay with people reprinting in publications like books, blogs, newsletters, course-material, papers, wikipedia and presentations (with clear attribution).

Want to stay up to date? Follow us!

Get all these articles in 1 guide.

Want the full version to study at home, take to school or just scribble on?

Whether you are an academic novice, or you simply want to brush up your skills, this book will take your academic writing skills to the next level.

Download electronic versions: - Epub for mobiles and tablets - For Kindle here - For iBooks here - PDF version here

Save this course for later

Don't have time for it all now? No problem, save it as a course and come back to it later.

Footer bottom

• Privacy Policy

• Subscribe to our RSS Feed
• Like us on Facebook
• Follow us on Twitter

• Request Info
• Pirate Port

Research Methods

• Research Process
• Research Design & Method

Qualitative vs. Quantiative

Correlational vs. experimental, empirical vs. non-empirical.

• Resources for Research
• Survey Research
• Survey & Interview Data Analysis
• Ethical Considerations in Research
• Citation Resources This link opens in a new window

Locate Subject Librarians

• For help researching in your subject area, contact your Subject Librarian

Qualitative Research gathers data about lived experiences, emotions or behaviors, and the meanings individuals attach to them. It helps researchers gain a better understanding of complex concepts, social interactions or cultural phenomena. This type of research explores how or why things have occurred, interprets events and describes actions.

Quantitative Research gathers numerical data which can be ranked, measured or categorized through statistical analysis. It assists with uncovering patterns or relationships, and making generalizations. This type of research is useful for finding out how many, how much, how often, or to what extent.

Correlational Research cannot determine causal relationships. Instead they examine relationships between variables.

Experimental Research can establish causal relationship and variables can be manipulated.

Empirical Studies are based on evidence. The data is collected through experimentation or observation.

Non-empirical Studies focus on theories, methods, and their implications for research.

• Empirical and Non-Empirical Methods
• << Previous: Research Design & Method
• Next: Resources for Research >>
• Last Updated: Aug 2, 2024 11:41 AM
• URL: https://libguides.whitworth.edu/c.php?g=1411342

• How it works

A Complete Guide to Experimental Research

Published by Carmen Troy at August 14th, 2021 , Revised On August 25, 2023

A Quick Guide to Experimental Research

Experimental research refers to the experiments conducted in the laboratory or observation under controlled conditions. Researchers try to find out the cause-and-effect relationship between two or more variables. 

The subjects/participants in the experiment are selected and observed. They receive treatments such as changes in room temperature, diet, atmosphere, or given a new drug to observe the changes. Experiments can vary from personal and informal natural comparisons. It includes three  types of variables ;

• Independent variable
• Dependent variable
• Controlled variable

Before conducting experimental research, you need to have a clear understanding of the experimental design. A true experimental design includes  identifying a problem , formulating a  hypothesis , determining the number of variables, selecting and assigning the participants,  types of research designs , meeting ethical values, etc.

There are many  types of research  methods that can be classified based on:

• The nature of the problem to be studied
• Number of participants (individual or groups)
• Number of groups involved (Single group or multiple groups)
• Types of data collection methods (Qualitative/Quantitative/Mixed methods)
• Number of variables (single independent variable/ factorial two independent variables)
• The experimental design

Types of Experimental Research

Laboratory Experiment  

It is also called experimental research. This type of research is conducted in the laboratory. A researcher can manipulate and control the variables of the experiment.

Example: Milgram’s experiment on obedience.

Field Experiment

Field experiments are conducted in the participants’ open field and the environment by incorporating a few artificial changes. Researchers do not have control over variables under measurement. Participants know that they are taking part in the experiment.

Natural Experiments

The experiment is conducted in the natural environment of the participants. The participants are generally not informed about the experiment being conducted on them.

Examples: Estimating the health condition of the population. Did the increase in tobacco prices decrease the sale of tobacco? Did the usage of helmets decrease the number of head injuries of the bikers?

Quasi-Experiments

A quasi-experiment is an experiment that takes advantage of natural occurrences. Researchers cannot assign random participants to groups.

Example: Comparing the academic performance of the two schools.

Does your Research Methodology Have the Following?

• Great Research/Sources
• Perfect Language
• Accurate Sources

If not, we can help. Our panel of experts makes sure to keep the 3 pillars of Research Methodology strong.

How to Conduct Experimental Research?

Step 1. identify and define the problem.

You need to identify a problem as per your field of study and describe your  research question .

Example: You want to know about the effects of social media on the behavior of youngsters. It would help if you found out how much time students spend on the internet daily.

Example: You want to find out the adverse effects of junk food on human health. It would help if you found out how junk food frequent consumption can affect an individual’s health.

Step 2. Determine the Number of Levels of Variables

You need to determine the number of  variables . The independent variable is the predictor and manipulated by the researcher. At the same time, the dependent variable is the result of the independent variable.

In the first example, we predicted that increased social media usage negatively correlates with youngsters’ negative behaviour.

In the second example, we predicted the positive correlation between a balanced diet and a good healthy and negative relationship between junk food consumption and multiple health issues.

Step 3. Formulate the Hypothesis

One of the essential aspects of experimental research is formulating a hypothesis . A researcher studies the cause and effect between the independent and dependent variables and eliminates the confounding variables. A  null hypothesis is when there is no significant relationship between the dependent variable and the participants’ independent variables. A researcher aims to disprove the theory. H0 denotes it.  The  Alternative hypothesis  is the theory that a researcher seeks to prove.  H1or HA denotes it. 

Why should you use a Plagiarism Detector for your Paper?

It ensures:

• Original work
• Structure and Clarity
• Zero Spelling Errors
• No Punctuation Faults

Step 4. Selection and Assignment of the Subjects

It’s an essential feature that differentiates the experimental design from other research designs . You need to select the number of participants based on the requirements of your experiment. Then the participants are assigned to the treatment group. There should be a control group without any treatment to study the outcomes without applying any changes compared to the experimental group.

Randomisation:  The participants are selected randomly and assigned to the experimental group. It is known as probability sampling. If the selection is not random, it’s considered non-probability sampling.

Stratified sampling : It’s a type of random selection of the participants by dividing them into strata and randomly selecting them from each level. 

Matching:   Even though participants are selected randomly, they can be assigned to the various comparison groups. Another procedure for selecting the participants is ‘matching.’ The participants are selected from the controlled group to match the experimental groups’ participants in all aspects based on the dependent variables.  

What is Replicability?

When a researcher uses the same methodology  and subject groups to carry out the experiments, it’s called ‘replicability.’ The  results will be similar each time. Researchers usually replicate their own work to strengthen external validity.

Step 5. Select a Research Design

You need to select a  research design  according to the requirements of your experiment. There are many types of experimental designs as follows.

Step 6. Meet Ethical and Legal Requirements

• Participants of the research should not be harmed.
• The dignity and confidentiality of the research should be maintained.
• The consent of the participants should be taken before experimenting.
• The privacy of the participants should be ensured.
• Research data should remain confidential.
• The anonymity of the participants should be ensured.
• The rules and objectives of the experiments should be followed strictly.
• Any wrong information or data should be avoided.

Tips for Meeting the Ethical Considerations

To meet the ethical considerations, you need to ensure that.

• Participants have the right to withdraw from the experiment.
• They should be aware of the required information about the experiment.
• It would help if you avoided offensive or unacceptable language while framing the questions of interviews, questionnaires, or Focus groups.
• You should ensure the privacy and anonymity of the participants.
• You should acknowledge the sources and authors in your dissertation using any referencing styles such as APA/MLA/Harvard referencing style.

Step 7. Collect and Analyse Data.

Collect the data  by using suitable data collection according to your experiment’s requirement, such as observations,  case studies ,  surveys ,  interviews , questionnaires, etc. Analyse the obtained information.

Step 8. Present and Conclude the Findings of the Study.

Write the report of your research. Present, conclude, and explain the outcomes of your study .  

Frequently Asked Questions

What is the first step in conducting an experimental research.

The first step in conducting experimental research is to define your research question or hypothesis. Clearly outline the purpose and expectations of your experiment to guide the entire research process.

You May Also Like

This introductory guide looks at what quantitative observation is in research, how it’s carried out, its purpose, and the methods involved.

This article provides the key advantages of primary research over secondary research so you can make an informed decision.

A dependent variable is one that completely depends on another variable, mostly the independent one.

USEFUL LINKS

LEARNING RESOURCES

COMPANY DETAILS

• How It Works

• Voxco Online
• Voxco Panel Management
• Voxco Panel Portal
• Voxco Audience
• Voxco Mobile Offline
• Voxco Dialer Cloud
• Voxco Dialer On-premise
• Voxco TCPA Connect
• Voxco Analytics
• Voxco Text & Sentiment Analysis

• 40+ question types
• Drag-and-drop interface
• Skip logic and branching
• Multi-lingual survey
• Text piping
• Question library
• CSS customization
• White-label surveys
• Customizable ‘Thank You’ page
• Customizable survey theme
• Reminder send-outs
• Survey rewards
• Social media
• Website surveys
• Correlation analysis
• Cross-tabulation analysis
• Trend analysis
• Real-time dashboard
• Customizable report
• Email address validation
• Recaptcha validation
• SSL security

Take a peek at our powerful survey features to design surveys that scale discoveries.

Download feature sheet.

• Hospitality
• Academic Research
• Customer Experience
• Employee Experience
• Product Experience
• Market Research
• Social Research
• Data Analysis

Explore Voxco 

Need to map Voxco’s features & offerings? We can help!

Watch a Demo 

Download Brochures 

Get a Quote

• NPS Calculator
• CES Calculator
• A/B Testing Calculator
• Margin of Error Calculator
• Sample Size Calculator
• CX Strategy & Management Hub
• Market Research Hub
• Patient Experience Hub
• Employee Experience Hub
• NPS Knowledge Hub
• Market Research Guide
• Customer Experience Guide
• The Voxco Guide to Customer Experience
• Survey Research Guides
• Survey Template Library
• Webinars and Events
• Feature Sheets
• Try a sample survey
• Professional Services

Get exclusive insights into research trends and best practices from top experts! Access Voxco’s ‘State of Research Report 2024 edition’ .

We’ve been avid users of the Voxco platform now for over 20 years. It gives us the flexibility to routinely enhance our survey toolkit and provides our clients with a more robust dataset and story to tell their clients.

VP Innovation & Strategic Partnerships, The Logit Group

• Client Stories
• Voxco Reviews
• Why Voxco Research?
• Careers at Voxco
• Vulnerabilities and Ethical Hacking

Explore Regional Offices

• Survey Software The world’s leading omnichannel survey software
• Online Survey Tools Create sophisticated surveys with ease.
• Mobile Offline Conduct efficient field surveys.
• Text Analysis
• Close The Loop
• Automated Translations
• NPS Dashboard
• CATI Manage high volume phone surveys efficiently
• Cloud/On-premise Dialer TCPA compliant Cloud on-premise dialer
• IVR Survey Software Boost productivity with automated call workflows.
• Analytics Analyze survey data with visual dashboards
• Panel Manager Nurture a loyal community of respondents.
• Survey Portal Best-in-class user friendly survey portal.
• Voxco Audience Conduct targeted sample research in hours.
• Predictive Analytics
• Customer 360
• Customer Loyalty
• Fraud & Risk Management
• AI/ML Enablement Services
• Credit Underwriting

Find the best survey software for you! (Along with a checklist to compare platforms)

Get Buyer’s Guide

• 100+ question types
• SMS surveys
• Financial Services
• Banking & Financial Services
• Retail Solution
• Risk Management
• Customer Lifecycle Solutions
• Net Promoter Score
• Customer Behaviour Analytics
• Customer Segmentation
• Data Unification

Explore Voxco 

Watch a Demo 

Download Brochures 

• CX Strategy & Management Hub
• Professional services
• Blogs & White papers
• Case Studies

Find the best customer experience platform

Uncover customer pain points, analyze feedback and run successful CX programs with the best CX platform for your team.

Get the Guide Now

VP Innovation & Strategic Partnerships, The Logit Group

• Why Voxco Intelligence?
• Our clients
• Client stories
• Featuresheets

Definition, Examples and Types of Experimental Research Designs

SHARE THE ARTICLE ON

What is Experimental Research ?

Experimental research is a scientific methodology of understanding relationships between two or more variables. These sets consist of independent and dependent variables which are experimentally tested to deduce a correlation between such variables in terms of the nature and strength of such relation. Such assessment helps in deriving a cause and effect relationship and is even used for the purpose of hypothesis testing.

In such a mechanism , independent variables involved are adjusted to discover their impact on the dependent variables. The degree to which a change in the independent variables influences dependent variables is the basis of gauging the degree of strength. Such variations are recorded over a specific period of time to ensure that the conclusions drawn about the relationship are substantive and reliable enough to assist intelligent decision making.

Experimental research deals with quantitative data and its statistical analysis which makes the study extremely useful and accurate. It finds its usability in fields of psychology , social sciences , physical evaluation and academics and are time bound studies usually used for verification purposes.

Types of Experimental Research designs

1) Pre-experimental research design :

This is an observational research mechanism used to evaluate changes in a group or various groups of dependent variables after changing the independent variable values. This is the simplest form of experimental research used to assess the need for further inspection, if satisfactory results are not derived from the observations registered.

This can further be subdivided as :

• One-shot Case Study Research Design: A post-test study relying only on a single set of variables for observational purposes.
• One-group Pretest-posttest Research Design : This is a combination of pre and post tests that studies a single set of variables before and after the method of testing has been implemented.
• Static-group Comparison: The total groups of variables gets divided into 2 sub-groups, one subjected to the testing while the other group remains as it as . Observations at the end of the testing reveal the contrast between the tested and the non-tested panel.

2) True experimental research :

This is a statistical approach to establish a cause and effect relationship within a variable set. The quantitative approach of this study makes it highly accurate. The assignment of test units and treatments takes place in a randomized manner.

Apart from this , it uses the availability of a control group along with an independent variable that can be manipulated to obtain the required results.

3) Quasi- experimental research design :

Quasi-experimental research design is a partial representation of true experimental research such that it seeks to establish a cause and effect relationship by manipulating an independent variable, the only difference being that it does not adhere to random distribution of participants into groups.

Thus , Quasi- Experimental research design is only applied to those situations where there is no relevance or possibility for random distribution.

Some examples of Experimental Research design

Employee recruitment and screening 

The recruitment of an employee to an organization requires the employee to go through a rigorous selection procedure that filters the highly suited individuals for the job from the rest of the lot. A screening process is conducted that tests the skills , qualification , experience and knowledge of the applicants before going ahead with selecting the required number of people. The selected individuals are then recruited and trained with respect to the work to be done. Following this training , these individuals are then observed for a specific frame. At the end of this time period , employee appraisals take place which reviews the performance of the employee to identify the need for any improvement or if the employee is capable of handling extra work while maintaining the same level of performance and consistency levels.

This is a simple example of one group pretest posttest research design that assists the creation of a progressive work environment that provides the room for employees to grow along with pushing the organization towards achieving objectives in an efficient manner.

Impact of online tuitions

A group of students belonging to the same class and scoring the same grades in their first term exams are selected to try out a new e-tuition app as against their existing tuition classes. This sample of students are divided into two groups : one that switches to the online educational tuition app while the other group continues to attend their existing tuition classes. This study continues till the next examination cycle , as it observes the differences in the students ability to learn , grasp concepts and their general attitude towards the process of online learning . At the end of the study , the students belonging to both the groups give their term end examinations and the differences between the performance of the students are noted to contrast the teaching methods and effectiveness of online learning vs e-learning.

Such a study is an example of static group comparison that helps in comparing , analysing and establishing one of the alternatives as a viable choice under the current scenario.

Disadvantages of Experimental Research

• The chances of error and bias being involved in experimental research are very high. The process of controlling independent variables to study changes in the dependent variables is highly prone to human error. Further , the results can even be skewed if the values are manipulated by the researcher.
• It is a highly expensive, time consuming and cumbersome process to carry out a thorough experimental research procedure.
• The observational nature of the pre-test experimental research study makes it a qualitative research mechanism that does not help in deriving substantive conclusions based on hard figures.
• It can produce artificial results . It is important to factor in all independent variables that bring out variation in the dependent variables . Failing to do this may not reflect the true picture with reference to the strength of the relationship between the variables in consideration.
• In certain situations, It is highly risky and can lead to ethical complications if treatment is not implemented carefully.

Methods of data collection

1) Surveys :

Surveys are the easiest and the most commonly used data collection mechanism. Surveys help in achieving the coverage of all relevant areas of interest by framing a questionnaire to be filled out by the targeted respondent. This can be done physically , however , the attractions offered by the online research software allow for advance designing , distribution, collection , reporting and analysis of the information gathered. This provides a viable alternative that offers enhanced research procedures to be conducted in a swift and efficient manner.

Care needs to be taken while designing the survey as well as selecting the limited number of respondents who will assist the surveying organizations in finding answers to their research questions to fuel intelligent decision making.

2) Observation :

This method of data collection involves keeping a check on the variables under study to monitor changes and observe behaviour. It takes a long period of observation to make significant conclusions. This method also largely relies on the observer’s judgement and so is highly subjective.

3) Simulation :

Simulation replicates real life processes and situations to understand variables under consideration. The reliability of such a method heavily depends upon the accuracy with which the simulation has been created. This method finds its applicability in fields such as operational research which seeks to break down the whole idea to study narrow concepts involved. Simulations are an effective choice where accessibility and implementation are not feasible.

4) Experiments :

Experiments are carried out in a controlled environment such as a lab where influencing factors can be controlled. This also circles around field experiments, numerical and AI studies. The usage of computerized software makes data handling and management an easy task.

Experiments assist a comprehensive overview of the variables under the scope of the study. They are statistically compatible and so deliver substantive results which are objective in nature.

Difference between experimental and non- experimental research

1) Experimental research focuses on understanding the nature of relationship between independent and dependent variables involved under a particular field of study. On the other hand , Non-experimental research is descriptive in nature and so , focuses on defining a process , situation or idea.

2) Experimental research provides the freedom to control external independent variables to decipher relationships, however , such a control mechanism is absent in Non- experimental research.

3) Experimental data does not make use of case studies and published works for establishing relationships while non-experimental research cannot be carried out using simulations.

4) Experimental research involves a scientific approach whereas such an approach is absent in non-experimental research due to the descriptive nature of the study.

  The 3 types of experimental designs are :

• Pre- experimental research 
• True experimental research 
• Quasi- experimental research 

The study of the impact of different educational levels , experience and additional skills on the nature of jobs , salaries and the type of work environment is a simple example that can be used to understand experimental research.

  Experimental research is a methodology used to gauge the nature of relationship between the variables in consideration.

Experimental designs are written in terms of the hypothesis that a study tries to prove or the variables the research tries to study.

Explore Voxco Survey Software

+ Omnichannel Survey Software 

+ Online Survey Software 

+ CATI Survey Software 

+ IVR Survey Software 

+ Market Research Tool

+ Customer Experience Tool 

+ Product Experience Software 

+ Enterprise Survey Software 

We use cookies in our website to give you the best browsing experience and to tailor advertising. By continuing to use our website, you give us consent to the use of cookies. Read More

Experimental Research: Meaning And Examples Of Experimental Research

Ever wondered why scientists across the world are being lauded for discovering the Covid-19 vaccine so early? It’s because every…

Ever wondered why scientists across the world are being lauded for discovering the Covid-19 vaccine so early? It’s because every government knows that vaccines are a result of experimental research design and it takes years of collected data to make one. It takes a lot of time to compare formulas and combinations with an array of possibilities across different age groups, genders and physical conditions. With their efficiency and meticulousness, scientists redefined the meaning of experimental research when they discovered a vaccine in less than a year.

What Is Experimental Research?

Characteristics of experimental research design, types of experimental research design, advantages and disadvantages of experimental research, examples of experimental research.

Experimental research is a scientific method of conducting research using two variables: independent and dependent. Independent variables can be manipulated to apply to dependent variables and the effect is measured. This measurement usually happens over a significant period of time to establish conditions and conclusions about the relationship between these two variables.

Experimental research is widely implemented in education, psychology, social sciences and physical sciences. Experimental research is based on observation, calculation, comparison and logic. Researchers collect quantitative data and perform statistical analyses of two sets of variables. This method collects necessary data to focus on facts and support sound decisions. It’s a helpful approach when time is a factor in establishing cause-and-effect relationships or when an invariable behavior is seen between the two.  

Now that we know the meaning of experimental research, let’s look at its characteristics, types and advantages.

The hypothesis is at the core of an experimental research design. Researchers propose a tentative answer after defining the problem and then test the hypothesis to either confirm or disregard it. Here are a few characteristics of experimental research:

• Dependent variables are manipulated or treated while independent variables are exerted on dependent variables as an experimental treatment. Extraneous variables are variables generated from other factors that can affect the experiment and contribute to change. Researchers have to exercise control to reduce the influence of these variables by randomization, making homogeneous groups and applying statistical analysis techniques.
• Researchers deliberately operate independent variables on the subject of the experiment. This is known as manipulation.
• Once a variable is manipulated, researchers observe the effect an independent variable has on a dependent variable. This is key for interpreting results.
• A researcher may want multiple comparisons between different groups with equivalent subjects. They may replicate the process by conducting sub-experiments within the framework of the experimental design.

Experimental research is equally effective in non-laboratory settings as it is in labs. It helps in predicting events in an experimental setting. It generalizes variable relationships so that they can be implemented outside the experiment and applied to a wider interest group.

The way a researcher assigns subjects to different groups determines the types of experimental research design .

Pre-experimental Research Design

In a pre-experimental research design, researchers observe a group or various groups to see the effect an independent variable has on the dependent variable to cause change. There is no control group as it is a simple form of experimental research . It’s further divided into three categories:

• A one-shot case study research design is a study where one dependent variable is considered. It’s a posttest study as it’s carried out after treating what presumably caused the change.
• One-group pretest-posttest design is a study that combines both pretest and posttest studies by testing a single group before and after administering the treatment.
• Static-group comparison involves studying two groups by subjecting one to treatment while the other remains static. After post-testing all groups the differences are observed.

This design is practical but lacks in certain areas of true experimental criteria.

True Experimental Research Design

This design depends on statistical analysis to approve or disregard a hypothesis. It’s an accurate design that can be conducted with or without a pretest on a minimum of two dependent variables assigned randomly. It is further classified into three types:

• The posttest-only control group design involves randomly selecting and assigning subjects to two groups: experimental and control. Only the experimental group is treated, while both groups are observed and post-tested to draw a conclusion from the difference between the groups.
• In a pretest-posttest control group design, two groups are randomly assigned subjects. Both groups are presented, the experimental group is treated and both groups are post-tested to measure how much change happened in each group.
• Solomon four-group design is a combination of the previous two methods. Subjects are randomly selected and assigned to four groups. Two groups are tested using each of the previous methods.

True experimental research design should have a variable to manipulate, a control group and random distribution.

With experimental research, we can test ideas in a controlled environment before marketing. It acts as the best method to test a theory as it can help in making predictions about a subject and drawing conclusions. Let’s look at some of the advantages that make experimental research useful:

• It allows researchers to have a stronghold over variables and collect desired results.
• Results are usually specific.
• The effectiveness of the research isn’t affected by the subject.
• Findings from the results usually apply to similar situations and ideas.
• Cause and effect of a hypothesis can be identified, which can be further analyzed for in-depth ideas.
• It’s the ideal starting point to collect data and lay a foundation for conducting further research and building more ideas.
• Medical researchers can develop medicines and vaccines to treat diseases by collecting samples from patients and testing them under multiple conditions.
• It can be used to improve the standard of academics across institutions by testing student knowledge and teaching methods before analyzing the result to implement programs.
• Social scientists often use experimental research design to study and test behavior in humans and animals.
• Software development and testing heavily depend on experimental research to test programs by letting subjects use a beta version and analyzing their feedback.

Even though it’s a scientific method, it has a few drawbacks. Here are a few disadvantages of this research method:

• Human error is a concern because the method depends on controlling variables. Improper implementation nullifies the validity of the research and conclusion.
• Eliminating extraneous variables (real-life scenarios) produces inaccurate conclusions.
• The process is time-consuming and expensive
• In medical research, it can have ethical implications by affecting patients’ well-being.
• Results are not descriptive and subjects can contribute to response bias.

Experimental research design is a sophisticated method that investigates relationships or occurrences among people or phenomena under a controlled environment and identifies the conditions responsible for such relationships or occurrences

Experimental research can be used in any industry to anticipate responses, changes, causes and effects. Here are some examples of experimental research :

• This research method can be used to evaluate employees’ skills. Organizations ask candidates to take tests before filling a post. It is used to screen qualified candidates from a pool of applicants. This allows organizations to identify skills at the time of employment. After training employees on the job, organizations further evaluate them to test impact and improvement. This is a pretest-posttest control group research example where employees are ‘subjects’ and the training is ‘treatment’.
• Educational institutions follow the pre-experimental research design to administer exams and evaluate students at the end of a semester. Students are the dependent variables and lectures are independent. Since exams are conducted at the end and not the beginning of a semester, it’s easy to conclude that it’s a one-shot case study research.
• To evaluate the teaching methods of two teachers, they can be assigned two student groups. After teaching their respective groups on the same topic, a posttest can determine which group scored better and who is better at teaching. This method can have its drawbacks as certain human factors, such as attitudes of students and effectiveness to grasp a subject, may negatively influence results. 

Experimental research is considered a standard method that uses observations, simulations and surveys to collect data. One of its unique features is the ability to control extraneous variables and their effects. It’s a suitable method for those looking to examine the relationship between cause and effect in a field setting or in a laboratory. Although experimental research design is a scientific approach, research is not entirely a scientific process. As much as managers need to know what is experimental research , they have to apply the correct research method, depending on the aim of the study.

Harappa’s Thinking Critically program makes you more decisive and lets you think like a leader. It’s a growth-driven course for managers who want to devise and implement sound strategies, freshers looking to build a career and entrepreneurs who want to grow their business. Identify and avoid arguments, communicate decisions and rely on effective decision-making processes in uncertain times. This course teaches critical and clear thinking. It’s packed with problem-solving tools, highly impactful concepts and relatable content. Build an analytical mindset, develop your skills and reap the benefits of critical thinking with Harappa!

Explore Harappa Diaries to learn more about topics such as Main Objective Of Research , Definition Of Qualitative Research , Examples Of Experiential Learning and Collaborative Learning Strategies to upgrade your knowledge and skills.

• Privacy Policy

Home » Quantitative Research – Methods, Types and Analysis

Quantitative Research – Methods, Types and Analysis

Table of Contents

Quantitative Research

Quantitative research is a type of research that collects and analyzes numerical data to test hypotheses and answer research questions . This research typically involves a large sample size and uses statistical analysis to make inferences about a population based on the data collected. It often involves the use of surveys, experiments, or other structured data collection methods to gather quantitative data.

Quantitative Research Methods

Quantitative Research Methods are as follows:

Descriptive Research Design

Descriptive research design is used to describe the characteristics of a population or phenomenon being studied. This research method is used to answer the questions of what, where, when, and how. Descriptive research designs use a variety of methods such as observation, case studies, and surveys to collect data. The data is then analyzed using statistical tools to identify patterns and relationships.

Correlational Research Design

Correlational research design is used to investigate the relationship between two or more variables. Researchers use correlational research to determine whether a relationship exists between variables and to what extent they are related. This research method involves collecting data from a sample and analyzing it using statistical tools such as correlation coefficients.

Quasi-experimental Research Design

Quasi-experimental research design is used to investigate cause-and-effect relationships between variables. This research method is similar to experimental research design, but it lacks full control over the independent variable. Researchers use quasi-experimental research designs when it is not feasible or ethical to manipulate the independent variable.

Experimental Research Design

Experimental research design is used to investigate cause-and-effect relationships between variables. This research method involves manipulating the independent variable and observing the effects on the dependent variable. Researchers use experimental research designs to test hypotheses and establish cause-and-effect relationships.

Survey Research

Survey research involves collecting data from a sample of individuals using a standardized questionnaire. This research method is used to gather information on attitudes, beliefs, and behaviors of individuals. Researchers use survey research to collect data quickly and efficiently from a large sample size. Survey research can be conducted through various methods such as online, phone, mail, or in-person interviews.

Quantitative Research Analysis Methods

Here are some commonly used quantitative research analysis methods:

Statistical Analysis

Statistical analysis is the most common quantitative research analysis method. It involves using statistical tools and techniques to analyze the numerical data collected during the research process. Statistical analysis can be used to identify patterns, trends, and relationships between variables, and to test hypotheses and theories.

Regression Analysis

Regression analysis is a statistical technique used to analyze the relationship between one dependent variable and one or more independent variables. Researchers use regression analysis to identify and quantify the impact of independent variables on the dependent variable.

Factor Analysis

Factor analysis is a statistical technique used to identify underlying factors that explain the correlations among a set of variables. Researchers use factor analysis to reduce a large number of variables to a smaller set of factors that capture the most important information.

Structural Equation Modeling

Structural equation modeling is a statistical technique used to test complex relationships between variables. It involves specifying a model that includes both observed and unobserved variables, and then using statistical methods to test the fit of the model to the data.

Time Series Analysis

Time series analysis is a statistical technique used to analyze data that is collected over time. It involves identifying patterns and trends in the data, as well as any seasonal or cyclical variations.

Multilevel Modeling

Multilevel modeling is a statistical technique used to analyze data that is nested within multiple levels. For example, researchers might use multilevel modeling to analyze data that is collected from individuals who are nested within groups, such as students nested within schools.

Applications of Quantitative Research

Quantitative research has many applications across a wide range of fields. Here are some common examples:

• Market Research : Quantitative research is used extensively in market research to understand consumer behavior, preferences, and trends. Researchers use surveys, experiments, and other quantitative methods to collect data that can inform marketing strategies, product development, and pricing decisions.
• Health Research: Quantitative research is used in health research to study the effectiveness of medical treatments, identify risk factors for diseases, and track health outcomes over time. Researchers use statistical methods to analyze data from clinical trials, surveys, and other sources to inform medical practice and policy.
• Social Science Research: Quantitative research is used in social science research to study human behavior, attitudes, and social structures. Researchers use surveys, experiments, and other quantitative methods to collect data that can inform social policies, educational programs, and community interventions.
• Education Research: Quantitative research is used in education research to study the effectiveness of teaching methods, assess student learning outcomes, and identify factors that influence student success. Researchers use experimental and quasi-experimental designs, as well as surveys and other quantitative methods, to collect and analyze data.
• Environmental Research: Quantitative research is used in environmental research to study the impact of human activities on the environment, assess the effectiveness of conservation strategies, and identify ways to reduce environmental risks. Researchers use statistical methods to analyze data from field studies, experiments, and other sources.

Characteristics of Quantitative Research

Here are some key characteristics of quantitative research:

• Numerical data : Quantitative research involves collecting numerical data through standardized methods such as surveys, experiments, and observational studies. This data is analyzed using statistical methods to identify patterns and relationships.
• Large sample size: Quantitative research often involves collecting data from a large sample of individuals or groups in order to increase the reliability and generalizability of the findings.
• Objective approach: Quantitative research aims to be objective and impartial in its approach, focusing on the collection and analysis of data rather than personal beliefs, opinions, or experiences.
• Control over variables: Quantitative research often involves manipulating variables to test hypotheses and establish cause-and-effect relationships. Researchers aim to control for extraneous variables that may impact the results.
• Replicable : Quantitative research aims to be replicable, meaning that other researchers should be able to conduct similar studies and obtain similar results using the same methods.
• Statistical analysis: Quantitative research involves using statistical tools and techniques to analyze the numerical data collected during the research process. Statistical analysis allows researchers to identify patterns, trends, and relationships between variables, and to test hypotheses and theories.
• Generalizability: Quantitative research aims to produce findings that can be generalized to larger populations beyond the specific sample studied. This is achieved through the use of random sampling methods and statistical inference.

Examples of Quantitative Research

Here are some examples of quantitative research in different fields:

• Market Research: A company conducts a survey of 1000 consumers to determine their brand awareness and preferences. The data is analyzed using statistical methods to identify trends and patterns that can inform marketing strategies.
• Health Research : A researcher conducts a randomized controlled trial to test the effectiveness of a new drug for treating a particular medical condition. The study involves collecting data from a large sample of patients and analyzing the results using statistical methods.
• Social Science Research : A sociologist conducts a survey of 500 people to study attitudes toward immigration in a particular country. The data is analyzed using statistical methods to identify factors that influence these attitudes.
• Education Research: A researcher conducts an experiment to compare the effectiveness of two different teaching methods for improving student learning outcomes. The study involves randomly assigning students to different groups and collecting data on their performance on standardized tests.
• Environmental Research : A team of researchers conduct a study to investigate the impact of climate change on the distribution and abundance of a particular species of plant or animal. The study involves collecting data on environmental factors and population sizes over time and analyzing the results using statistical methods.
• Psychology : A researcher conducts a survey of 500 college students to investigate the relationship between social media use and mental health. The data is analyzed using statistical methods to identify correlations and potential causal relationships.
• Political Science: A team of researchers conducts a study to investigate voter behavior during an election. They use survey methods to collect data on voting patterns, demographics, and political attitudes, and analyze the results using statistical methods.

How to Conduct Quantitative Research

Here is a general overview of how to conduct quantitative research:

• Develop a research question: The first step in conducting quantitative research is to develop a clear and specific research question. This question should be based on a gap in existing knowledge, and should be answerable using quantitative methods.
• Develop a research design: Once you have a research question, you will need to develop a research design. This involves deciding on the appropriate methods to collect data, such as surveys, experiments, or observational studies. You will also need to determine the appropriate sample size, data collection instruments, and data analysis techniques.
• Collect data: The next step is to collect data. This may involve administering surveys or questionnaires, conducting experiments, or gathering data from existing sources. It is important to use standardized methods to ensure that the data is reliable and valid.
• Analyze data : Once the data has been collected, it is time to analyze it. This involves using statistical methods to identify patterns, trends, and relationships between variables. Common statistical techniques include correlation analysis, regression analysis, and hypothesis testing.
• Interpret results: After analyzing the data, you will need to interpret the results. This involves identifying the key findings, determining their significance, and drawing conclusions based on the data.
• Communicate findings: Finally, you will need to communicate your findings. This may involve writing a research report, presenting at a conference, or publishing in a peer-reviewed journal. It is important to clearly communicate the research question, methods, results, and conclusions to ensure that others can understand and replicate your research.

When to use Quantitative Research

Here are some situations when quantitative research can be appropriate:

• To test a hypothesis: Quantitative research is often used to test a hypothesis or a theory. It involves collecting numerical data and using statistical analysis to determine if the data supports or refutes the hypothesis.
• To generalize findings: If you want to generalize the findings of your study to a larger population, quantitative research can be useful. This is because it allows you to collect numerical data from a representative sample of the population and use statistical analysis to make inferences about the population as a whole.
• To measure relationships between variables: If you want to measure the relationship between two or more variables, such as the relationship between age and income, or between education level and job satisfaction, quantitative research can be useful. It allows you to collect numerical data on both variables and use statistical analysis to determine the strength and direction of the relationship.
• To identify patterns or trends: Quantitative research can be useful for identifying patterns or trends in data. For example, you can use quantitative research to identify trends in consumer behavior or to identify patterns in stock market data.
• To quantify attitudes or opinions : If you want to measure attitudes or opinions on a particular topic, quantitative research can be useful. It allows you to collect numerical data using surveys or questionnaires and analyze the data using statistical methods to determine the prevalence of certain attitudes or opinions.

Purpose of Quantitative Research

The purpose of quantitative research is to systematically investigate and measure the relationships between variables or phenomena using numerical data and statistical analysis. The main objectives of quantitative research include:

• Description : To provide a detailed and accurate description of a particular phenomenon or population.
• Explanation : To explain the reasons for the occurrence of a particular phenomenon, such as identifying the factors that influence a behavior or attitude.
• Prediction : To predict future trends or behaviors based on past patterns and relationships between variables.
• Control : To identify the best strategies for controlling or influencing a particular outcome or behavior.

Quantitative research is used in many different fields, including social sciences, business, engineering, and health sciences. It can be used to investigate a wide range of phenomena, from human behavior and attitudes to physical and biological processes. The purpose of quantitative research is to provide reliable and valid data that can be used to inform decision-making and improve understanding of the world around us.

Advantages of Quantitative Research

There are several advantages of quantitative research, including:

• Objectivity : Quantitative research is based on objective data and statistical analysis, which reduces the potential for bias or subjectivity in the research process.
• Reproducibility : Because quantitative research involves standardized methods and measurements, it is more likely to be reproducible and reliable.
• Generalizability : Quantitative research allows for generalizations to be made about a population based on a representative sample, which can inform decision-making and policy development.
• Precision : Quantitative research allows for precise measurement and analysis of data, which can provide a more accurate understanding of phenomena and relationships between variables.
• Efficiency : Quantitative research can be conducted relatively quickly and efficiently, especially when compared to qualitative research, which may involve lengthy data collection and analysis.
• Large sample sizes : Quantitative research can accommodate large sample sizes, which can increase the representativeness and generalizability of the results.

Limitations of Quantitative Research

There are several limitations of quantitative research, including:

• Limited understanding of context: Quantitative research typically focuses on numerical data and statistical analysis, which may not provide a comprehensive understanding of the context or underlying factors that influence a phenomenon.
• Simplification of complex phenomena: Quantitative research often involves simplifying complex phenomena into measurable variables, which may not capture the full complexity of the phenomenon being studied.
• Potential for researcher bias: Although quantitative research aims to be objective, there is still the potential for researcher bias in areas such as sampling, data collection, and data analysis.
• Limited ability to explore new ideas: Quantitative research is often based on pre-determined research questions and hypotheses, which may limit the ability to explore new ideas or unexpected findings.
• Limited ability to capture subjective experiences : Quantitative research is typically focused on objective data and may not capture the subjective experiences of individuals or groups being studied.
• Ethical concerns : Quantitative research may raise ethical concerns, such as invasion of privacy or the potential for harm to participants.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Observational Research – Methods and Guide

Survey Research – Types, Methods, Examples

Qualitative Research Methods

Experimental Design – Types, Methods, Guide

Transformative Design – Methods, Types, Guide

Research Methods – Types, Examples and Guide

Chapter 10 Experimental Research

Experimental research, often considered to be the “gold standard” in research designs, is one of the most rigorous of all research designs. In this design, one or more independent variables are manipulated by the researcher (as treatments), subjects are randomly assigned to different treatment levels (random assignment), and the results of the treatments on outcomes (dependent variables) are observed. The unique strength of experimental research is its internal validity (causality) due to its ability to link cause and effect through treatment manipulation, while controlling for the spurious effect of extraneous variable.

Experimental research is best suited for explanatory research (rather than for descriptive or exploratory research), where the goal of the study is to examine cause-effect relationships. It also works well for research that involves a relatively limited and well-defined set of independent variables that can either be manipulated or controlled. Experimental research can be conducted in laboratory or field settings. Laboratory experiments , conducted in laboratory (artificial) settings, tend to be high in internal validity, but this comes at the cost of low external validity (generalizability), because the artificial (laboratory) setting in which the study is conducted may not reflect the real world. Field experiments , conducted in field settings such as in a real organization, and high in both internal and external validity. But such experiments are relatively rare, because of the difficulties associated with manipulating treatments and controlling for extraneous effects in a field setting.

Experimental research can be grouped into two broad categories: true experimental designs and quasi-experimental designs. Both designs require treatment manipulation, but while true experiments also require random assignment, quasi-experiments do not. Sometimes, we also refer to non-experimental research, which is not really a research design, but an all-inclusive term that includes all types of research that do not employ treatment manipulation or random assignment, such as survey research, observational research, and correlational studies.

Basic Concepts

Treatment and control groups. In experimental research, some subjects are administered one or more experimental stimulus called a treatment (the treatment group ) while other subjects are not given such a stimulus (the control group ). The treatment may be considered successful if subjects in the treatment group rate more favorably on outcome variables than control group subjects. Multiple levels of experimental stimulus may be administered, in which case, there may be more than one treatment group. For example, in order to test the effects of a new drug intended to treat a certain medical condition like dementia, if a sample of dementia patients is randomly divided into three groups, with the first group receiving a high dosage of the drug, the second group receiving a low dosage, and the third group receives a placebo such as a sugar pill (control group), then the first two groups are experimental groups and the third group is a control group. After administering the drug for a period of time, if the condition of the experimental group subjects improved significantly more than the control group subjects, we can say that the drug is effective. We can also compare the conditions of the high and low dosage experimental groups to determine if the high dose is more effective than the low dose.

Treatment manipulation. Treatments are the unique feature of experimental research that sets this design apart from all other research methods. Treatment manipulation helps control for the “cause” in cause-effect relationships. Naturally, the validity of experimental research depends on how well the treatment was manipulated. Treatment manipulation must be checked using pretests and pilot tests prior to the experimental study. Any measurements conducted before the treatment is administered are called pretest measures , while those conducted after the treatment are posttest measures .

Random selection and assignment. Random selection is the process of randomly drawing a sample from a population or a sampling frame. This approach is typically employed in survey research, and assures that each unit in the population has a positive chance of being selected into the sample. Random assignment is however a process of randomly assigning subjects to experimental or control groups. This is a standard practice in true experimental research to ensure that treatment groups are similar (equivalent) to each other and to the control group, prior to treatment administration. Random selection is related to sampling, and is therefore, more closely related to the external validity (generalizability) of findings. However, random assignment is related to design, and is therefore most related to internal validity. It is possible to have both random selection and random assignment in well-designed experimental research, but quasi-experimental research involves neither random selection nor random assignment.

Threats to internal validity. Although experimental designs are considered more rigorous than other research methods in terms of the internal validity of their inferences (by virtue of their ability to control causes through treatment manipulation), they are not immune to internal validity threats. Some of these threats to internal validity are described below, within the context of a study of the impact of a special remedial math tutoring program for improving the math abilities of high school students.

• History threat is the possibility that the observed effects (dependent variables) are caused by extraneous or historical events rather than by the experimental treatment. For instance, students’ post-remedial math score improvement may have been caused by their preparation for a math exam at their school, rather than the remedial math program.
• Maturation threat refers to the possibility that observed effects are caused by natural maturation of subjects (e.g., a general improvement in their intellectual ability to understand complex concepts) rather than the experimental treatment.
• Testing threat is a threat in pre-post designs where subjects’ posttest responses are conditioned by their pretest responses. For instance, if students remember their answers from the pretest evaluation, they may tend to repeat them in the posttest exam. Not conducting a pretest can help avoid this threat.
• Instrumentation threat , which also occurs in pre-post designs, refers to the possibility that the difference between pretest and posttest scores is not due to the remedial math program, but due to changes in the administered test, such as the posttest having a higher or lower degree of difficulty than the pretest.
• Mortality threat refers to the possibility that subjects may be dropping out of the study at differential rates between the treatment and control groups due to a systematic reason, such that the dropouts were mostly students who scored low on the pretest. If the low-performing students drop out, the results of the posttest will be artificially inflated by the preponderance of high-performing students.
• Regression threat , also called a regression to the mean, refers to the statistical tendency of a group’s overall performance on a measure during a posttest to regress toward the mean of that measure rather than in the anticipated direction. For instance, if subjects scored high on a pretest, they will have a tendency to score lower on the posttest (closer to the mean) because their high scores (away from the mean) during the pretest was possibly a statistical aberration. This problem tends to be more prevalent in non-random samples and when the two measures are imperfectly correlated.

Two-Group Experimental Designs

The simplest true experimental designs are two group designs involving one treatment group and one control group, and are ideally suited for testing the effects of a single independent variable that can be manipulated as a treatment. The two basic two-group designs are the pretest-posttest control group design and the posttest-only control group design, while variations may include covariance designs. These designs are often depicted using a standardized design notation, where R represents random assignment of subjects to groups, X represents the treatment administered to the treatment group, and O represents pretest or posttest observations of the dependent variable (with different subscripts to distinguish between pretest and posttest observations of treatment and control groups).

Pretest-posttest control group design . In this design, subjects are randomly assigned to treatment and control groups, subjected to an initial (pretest) measurement of the dependent variables of interest, the treatment group is administered a treatment (representing the independent variable of interest), and the dependent variables measured again (posttest). The notation of this design is shown in Figure 10.1.

Figure 10.1. Pretest-posttest control group design

The effect E of the experimental treatment in the pretest posttest design is measured as the difference in the posttest and pretest scores between the treatment and control groups:

E = (O 2 – O 1 ) – (O 4 – O 3 )

Statistical analysis of this design involves a simple analysis of variance (ANOVA) between the treatment and control groups. The pretest posttest design handles several threats to internal validity, such as maturation, testing, and regression, since these threats can be expected to influence both treatment and control groups in a similar (random) manner. The selection threat is controlled via random assignment. However, additional threats to internal validity may exist. For instance, mortality can be a problem if there are differential dropout rates between the two groups, and the pretest measurement may bias the posttest measurement (especially if the pretest introduces unusual topics or content).

Posttest-only control group design . This design is a simpler version of the pretest-posttest design where pretest measurements are omitted. The design notation is shown in Figure 10.2.

Figure 10.2. Posttest only control group design.

The treatment effect is measured simply as the difference in the posttest scores between the two groups:

E = (O 1 – O 2 )

The appropriate statistical analysis of this design is also a two- group analysis of variance (ANOVA). The simplicity of this design makes it more attractive than the pretest-posttest design in terms of internal validity. This design controls for maturation, testing, regression, selection, and pretest-posttest interaction, though the mortality threat may continue to exist.

Covariance designs . Sometimes, measures of dependent variables may be influenced by extraneous variables called covariates . Covariates are those variables that are not of central interest to an experimental study, but should nevertheless be controlled in an experimental design in order to eliminate their potential effect on the dependent variable and therefore allow for a more accurate detection of the effects of the independent variables of interest. The experimental designs discussed earlier did not control for such covariates. A covariance design (also called a concomitant variable design) is a special type of pretest posttest control group design where the pretest measure is essentially a measurement of the covariates of interest rather than that of the dependent variables. The design notation is shown in Figure 10.3, where C represents the covariates:

Figure 10.3. Covariance design

Because the pretest measure is not a measurement of the dependent variable, but rather a covariate, the treatment effect is measured as the difference in the posttest scores between the treatment and control groups as:

Figure 10.4. 2 x 2 factorial design

Factorial designs can also be depicted using a design notation, such as that shown on the right panel of Figure 10.4. R represents random assignment of subjects to treatment groups, X represents the treatment groups themselves (the subscripts of X represents the level of each factor), and O represent observations of the dependent variable. Notice that the 2 x 2 factorial design will have four treatment groups, corresponding to the four combinations of the two levels of each factor. Correspondingly, the 2 x 3 design will have six treatment groups, and the 2 x 2 x 2 design will have eight treatment groups. As a rule of thumb, each cell in a factorial design should have a minimum sample size of 20 (this estimate is derived from Cohen’s power calculations based on medium effect sizes). So a 2 x 2 x 2 factorial design requires a minimum total sample size of 160 subjects, with at least 20 subjects in each cell. As you can see, the cost of data collection can increase substantially with more levels or factors in your factorial design. Sometimes, due to resource constraints, some cells in such factorial designs may not receive any treatment at all, which are called incomplete factorial designs . Such incomplete designs hurt our ability to draw inferences about the incomplete factors.

In a factorial design, a main effect is said to exist if the dependent variable shows a significant difference between multiple levels of one factor, at all levels of other factors. No change in the dependent variable across factor levels is the null case (baseline), from which main effects are evaluated. In the above example, you may see a main effect of instructional type, instructional time, or both on learning outcomes. An interaction effect exists when the effect of differences in one factor depends upon the level of a second factor. In our example, if the effect of instructional type on learning outcomes is greater for 3 hours/week of instructional time than for 1.5 hours/week, then we can say that there is an interaction effect between instructional type and instructional time on learning outcomes. Note that the presence of interaction effects dominate and make main effects irrelevant, and it is not meaningful to interpret main effects if interaction effects are significant.

Hybrid Experimental Designs

Hybrid designs are those that are formed by combining features of more established designs. Three such hybrid designs are randomized bocks design, Solomon four-group design, and switched replications design.

Randomized block design. This is a variation of the posttest-only or pretest-posttest control group design where the subject population can be grouped into relatively homogeneous subgroups (called blocks ) within which the experiment is replicated. For instance, if you want to replicate the same posttest-only design among university students and full -time working professionals (two homogeneous blocks), subjects in both blocks are randomly split between treatment group (receiving the same treatment) or control group (see Figure 10.5). The purpose of this design is to reduce the “noise” or variance in data that may be attributable to differences between the blocks so that the actual effect of interest can be detected more accurately.

Figure 10.5. Randomized blocks design.

Solomon four-group design . In this design, the sample is divided into two treatment groups and two control groups. One treatment group and one control group receive the pretest, and the other two groups do not. This design represents a combination of posttest-only and pretest-posttest control group design, and is intended to test for the potential biasing effect of pretest measurement on posttest measures that tends to occur in pretest-posttest designs but not in posttest only designs. The design notation is shown in Figure 10.6.

Figure 10.6. Solomon four-group design

Switched replication design . This is a two-group design implemented in two phases with three waves of measurement. The treatment group in the first phase serves as the control group in the second phase, and the control group in the first phase becomes the treatment group in the second phase, as illustrated in Figure 10.7. In other words, the original design is repeated or replicated temporally with treatment/control roles switched between the two groups. By the end of the study, all participants will have received the treatment either during the first or the second phase. This design is most feasible in organizational contexts where organizational programs (e.g., employee training) are implemented in a phased manner or are repeated at regular intervals.

Figure 10.7. Switched replication design.

Quasi-Experimental Designs

Quasi-experimental designs are almost identical to true experimental designs, but lacking one key ingredient: random assignment. For instance, one entire class section or one organization is used as the treatment group, while another section of the same class or a different organization in the same industry is used as the control group. This lack of random assignment potentially results in groups that are non-equivalent, such as one group possessing greater mastery of a certain content than the other group, say by virtue of having a better teacher in a previous semester, which introduces the possibility of selection bias . Quasi-experimental designs are therefore inferior to true experimental designs in interval validity due to the presence of a variety of selection related threats such as selection-maturation threat (the treatment and control groups maturing at different rates), selection-history threat (the treatment and control groups being differentially impact by extraneous or historical events), selection-regression threat (the treatment and control groups regressing toward the mean between pretest and posttest at different rates), selection-instrumentation threat (the treatment and control groups responding differently to the measurement), selection-testing (the treatment and control groups responding differently to the pretest), and selection-mortality (the treatment and control groups demonstrating differential dropout rates). Given these selection threats, it is generally preferable to avoid quasi-experimental designs to the greatest extent possible.

Many true experimental designs can be converted to quasi-experimental designs by omitting random assignment. For instance, the quasi-equivalent version of pretest-posttest control group design is called nonequivalent groups design (NEGD), as shown in Figure 10.8, with random assignment R replaced by non-equivalent (non-random) assignment N . Likewise, the quasi -experimental version of switched replication design is called non-equivalent switched replication design (see Figure 10.9).

Figure 10.8. NEGD design.

Figure 10.9. Non-equivalent switched replication design.

In addition, there are quite a few unique non -equivalent designs without corresponding true experimental design cousins. Some of the more useful of these designs are discussed next.

Regression-discontinuity (RD) design . This is a non-equivalent pretest-posttest design where subjects are assigned to treatment or control group based on a cutoff score on a preprogram measure. For instance, patients who are severely ill may be assigned to a treatment group to test the efficacy of a new drug or treatment protocol and those who are mildly ill are assigned to the control group. In another example, students who are lagging behind on standardized test scores may be selected for a remedial curriculum program intended to improve their performance, while those who score high on such tests are not selected from the remedial program. The design notation can be represented as follows, where C represents the cutoff score:

Figure 10.10. RD design.

Because of the use of a cutoff score, it is possible that the observed results may be a function of the cutoff score rather than the treatment, which introduces a new threat to internal validity. However, using the cutoff score also ensures that limited or costly resources are distributed to people who need them the most rather than randomly across a population, while simultaneously allowing a quasi-experimental treatment. The control group scores in the RD design does not serve as a benchmark for comparing treatment group scores, given the systematic non-equivalence between the two groups. Rather, if there is no discontinuity between pretest and posttest scores in the control group, but such a discontinuity persists in the treatment group, then this discontinuity is viewed as evidence of the treatment effect.

Proxy pretest design . This design, shown in Figure 10.11, looks very similar to the standard NEGD (pretest-posttest) design, with one critical difference: the pretest score is collected after the treatment is administered. A typical application of this design is when a researcher is brought in to test the efficacy of a program (e.g., an educational program) after the program has already started and pretest data is not available. Under such circumstances, the best option for the researcher is often to use a different prerecorded measure, such as students’ grade point average before the start of the program, as a proxy for pretest data. A variation of the proxy pretest design is to use subjects’ posttest recollection of pretest data, which may be subject to recall bias, but nevertheless may provide a measure of perceived gain or change in the dependent variable.

Figure 10.11. Proxy pretest design.

Separate pretest-posttest samples design . This design is useful if it is not possible to collect pretest and posttest data from the same subjects for some reason. As shown in Figure 10.12, there are four groups in this design, but two groups come from a single non-equivalent group, while the other two groups come from a different non-equivalent group. For instance, you want to test customer satisfaction with a new online service that is implemented in one city but not in another. In this case, customers in the first city serve as the treatment group and those in the second city constitute the control group. If it is not possible to obtain pretest and posttest measures from the same customers, you can measure customer satisfaction at one point in time, implement the new service program, and measure customer satisfaction (with a different set of customers) after the program is implemented. Customer satisfaction is also measured in the control group at the same times as in the treatment group, but without the new program implementation. The design is not particularly strong, because you cannot examine the changes in any specific customer’s satisfaction score before and after the implementation, but you can only examine average customer satisfaction scores. Despite the lower internal validity, this design may still be a useful way of collecting quasi-experimental data when pretest and posttest data are not available from the same subjects.

Figure 10.12. Separate pretest-posttest samples design.

Nonequivalent dependent variable (NEDV) design . This is a single-group pre-post quasi-experimental design with two outcome measures, where one measure is theoretically expected to be influenced by the treatment and the other measure is not. For instance, if you are designing a new calculus curriculum for high school students, this curriculum is likely to influence students’ posttest calculus scores but not algebra scores. However, the posttest algebra scores may still vary due to extraneous factors such as history or maturation. Hence, the pre-post algebra scores can be used as a control measure, while that of pre-post calculus can be treated as the treatment measure. The design notation, shown in Figure 10.13, indicates the single group by a single N , followed by pretest O 1 and posttest O 2 for calculus and algebra for the same group of students. This design is weak in internal validity, but its advantage lies in not having to use a separate control group.

An interesting variation of the NEDV design is a pattern matching NEDV design , which employs multiple outcome variables and a theory that explains how much each variable will be affected by the treatment. The researcher can then examine if the theoretical prediction is matched in actual observations. This pattern-matching technique, based on the degree of correspondence between theoretical and observed patterns is a powerful way of alleviating internal validity concerns in the original NEDV design.

Figure 10.13. NEDV design.

Perils of Experimental Research

Experimental research is one of the most difficult of research designs, and should not be taken lightly. This type of research is often best with a multitude of methodological problems. First, though experimental research requires theories for framing hypotheses for testing, much of current experimental research is atheoretical. Without theories, the hypotheses being tested tend to be ad hoc, possibly illogical, and meaningless. Second, many of the measurement instruments used in experimental research are not tested for reliability and validity, and are incomparable across studies. Consequently, results generated using such instruments are also incomparable. Third, many experimental research use inappropriate research designs, such as irrelevant dependent variables, no interaction effects, no experimental controls, and non-equivalent stimulus across treatment groups. Findings from such studies tend to lack internal validity and are highly suspect. Fourth, the treatments (tasks) used in experimental research may be diverse, incomparable, and inconsistent across studies and sometimes inappropriate for the subject population. For instance, undergraduate student subjects are often asked to pretend that they are marketing managers and asked to perform a complex budget allocation task in which they have no experience or expertise. The use of such inappropriate tasks, introduces new threats to internal validity (i.e., subject’s performance may be an artifact of the content or difficulty of the task setting), generates findings that are non-interpretable and meaningless, and makes integration of findings across studies impossible.

The design of proper experimental treatments is a very important task in experimental design, because the treatment is the raison d’etre of the experimental method, and must never be rushed or neglected. To design an adequate and appropriate task, researchers should use prevalidated tasks if available, conduct treatment manipulation checks to check for the adequacy of such tasks (by debriefing subjects after performing the assigned task), conduct pilot tests (repeatedly, if necessary), and if doubt, using tasks that are simpler and familiar for the respondent sample than tasks that are complex or unfamiliar.

In summary, this chapter introduced key concepts in the experimental design research method and introduced a variety of true experimental and quasi-experimental designs. Although these designs vary widely in internal validity, designs with less internal validity should not be overlooked and may sometimes be useful under specific circumstances and empirical contingencies.

• Social Science Research: Principles, Methods, and Practices. Authored by : Anol Bhattacherjee. Provided by : University of South Florida. Located at : http://scholarcommons.usf.edu/oa_textbooks/3/ . License : CC BY-NC-SA: Attribution-NonCommercial-ShareAlike

Grab your spot at the free arXiv Accessibility Forum

Help | Advanced Search

Computer Science > Artificial Intelligence

Title: inductive or deductive rethinking the fundamental reasoning abilities of llms.

Abstract: Reasoning encompasses two typical types: deductive reasoning and inductive reasoning. Despite extensive research into the reasoning capabilities of Large Language Models (LLMs), most studies have failed to rigorously differentiate between inductive and deductive reasoning, leading to a blending of the two. This raises an essential question: In LLM reasoning, which poses a greater challenge - deductive or inductive reasoning? While the deductive reasoning capabilities of LLMs, (i.e. their capacity to follow instructions in reasoning tasks), have received considerable attention, their abilities in true inductive reasoning remain largely unexplored. To delve into the true inductive reasoning capabilities of LLMs, we propose a novel framework, SolverLearner. This framework enables LLMs to learn the underlying function (i.e., $y = f_w(x)$), that maps input data points $(x)$ to their corresponding output values $(y)$, using only in-context examples. By focusing on inductive reasoning and separating it from LLM-based deductive reasoning, we can isolate and investigate inductive reasoning of LLMs in its pure form via SolverLearner. Our observations reveal that LLMs demonstrate remarkable inductive reasoning capabilities through SolverLearner, achieving near-perfect performance with ACC of 1 in most cases. Surprisingly, despite their strong inductive reasoning abilities, LLMs tend to relatively lack deductive reasoning capabilities, particularly in tasks involving ``counterfactual'' reasoning.

Submission history

Access paper:.

• HTML (experimental)
• Other Formats

References & Citations

• Google Scholar
• Semantic Scholar

BibTeX formatted citation

Bibliographic and Citation Tools

Code, data and media associated with this article, recommenders and search tools.

• Institution

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 31 July 2024

Liars know they are lying: differentiating disinformation from disagreement

• Stephan Lewandowsky   ORCID: orcid.org/0000-0003-1655-2013 1 , 2 ,
• Ullrich K. H. Ecker   ORCID: orcid.org/0000-0003-4743-313X 3 ,
• John Cook 4 ,
• Sander van der Linden 5 ,
• Jon Roozenbeek 6 ,
• Naomi Oreskes 7 &
• Lee C. McIntyre 8  

Humanities and Social Sciences Communications volume  11 , Article number:  986 ( 2024 ) Cite this article

4186 Accesses

194 Altmetric

Metrics details

• Politics and international relations

Mis- and disinformation pose substantial societal challenges, and have thus become the focus of a substantive field of research. However, the field of misinformation research has recently come under scrutiny on two fronts. First, a political response has emerged, claiming that misinformation research aims to censor conservative voices. Second, some scholars have questioned the utility of misinformation research altogether, arguing that misinformation is not sufficiently identifiable or widespread to warrant much concern or action. Here, we rebut these claims. We contend that the spread of misinformation—and in particular willful disinformation—is demonstrably harmful to public health, evidence-informed policymaking, and democratic processes. We also show that disinformation and outright lies can often be identified and differ from good-faith political contestation. We conclude by showing how misinformation and disinformation can be at least partially mitigated using a variety of empirically validated, rights-preserving methods that do not involve censorship.

Similar content being viewed by others

A meta-analysis of correction effects in science-relevant misinformation

Understanding belief in political statements using a model-driven experimental approach: a registered report

Framing fact-checks as a “confirmation” increases engagement with corrections of misinformation: a four-country study

“If everybody always lies to you, the consequence is not that you believe the lies, but rather that nobody believes anything any longer.”

— Hannah Arendt

One of the normative goods on which democracy relies is accountable representation through fair elections (Tenove, 2020 ). This good is at risk when public perception of the integrity of elections is significantly distorted by false or misleading information (H. Farrell and Schneier, 2018 ). The two most recent presidential elections in the U.S. were accompanied by a plethora of false or misleading information, which grew from false information about voting procedures in 2016 (Stapleton, 2016 ) to the “big lie” that the 2020 election was stolen from Donald Trump, which he and his allies have baselessly and ceaselessly repeated (Henricksen and Betz, 2023 ; Jacobson, 2023 ). Misleading or false information has always been part and parcel of political debate (Lewandowsky et al., 2017 ), and the public arguably accepts a certain amount of dishonesty from politicians (e.g., McGraw, 1998 ; Swire-Thompson et al., 2020 ). However, Trump’s big lie differs from conventional, often accidentally disseminated, mis information by being a deliberate attempt to dis inform the public.

Scholars tend to think of disinformation as a type of misinformation and technically that is true: intentional falsehoods are but one subset of falsehoods (Lewandowsky et al., 2013 ) and intentionality does not affect how people’s cognitive apparatus processes the information (e.g., L. K. Fazio et al., 2015 ). But given the real-world risks that disinformation poses for democracy (Lewandowsky et al., 2023 ), we think it is important to be clear at the outset whether we are dealing with a mistake versus a lie.

The tobacco industry’s 50-year-long campaign of disinformation about the health risks from smoking is a classic case of deliberate deception and has been recognized as such by the U.S. Federal Courts (Smith et al., 2011 , see also Civil Action 99-2496(GK) United States District Court, District of Columbia. United States v. Philip Morris Inc.). This article focuses primarily on the nature of disinformation and how it can be identified, and places it into the contemporary societal context. Wherever we make a broader point about the prevalence of false information, its identifiability or its effects, we use the term misinformation to indicate that intentionality is secondary or unknown.

An analysis of mis- and disinformation cannot be complete without also considering the role of the audience, in particular when people share information with others, where the distinction between mis- and disinformation becomes more fluid. In most instances, when people share information, they do so based on the justifiable default expectation that it is true (Grice, 1975 ). However, occasionally people also share information that they know to be false, a phenomenon known as “participatory propaganda” (e.g., Lewandowsky, 2022 ; Wanless and Berk, 2019 ). One factor that may underlie participatory propaganda is the social utility that persons can derive from beliefs, even if they are false, which may stimulate them into rationalizing belief in falsehoods (Williams, 2022 ). The converse may also occur, where members of the public accurately report an experience, which is then taken up by others, usually political operatives or elites, and redeployed for a malign purpose. For example, technical problems with some voting machines in Arizona in 2022 were seized on by Trump and his allies as being an attempt to disenfranchise conservative voters (Reid, 2022 ). Both cases underscore the importance of audience involvement and the reverberating feedback loops between political actors and the public which can often amplify and extend the reach of intentional disinformation (Starbird et al., 2023 ; Vosoughi et al., 2018 ), and which can often involve non-epistemic but nonetheless rational choices (Williams, 2021 , 2022 ).

The circular and mutually reinforcing relationship between political actors and the public was a particularly pernicious aspect of the rhetoric associated with Trump’s big lie (for a detailed analysis, see Starbird et al., 2023 ). During the joint session of Congress to certify the election on 6 January 2021, politicians speaking in support of Donald Trump and his unsubstantiated claims about election irregularities appealed not to evidence or facts but to public opinion. For example, Senator Ted Cruz cited a poll result that 39% of the public believed the election had been “rigged”. Similarly, Representative Jim Jordan (R-Ohio), who is now Chairman of the House Judiciary Committee, argued against certification of the election by arguing that “80 million of our fellow citizens, Republicans and Democrats, have doubts about this election; and 60 million people, 60 million Americans think it was stolen” (Salek, 2023 ). The appeal to public opinion to buttress false claims is cynical in light of the fact that public opinion was the result of systematic disinformation in the first place. While nearly 75% of Republicans considered the election result legitimate on election day, this share dropped to around 40% within a few days (Arceneaux and Truex, 2022 ), coinciding with the period during which Trump ramped up his false claims about the election being stolen. By December 2020, 28% of American conservatives did not support a peaceful transfer of power (Weinschenk et al., 2021 ), perhaps the most important bedrock of democracy. Among liberals, by contrast, this attitude was far more marginal (3%).

Public opinion has shifted remarkably little since the election. In August 2023, nearly 70% of Republican voters continued to question the legitimacy of President Biden’s electoral win in 2020. More than half of those who questioned Biden’s win believed that there was solid evidence proving that the election was not legitimate (Agiesta and Edwards-Levy, 2023 ). However, the purported evidence marshaled in support of this view has been repeatedly shown to be false (Canon and Sherman, 2021 ; Eggers et al., 2021 ; Grofman and Cervas, 2023 ). Footnote 1 It is particularly striking that high levels of false election beliefs are found even under conditions known to reduce “expressive responding”—that is, responses that express support for a position but do not reflect true belief (Graham and Yair, 2023 ).

The entrenchment of the big lie erodes the core of American democracy and puts pressure on Republican politicians to cater to antidemocratic forces (Arceneaux and Truex, 2022 ; Jacobson, 2021 , 2023 ). It has demonstrably decreased trust in the electoral system (Berlinski et al., 2021 ), and a violent constitutional crisis has been identified as a “tail risk” for the United States in 2024 (McLauchlin, 2023 ). Similar crises in which right-wing authoritarian movements are dismantling democratic institutions and safeguards have found traction in many countries around the world including liberal democracies (Cooley and Nexon, 2022 ).

In this context, it is worth noting that the situation in other countries, notably in the Global South, may differ from the situation in the U.S. (Badrinathan and Chauchard, 2024 ). On the one hand, low state capacity and infrastructure constraints may curtail the ability of powerful actors to spread disinformation and propaganda (though see Kellow and Steeves, 1998 ; Li, 2004 , for discussion of the role of government-adjacent radio station RTLM in facilitating the 1994 Rwandan genocide). On the other hand, such spread can be facilitated by the fact that closed, encrypted social-media channels are particularly popular in the Global South, sometimes providing an alternative source of news when broadcast channels and other conventional media have limited reach. In those cases, dissemination strategies will also be less direct, relying more on distributed “cyber-armies” than direct one-to-millions broadcasts such as Trump’s social-media posts (Badrinathan, 2021 ; Jalli and Idris, 2019 ). The harm that can be caused by such distributed systems was vividly illustrated by the false rumors about child kidnapers shared in Indian WhatsApp groups in 2018, which incited at least 16 mob lynchings, causing the deaths of 29 innocent people (Dixit and Mac, 2018 ). The ensuing interplay between the attempts of the Indian government to hold WhatsApp accountable and Meta, the platform’s owner, highlights the limited power that governments in the Global South hold over multinational technology corporations (Arun, 2019 ). As a result, many platforms do not even have moderation tools for problematic content in popular non-Western languages (Shahid and Vashistha, 2023 ).

The power asymmetry between corporations and the Global South has been noted repeatedly, and recent calls for action include the idea of collective action by countries in the Global South to insist on regulation of platforms (Takhshid, 2021 ). We have only scratched the surface of a big global issue that is in urgent need of being addressed.

Despite these differences between the Global North and South, beliefs in political misinformation can be pervasive regardless of regime type or development level (e.g., for a discussion in the context of the “developing democracy” of Brazil, see Dourado and Salgado, 2021 ; Pereira et al., 2022 ).

The political landscape of disinformation

Given that the 2020 election was lost by the Republican candidate, the finding that conservatives are more likely than liberals to believe false election claims is explainable on the basis of motivated cognition and the general finding that conspiracy theories “are for losers” (Uscinski and Parent, 2014 ); that is, they provide an explanation—even if only a chimerical one—for a political setback to the losing parties. There is no a priori reason to assume that susceptibility to disinformation is skewed across the political spectrum.

However, a large body of recent research on the American public and U.S. political actors has consistently identified a pervasive ideological asymmetry, with conservatives and people from the populist right being far more likely to consume, share, and believe false information than their liberal counterparts (Benkler et al., 2018 ; Garrett and Bond, 2021 ; González-Bailón et al., 2023 ; Grinberg et al., 2019 ; Guess et al., 2020a ; Guess et al., 2020b ; Guess et al., 2019 ; Ognyanova et al., 2020 ). Research into the asymmetry culminated in a recent analysis of the news diet of 208 million Facebook users in the U.S., which discovered that a substantial segment of the news ecosystem is consumed exclusively by conservatives and that most misinformation exists within this ideological bubble (González-Bailón et al., 2023 ). Although the reasons for this asymmetry are not fully understood, Lasser et al. ( 2022 ) recently showed that it also held for politicians, with Republican members of Congress disseminating far more low-quality information on Twitter/X than their Democratic counterparts. Greene ( 2024 ) reported a parallel analysis for Facebook and found the same asymmetry between politicians of the two major parties. Similarly, Benkler et al. ( 2018 ) showed how the particular structure of the American media scene, with a dense interconnected cluster of right-wing sources that is separate from the remaining mainstream, fosters political asymmetry in the use and consumption of disinformation.

This asymmetry extends beyond the political domain to health-related information, which might at first glance appear to be of sufficient importance for most people to cast aside their political leanings. A recent systematic review discovered eight studies that identified conservatism as a predictor of susceptibility to health misinformation, seven studies that found no association involving political leanings, and not a single study that showed liberals to be more misinformed on health topics than conservatives (Nan et al., 2022 ). The observed political asymmetry is also not limited to survey results or other behavioral measures. Wallace et al. ( 2023 ) examined vaccination and mortality data from two U.S. states (Ohio and Florida) during the COVID-19 pandemic and found a widening partisan gap in excess mortality. Specifically, whereas mortality rates were equal for registered Republican and Democratic voters pre-pandemic, a wide partisan gap—with excess death rates among Republicans being up to 43% greater than among Democratic voters—was observed after vaccines had become available for everyone. The gap was greatest in counties with the lowest share of vaccinated people and it almost disappeared for the most vaccinated counties. Similar results have been reported across U.S. states (Leonhardt, 2021 ). One explanation for these patterns invokes the frequent false statements by Republican politicians and conservative news networks—foremost Fox News—that discredited the COVID-19 vaccines (Hotez, 2023 ). In support, consumption of Fox News has been causally linked to lower vaccination rates (Pinna et al., 2022 ).

Moreover, a recent analysis identified a specific “Trump effect” such that even conditional on the Republican vote share, support for Trump was additionally and causally associated with a lower vaccination rate (Jung and Lee, 2023 ).

The political asymmetry surrounding the dissemination and consumption of misinformation must be caveated in two ways. First, although the asymmetry is substantial and pervasive it is not absolute. For some materials, such as specific conspiracy theories, the asymmetry is found to be attenuated in some studies (A. Enders et al., 2022 ; M. Enders and Uscinski, 2021 ). Second, the asymmetry observed among American politicians does not necessarily hold in other countries. Lasser et al. ( 2022 ) examined tweets by British and German parliamentarians and showed that with the exception of the extreme right in Germany (the AfD party), politicians across the mainstream spectrum were equally judicious in what information they shared in their tweets. This finding suggests that it is not conservatism per se that is associated with asymmetric reliance on misinformation, but the specific manifestation of conservatism currently dominant in the American political landscape.

Notwithstanding those caveats, the political asymmetry surrounding the dissemination and consumption of misinformation in the U.S. has been accompanied by at least two major issues: First, there has been a strong political response by Republicans in Congress who have commenced a campaign against misinformation research and researchers, claiming that the research seeks to censor conservative voices. Second, the political backlash has coincided with growing self-reflection and critique among scholars, some of whom began to question the misinformation research effort, culminating in claims that misinformation may not be sufficiently identifiable or widespread to warrant concern or countermeasures. We now take up these two issues in turn.

The politicization of misinformation research

At the time of this writing, Representative Jim Jordan, R-Ohio, has been leading a campaign against misinformation research and misinformation researchers in his role as Chairman of the House Judiciary Committee. The core allegation by Jordan and his allies Footnote 2 is that misinformation researchers are part of a purported “Censorship Industrial Complex” that is assisting the Biden administration in its purported endeavor to pressure platforms into suppressing conservative viewpoints (U.S. House of Representatives Judiciary Committee, 2023 ). The allegation is, however, problematic for at least four reasons: it rests on false assertions; it ironically denies first-amendment rights to researchers; it rests on a basic premise that is false; and it misunderstands the role of platforms in content moderation.

Concerning the first point, Jordan has subpoenaed several prominent academics engaged in the study of mis- and disinformation based on false assertions. For example, Dr Kate Starbird, an expert on disinformation from the University of Washington, was called to testify before Jordan’s subcommittee and had to defend herself against accusations that she was colluding with the Biden administration in an effort to chill conservative speech (Nix and Menn, 2023 ). Core to the specific allegations against Starbird and her colleagues is a claim—initially voiced by online conspiracy theorists—that they colluded with the Department of Homeland Security to censor 22 million tweets during the 2020 election campaign. In actual fact, the researchers collected 22 million tweets for analysis, and flagged about 3000 (0.0001 of the total) for potential violations of Twitter’s terms of use (Blitzer, 2023 ).

Second, Jordan’s purported championing of free speech is difficult to reconcile with the chilling effect the House Committee’s actions have had on the first-amendment rights of researchers. According to Starbird, “The people that benefit from the spread of disinformation have effectively silenced many of the people that would try to call them out” (Rutenberg and Myers, 2024 ). The deterring effect on the research community is widespread (Bernstein, 2023 ; Nix et al., 2023 ). Similarly, Facebook and YouTube have reversed their restrictions on content claiming that the 2020 election was stolen. Election disinformation, unsurprisingly, has seen an uptick in response (Rutenberg and Myers, 2024 ).

Third, Jordan’s campaign rests on a false premise, namely that social-media platforms are biased against conservatives. Together with other conservative figures such as Tucker Carlson (formerly with Fox News) and Ben Shapiro, Jordan claimed in 2020 that “Big Tech is out to get conservatives”. This claim has been shown to be wrong by several studies. For example, an analysis of Facebook engagements during the 2016 election campaign revealed that conservative outlets (Fox News, Breitbart, and Daily Caller) amassed 839 million interactions, dwarfing more centrist outlets (CNN with 191 million and ABC news with 138 million), and totaling more than the remaining seven mainstream pages in the top 10 (Barrett and Sims, 2021 ). Another analysis involving millions of Twitter users and 6.2 million news articles shared on the platform also found that conservatives enjoy greater algorithmic amplification than people on the political left (Huszár et al., 2022 ). Moreover, the Congressional January 6th Committee detailed the way in which major platforms, including Twitter and Facebook, facilitated the organization of the violent insurrection in a 122-page memo, although much of that information did not make it into the final committee report (Zakrzewski et al., 2023 ). Congressional investigators discovered that the platforms failed to heed their own experts’ warnings about violent rhetoric on their platforms, and selectively failed to enforce existing rules to avoid antagonizing conservatives for fear of reprisals (Zakrzewski et al., 2023 ).

Finally, and perhaps most important, Jordan’s pursuit fails to differentiate between the roles of government and the platforms, and in particular ignores the crucial role that platforms already play in shaping people’s information diet (Lewandowsky et al., 2023a ). In a nutshell, the internet is currently neither unregulated nor is all information on the internet equally free. Instead, nearly all content on social media is curated by algorithms that are designed to maximize dwell time in pursuit of the platforms’ advertising profit (Lewandowsky and Pomerantsev, 2022 ; Wu, 2017 ). Algorithms therefore favor captivating information that keeps users engaged. Unfortunately, human attention is known to be biased towards negative information (Soroka et al., 2019 ), which creates an incentive for platforms to drench users in outrage-evoking content. Similar to junk food that supermarkets strategically place at checkout lanes, the information that is preferentially curated by platforms may satisfy our presumed momentary preferences while reducing our long-term well-being. If platforms were to address their role in those dynamics, for example by redesigning their algorithms, this would hardly constitute censorship. Solving a problem one has caused is good iterative design rather than bias or suppression of opinions. No one would accuse a supermarket of suppressing consumers’ preferences if the checkout lanes put on offer celery instead of chocolate bars.

In summary, far from being a restorative effort in defense of free speech, Jordan’s attacks are reminiscent of similar campaigns launched against inconvenient scientists by the tobacco and fossil-fuel industries (Lewandowsky et al., 2023b ). In all cases scientists have been subject to personal abuse, their email correspondence is hacked or subpoenaed, and allegations are woven together from snippets of decontextualized actions or events (Blitzer, 2023 ). Because these attacks are systemic, the response also requires a systemic approach (Desikan et al., 2023 ). However, any such response seems unlikely to be achievable in the current political landscape. Scientists who work under such challenging conditions must therefore rely on other avenues to protect their integrity. The U.S. National Academy of Sciences has published a list of resources for scientists under attack. Footnote 3 Specific recommendations include responding publicly to valid criticism (without, however, engaging in a long drawn-out direct conversation with an attacker), reporting abusive messages to the authorities, and seeking support from colleagues who have been in similar situations (Kinser, 2020 ).

The attacks have also coincided with moves by the platforms and the courts that align with Jordan’s claims. For example, the major platforms (Meta, Google, Twitter/X, and Amazon) have cut back on the number of staff dedicated to combating hate speech and misinformation (Field and Vanian, 2023 ). Meta (the parent company of Facebook) has been laying off employees in its “content review” team, which had been involved in countering misinformation and disinformation in the 2022 midterm election, citing confidence in improved electronic tools for detecting inauthentic accounts. It remains to be seen how the platform actions will play out during the 2024 presidential election.

In the legal arena, a Trump-appointed federal judge in Louisiana barred the Biden administration from having any contact with social-media companies and certain research institutions to discuss safeguarding elections in July 2023. The judgement echoed the claims by Jim Jordan and other Republicans that there was collusion between the White House and the social-media companies to censor conservative voices under the guise of fighting disinformation about COVID-19 during the pandemic and false election claims during the 2022 midterms. Although there are important and potentially problematic implications for free speech that arise whenever a government gets involved in managing what it considers misinformation (Neo, 2022 ; Vese, 2022 ), the Louisiana ruling was particularly broad in its prohibitions (West, 2023 ). The implications of the ruling include denying election officials access to information gathered by independent research bodies (the ruling lists “the Election Integrity Partnership, the Virality Project, the Stanford Internet Observatory, or any like project or group”) that would enable them to debunk false election-related information and provide more accurate information instead. The Supreme Court blocked the Louisiana ruling in October 2023 (Hurley, 2023 ) but agreed to a full hearing later in its current term. We return to the conflict between free speech and the adverse effects of disinformation later.

The post-modern critiques of misinformation research

At the heart of research on misinformation is the belief that the concepts of truth and falsehood are essential to democracy, to cognition, and to daily life, and that the status of many, but of course not all claims, can be determined with sufficient accuracy to warrant rebuttal of false information. For example, the “big lie” about a stolen election is just that—it is a lie with no sustainable evidentiary support and it is routinely referred to as such in the scholarly literature (e.g., Arceneaux and Truex, 2022 ; Canon and Sherman, 2021 ; Graham and Yair, 2023 ; Henricksen and Betz, 2023 ; Jacobson, 2021 , 2023 ; Painter and Fernandes, 2022 ). The lie has been rejected by 62 American courts, all of which dismissed or ruled against law suits questioning the legitimacy of the election by Donald Trump or his supporters. Footnote 4

It is curious that the reaction by Trump and some of his most ardent public supporters to such determinative judgments about the falsity of his claims has not been to claim that they are in fact true, but to attack the idea that objective knowledge is even possible. When confronted with a lie, Trump’s adviser Kellyanne Conway once famously quipped that she was presenting “alternative facts.” On another occasion, Trump’s attorney Rudy Giuliani declared that “truth isn’t truth.” Such a strategy seems oddly reminiscent of the postmodernist critique of the possibility of objective knowledge, which first arose as a core aspect of 1930s fascism and was then adapted by left-wing literary criticism from the 1960s onward (Lewandowsky, 2020 ). At that time, humanities scholars had grown increasingly uncomfortable with the idea that facts were just facts, and that there was no role for considering the personal or political interests of those who were engaged in the pursuit of empirical knowledge. In this, postmodernists raised an important point of self-reflection for scientists and others who blithely claimed that there was an impenetrable wall between facts and values. But then they took things too far. Derrida claimed that there was no such thing as objective knowledge. Foucault went on to suggest that —given this— all knowledge claims were nothing more than an assertion of the political interests of the investigator (McIntyre, 2018 , p. 124).

This led to the “science wars” of the 1990s, when scientists and their allies fought back against subjectivism and relativism to defend the importance of objective knowledge at least as a regulative ideal of empirical inquiry. This particular attack on science eventually dissipated —and in the face of damage it had done to objective knowledge claims like the reality of global warming, some postmodernists such as Bruno Latour eventually even apologized (Latour, 2004 )— but the damage was already done. Meanwhile, both the corporate sector and the religious and political right wing had once again taken up the strategy in their attacks on science. The advantage of post-modernism for anti-democratic purposes is obvious, and has echoes of authoritarian attacks on truth-tellers and their defenders throughout history. Indeed, to someone who embraces the idea that their political ideology should have supremacy over objective reality, the advantages of postmodernism are clear. Not only can falsehoods about the economy, crime, and political violence be offered as “alternative narratives” to carefully-measured statistics or other forms of evidence, but the credibility of any party as an objective truth-teller can be undermined. And this suits the authoritarian just fine—for where there is no truth then there can be no blame or accountability either.

Hannah Arendt long ago recognized the dangers of this strategy when she wrote: “the ideal subject of totalitarian rule is not the convinced Nazi or the convinced communist, but people for whom the distinction between fact and fiction … true and false … no longer exist.” This easy political slide into postmodernism does violence to the idea that truth matters, that facts can be discovered through empirical analysis, and that it is crucial to attempt to discern the facts before we can make good policy—especially when we hold competing values that will impact policy choice. And this is true even more so in an era when the creation and amplification of knowledge claims are so easily subject to digital manipulation and weaponization by anyone who has a personal or political interest. Fortunately, researchers have developed conceptual, cognitive, and computational tools that permit the differentiation between legitimate contestation of facts on the one hand, and misinformation and willful disinformation on the other.

The identifiability of contested facts

Notwithstanding our rejection of the postmodernist project, we do not dispute its core idea that many contested assertions cannot be unambiguously adjudicated by referring to “facts”. There are indeed cases in which different actors may legitimately question each other’s “facts”. In our view, these ambiguous cases are precisely those that merit democratic debate and contestation. When conducted in good-faith, such debates can be particularly revealing because both sides can marshal evidence in support.

To illustrate, consider the recent controversy surrounding a machine-learning tool known as COMPAS (Dressel and Farid, 2018 ), which is intended to assist judges in the U.S. by predicting the likelihood of recidivism of a specific offender. Critics accused COMPAS of being racially biased based on statistical analysis of the evidence (Angwin et al., 2016 ). The case rested on the observation that among defendants who ultimately did not re-offend, the algorithm misclassified African-Americans as being at risk of re-offending more than twice as often as White offenders. This misclassification can have serious consequences for a person because judges are inclined to treat high-risk defendants more harshly.

Proponents of COMPAS rejected this charge and argued that the algorithm was not racially biased because it predicted recidivism equally for Black and White offenders for each of its 10 risk categories. That is, the classification into risk categories based on a large number of indicator variables was racially unbiased—a Black person’s actual probability of re-offending was the same as that of a White person with the same risk score (Dieterich et al., 2016 ).

It turns out that it is mathematically impossible to simultaneously satisfy both forms of fairness—calibration and classification—when the base rates of re-offending differ between groups (Berk et al., 2021 ; Lagioia et al., 2023 ). That is, if a greater share of Black people are classified as high-risk—which the algorithm does in an unbiased manner—then it necessarily follows that a greater share of Black defendants who do not re-offend will also be mistakenly classified as high-risk. In those circumstances, it would be inappropriate to accuse one or the other side of spreading misinformation, as each party has mathematical justification for their position and a resolution can only be attained through a value-laden policy discussion. Indeed, to our knowledge, the main contestants in this debate—Northpointe, the manufacturer of COMPAS (Dieterich et al., 2016 ) and ProPublica, a public-interest media organization (Angwin et al., 2016 )—did not level charges of misinformation against each other despite engaging in robust debate.

A similar controversy with even greater stakes arose in the context of the COVID-19 vaccine rollout in the U.S. in 2021. Unlike most other countries, which vaccinated their populations according to age alone—with the elderly being given highest priority because of their much higher mortality rate from COVID-19—the U.S. Advisory Committee on Immunization Practices (ACIP) favored a policy that gave higher priority to essential workers (e.g., food and transport workers) than the elderly. This policy was partially motivated by the fact that racial minorities (Blacks and Hispanics) are underrepresented among adults over 65, whereas they are slightly over-represented among essential workers—thus, under an age-based policy the share of Whites who receive the vaccine would have initially been greater than their proportion in the population would have warranted. Conversely, Blacks would have been underrepresented among the vaccinated early on (Mounk, 2023 ). This inequity could be avoided by first vaccinating essential workers among whom racial minorities were over-represented. However, because the age distribution of essential workers has a much lower average, fewer lives were saved among vaccinated essential workers—whose young age rendered their risk of dying from COVID-19 low to begin with—than would have been saved among the elderly had they been vaccinated (Rumpler et al., 2023 ). Modeling has confirmed that while the essential-worker policy introduced racial equity in terms of doses administered, more lives would have been saved in all ethnic groups under an age-based policy (Rumpler et al., 2023 ). Again, the apparent fairness of a policy depended on the outcome measure: doses administered vs. lives saved. Given the unequal distributions of different ethnic and racial groups across different ages, no mathematical possibility exists to settle on a single “fair” policy. Public opinion appears to have been broadly in line with the policy ultimately adopted by ACIP (Persad et al., 2021 ).

The controversies surrounding COMPAS and ACIP’s vaccination policy are just two instances of a much wider problem, which is that when issues become sufficiently complex, even good-faith actors may find it impossible to agree. One reason is that cognitive limitations prevent a full Bayesian representation (the gold standard of rationality) of the problem (Pothos et al., 2021 ). Instead, people are forced to simplify their representations, for example by partitioning their knowledge (Lewandowsky et al., 2002 ). Persistent and irresolvable disagreements are thus almost ensured by human cognitive limitations (Pothos et al., 2021 ). The second reason is that people differ in their values and weigh evidence differently even if all parties can agree on underlying facts (Walasek and Brown, 2023 ).

Nonetheless, controversies such as those surrounding COMPAS and ACIP’s vaccination policy do not give licence to political actors to obscure the debate through falsehoods, misleading claims, or lies. On the contrary, proper debate of those issues is only possible in the absence of falsehoods because their resolution ultimately requires a trade-off of values that is best arrived at by weighing the importance of different competing sets of evidence. We therefore reject recent academic voices that have questioned whether misinformation can be reliably identified at all (Acerbi et al., 2022 ; Adams et al., 2023 ; Harris, 2022 ; van Doorn, 2023 ; Yee, 2023a , 2023b ). We suggest that its identification is essential and, as we show next, empirically well supported.

The identifiability of misinformation

We place our case into the context of the more extreme end of the academic critique because it involves positions that are antithetical to ours, calling into question the entire idea of fact-checking. For example, Uscinski ( 2015 ) raised the specter that fact-checking is merely a “veiled continuation of politics by means of journalism” (p. 243). Yee ( 2023a ) argued more broadly that any deference to “epistemic elites”—including not only fact-checkers but also academics, researchers, or journalists—is problematic, and assessment of the quality of information should include democratic elements “that are participatory, transparent, and fully negotiable by average citizens” (Yee, 2023a , p. 1111). This demand has several problematic implications. First, it does not explain who counts as “average citizen” and who would belong to the “elite”. At what point should individuals seeking to counter misinformation begin to recuse themselves for fear of accidentally treading on “average” citizens? Is a virologist too “elite” to correct misinformation surrounding the origin of a new virus? What about citizens with a PhD or Master’s degree, how are they being classified? Second, why exactly would one exclude epistemic elites, such as investigative journalists or forensic IT experts, from identifying bad-faith actors such as foreign “bots” or “trolls”? Are average citizens really better at this task than network scientists? Should we decide by social-media poll whether a new strain of avian flu is contagious to humans (Lewandowsky et al., 2017 )? Probably not. There are obviously many domains that benefit from expert assessment of claims.

Nonetheless, there has been much research that has revealed the competence of crowds in the context of fact-checking. For example, Pennycook and Rand ( 2019 ) showed that crowdsourced trust ratings of media outlets were quite successful in the aggregate when compared to ratings by professionals, notwithstanding substantial partisan differences. This basic finding has been replicated and extended several times (M. R. Allen et al., 2024 ; Martel et al., 2024 ), with community-based fact-checking of COVID-19 content being 97% accurate in one study (M. R. Allen et al., 2024 ). Care must, however, be taken that crowds are politically balanced. When people can choose what content to evaluate, as in Twitter/X’s crowdsourced “Birdwatch” fact-checking program (now known as Community Notes), partisan differences among contributors may limit the value of the crowdsourcing (J. Allen et al., 2022 ). The crowdsourcing results show not only that average citizens can match the competence of experts in the aggregate, but they also reaffirm that misinformation is identifiable.

Much recent research has uncovered specific “fingerprints” that can enable people as well as machines to infer the likely quality or accuracy of content. Misinformation has been shown to be suffused with emotions, logical fallacies, and conspiratorial reasoning (Blassnig et al., 2019 ; Carrasco-Farré, 2022 ; Fong et al., 2021 ; Musi et al., 2022 ; Musi and Reed, 2022 ). For example, critical thinking methods offer a qualitative approach to deconstructing arguments in order to identify the presence of reasoning fallacies (Cook et al., 2018 ).

Quantitatively, one study found that compared to reliable information, misinformation is less cognitively complex and 10 times more likely to rely on negative emotional appeals (Carrasco-Farré, 2022 ). In confirmation, numerous other studies show that misinformation is, on average, more emotional than factual information (for a systematic review, see Peng et al., 2023 ) Upward of 75% of anti-vaccination websites use negative emotional appeals (Bean, 2011 ) and linguistic analyses show that conspiracy theorists use significantly more fear-driven language as compared to scientists (Fong et al., 2021 ).

Emotion also plays a role in the receivers’ behavior. People have been shown to be more susceptible to misinformation when put in an emotional state (Martel et al., 2020 ), which helps explain the preferential and more rapid diffusion of unreliable versus reliable information online (Pröllochs et al., 2021 ; Vosoughi et al., 2018 ).

Critics may argue that the datasets used for determining what constitutes “misinformation” and “reliable” information are limited or biased or that the mere prevalence of these cues is not evidence of their diagnosticity in real-world contexts. However, computational machine-learning work relying on a large variety of different URL sources and fact-checked datasets has confirmed that the results are robust and generalizable (Ghanem et al., 2020 ; Kumari et al., 2022 ; Lebernegg et al., 2024 ). A recent comprehensive study which combined many of the available cues found that they have high diagnostic and predictive validity and help discriminate between false and true information, with state-of-the-art models reaching over 83% classification accuracy (Lebernegg et al., 2024 ). Moreover, real-world training on fake news detection, such as logical fallacy training, helps people accurately discriminate between misleading and credible news (e.g., Hruschka and Appel, 2023 ; Lu et al., 2023 ; Roozenbeek et al., 2022 ).

In summary, the available evidence shows quite convincingly that misinformation can be identified by both humans and machines with considerable accuracy. As we show next, we can go beyond mere identification as there are also at least three ways in which one can ascertain the deceptive intent underlying disinformation if present. Identification of deceptive intent is particularly pertinent because it allows information to be safely discounted without requiring a detailed analysis of its factual status.

The identifiability of willful disinformation

For decades, the hallmark of Western news coverage about politicians’ false or misleading claims was an array of circumlocutions that carefully avoided the charge of lying—that is, knowingly telling an untruth with intent to deceive (Lackey, 2013 )—and instead used adverbs such as “falsely”, “wrongly”, “bogus”, or “baseless” when describing a politician’s speech. Other choice phrases referred to “unverified claims” or “repeatedly debunked claims”. This changed in late 2016, when the New York Times first used the word “lie” to characterize an utterance by Donald Trump (Borchers, 2016 ). The paper again referred to Donald Trump’s lies within days of the inauguration in January 2017 (Barry, 2017 ) and it has grown into a routine part of its coverage from then on. Many other mainstream news organizations soon followed suit and it has now become widely accepted practice to refer to Trump’s lies as lies.

Given that lying involves the intentional uttering of false statements, what tools are at our disposal to infer a person’s intention when they utter falsehoods? How can we know a person is lying rather than being confused? How can we infer intentionality?

Anecdotally, defenders of Donald Trump’s lies have raised precisely that objection to the use of the word “lie” in connection with his falsehoods. This objection runs afoul of centuries of legal scholarship and Western jurisprudence. Brown ( 2022 ) argues that inferring intentionality from the evidence is “ordinary and ubiquitous and pervades every area of the law” (p. 2). Inferring intentionality is the difference between manslaughter and murder and is at the heart of the concept of perjury—namely, willfully or knowingly making a false material declaration (Douglis, 2018 ).

There are at least three approaches that can be pursued to infer intentional deception by a communicating agent with varying degrees of confidence. The first approach is statistical and relies on linguistic analysis of material. Unlike people, who are not very good lie detectors despite performing (slightly) above chance (Bond and DePaulo, 2006 ; Mattes et al., 2023 ), recent advances in natural language processing (NLP) have given rise to machine-learning models that can classify texts as deceptive or honest based on subtle linguistic clues (e.g., Braun et al., 2015 ; Davis and Sinnreich, 2020 ; Van Der Zee et al., 2021 ). To illustrate, a model that relied on analysis of the distribution of different types of words achieved 67% accuracy (considerably better than the 52% achieved by human judges) on texts generated by speakers who were either instructed to lie or to be honest. Using the same analysis approach, Davis and Sinnreich ( 2020 ) trained a model to classify tweets by Donald Trump as true or false by using independent fact-checks as ground truth. The model was able to classify tweets with more than 90% accuracy, suggesting that Trump uses subtly different language (e.g., more negative emotion, more prepositions and discrepancies) when communicating untruths. A similar model of Trump’s tweets was developed by Van Der Zee et al. ( 2021 ), who additionally applied 26 extant models from the literature to Trump’s tweets and showed that most of them performed above chance despite being developed on very different materials. In summary, NLP-based approaches have repeatedly shown their value in the classification of speech into honest and deceptive. The fact that those models succeed also when applied to the tweets of Donald Trump implies at the very least that Trump’s falsehoods are not uttered at random or accidentally but are deployed using specific linguistic techniques.

In general, machine-learning approaches to deception detection have shown promise. A recent systematic review identified 81 studies, 19 of which achieved accuracies in excess of 90%, with a further 15 exceeding 80% accuracy (Constâncio et al., 2023 ). The machine-learning models in that ensemble were trained on a variety of corpora, ranging from reviews on Tripadvisor (either true or generated with the intent to deceive; Barsever et al., 2020 ) to segments of a radio game show dedicated to bluff detection by the audience (Papantoniou et al., 2021 ). In all cases, the ground truth (i.e., whether or not deceptive intent was present) was unambiguously known, and the models learned to identify deceptive text based on linguistic analysis with considerable albeit imperfect success.

The second approach to establish willful deception relies on analysis of internal documents of institutions such as governments or corporations. Comparison of the internal knowledge to public stances of the same entities can identify active deception, especially when it is large-scale. Numerous such cases exist, mainly involving corporations and their associated infrastructure such as think-tanks and other front groups (Ceccarelli, 2011 ; Oreskes and Conway, 2010 ). For example, as early as the 1920s, the electricity industry organized a propaganda campaign to falsely insist that private sector electricity was cheaper and more reliable than electricity generated in the public sector (Oreskes and Conway, 2023 ). The tobacco industry’s activities to mislead the public about the dangers from smoking are well documented and established beyond reasonable doubt (e.g., Cataldo et al., 2010 ; Fallin et al., 2013 ; Francey and Chapman, 2000 ; Proctor, 2012 ). The tobacco industry was well aware of the link between smoking and lung cancer in the 1950s and 1960s (Proctor, 2012 ), and yet continued publicly to dispute that medical fact using a variety of propagandistic means (Landman and Glantz, 2009 ; Proctor, 2011 ). Similarly, analysis of internal documents of the fossil-fuel industry has revealed that industry leaders, in particular ExxonMobil, were fully aware of the reality of climate change and its underlying causes (Supran and Oreskes, 2017 , 2021 ) while simultaneously expending large sums to deny its existence in public (J. Farrell, 2016 ) and to prevent Congress from enacting climate-mitigation legislation (Brulle, 2018 ). Ironically, ExxonMobil’s scientists projected global temperatures in the 1970s and 1980s with comparable skill as independent academics at the time (Supran et al., 2023 ). As Baker and Oreskes ( 2017 ) argued, the best explanation for ExxonMobil’s conduct is that they knowingly deceived the public by funding a disinformation machine that denied the realities of climate change. This approach admittedly requires considerable resources and skill, and it is comparatively slow, but in exchange the results it yields are particularly diagnostic and demonstrably useful in litigation. In the case of the tobacco industry, this was the basis for a conviction of Phillip Morris under federal racketeering (RICO) law. The appeals in that case explicitly noted that Phillip Morris intentionally deceived the public and that first-amendment (free speech) rights did not apply as they do not protect fraud or deliberate misrepresentation (Farber et al., 2018 ). In the case of the fossil fuel industry, litigation has not been met with notable success at the time of this writing, but the “Exxon knew” campaign, based on research by Supran and colleagues (Supran et al., 2023 ; Supran and Oreskes, 2017 , 2021 ), has had considerable public impact with 178 relevant media articles identified by Google News. Footnote 5

The final approach to identifying intentional deception resembles the approach involving institutional documents but specifically focuses on lies promulgated by identifiable individuals. We illustrate this approach with Donald Trump’s big lie about the 2020 presidential elections, focusing on statements made in courts of law. Although Trump was making widespread public accusations of fraud, his lawyers—who filed more than 60 lawsuits in connection with the election—did not echo those accusations in court. Quite on the contrary, his lawyers frequently disavowed any mention of fraud in court despite their very different public stance. For example, Rudy Giuliani, one of Trump’s lead attorneys, stood outside a landscaping business on the day most networks declared the election for Biden, and thundered that “It’s [the election] a fraud, an absolute fraud.” Ten days later, being questioned by a federal judge in Pennsylvania during one of Trump’s lawsuits (dealing with whether local election officials in Pennsylvania should have allowed voters to fix problems with their mail-in ballots after submitting them), he declared “This is not a fraud case” (Lerer, 2020 ). This pattern was pervasive: Trump’s lawyers continued to back away from suggestions that the election was stolen and admitted in court that there was no evidence of fraud, all in contradiction to their plaintiff’s public statements (Lerer, 2020 ).

Notwithstanding the careful hedging of their claims in court, the frivolous suits filed on behalf of Trump resulted in sanctions for several of his attorneys. Two lawyers who did claim widespread voter fraud not only had their suit dismissed but were also sanctioned $187,000 by a federal judge in Colorado for their frivolous, meritless case (Polantz, 2021 ). The decision was upheld on appeal, and the Supreme Court declined to hear a further appeal by the lawyers (Scarcella, 2023 ). Altogether, 22 Trump lawyers have been identified who face sanctions in litigation, criminal prosecutions, and state bar disciplinary proceedings. In all cases, what appears to be at issue is violation of the Model Code of Conduct, in particular rules stipulating that claims must be meritorious and that lawyers must exhibit candor and truthfulness (Neff and Fredrickson, 2023 ).

Since the flurry of lawsuits in late 2020, Trump lawyer Sidney Powell has pleaded guilty to charges arising from her involvement in pushing the big lie. Ms Powell pleaded guilty to “conspiracy to commit intentional interference with performance of election duties” and agreed to cooperate with prosecutors in a criminal case against Donald Trump (Fausset and Hakim, 2023 ). Two further Trump lawyers have pleaded guilty in the same case and agreed to testify truthfully about other defendants (Blake, 2023 ).

In a civil suit brought against Rudy Giuliani by two election workers in Georgia, whom he had publicly accused of election fraud, Giuliani conceded before trial that those statements were false (Brumback, 2023 ). The election workers were awarded $148 million in damages, causing Giuliani to file for bankruptcy in late 2023 (Aratani and Oladipo, 2023 ). In a further twist, Giuliani repeated his false claims during the trial outside the court room even while his lawyers conceded in court that they were wrong (Hsu and Weiner, 2023 ).

Giuliani was promptly sued again by the election workers, and at the time of this writing the suit was still under way (Hsu and Weiner, 2023 ).

The big lie was not just curated and pushed by politicians seeking to cling to power and their attorneys. It is now public knowledge that one major news network, Rupert Murdoch’s Fox News, knowingly amplified claims about the election that network executives knew to be false. The fact that Fox lied became apparent during a defamation suit filed by Dominion Voting Machines against the network over false allegations that the voting machines had been rigged to steal the 2020 election. As trial was about to begin, Fox News agreed to pay Dominion $787.5 million and acknowledged that the network had broadcast false statements. The discovery process that preceded trial had uncovered numerous documents and emails that revealed that senior network executives and hosts were convinced that the allegations about the election made by Trump and his allies were untrue (e.g., Peltz, 2023 ; Terkel et al., 2023 ). The network continued to air those allegations and its CEO instructed staff that fact-checking “had to stop” because it was bad for business (Levine, 2023 ). One scholar put it succinctly: “Fox News deliberately misleads the audience for profit” (Nyberg, 2023 , p. 1). Although Fox has been repeatedly implicated in spreading disinformation with harmful consequences for the American public (Ash et al., 2023 ; Bolin and Hamilton, 2018 ; Bursztyn et al., 2020 ; DellaVigna and Kaplan, 2007 ; Feldman et al., 2012 ; Kull et al., 2003 ; Simonov et al., 2022 ), the Dominion case provided a unique opportunity to ascertain that, at least in this case, the network was knowingly lying to its audience.

The preceding examples illustrate the approaches available to establish—with some degree of confidence—the intention to deceive that is the core element of lies. Our examples are not intended to be exhaustive but they illustrate the options available to researchers, journalists, and the public to uncover when they are being lied to. The examples also put to rest several generous auxiliary assumptions that have been made about lies in politics, such as their presumed inevitability because issues can be so nuanced that complete honesty is impossible. Contrary to that assumption, the fact that a person’s rhetoric can differ strikingly between courts of law—where penalties apply for misrepresentations and perjury—and politics—where accountability is notoriously absent—not only reveals the intention to deceive but also the person’s sensitivity to the consequences of their speech.

We have already noted that the contrast between what companies such as ExxonMobil or Philip Morris said in public about their products and what they discussed in private was sufficient to provoke legal consequences. Similar arguments, that fraudulent political speech should not be protected by the First Amendment, have been advanced in the context of Trump’s big lie (Henricksen and Betz, 2023 ).

Although our examination was necessarily limited to a small number of cases, they suffice to illustrate a pathway towards pinpointing intentional disinformation by analysing the utterances of the liars themselves, be they corporations, politicians, or media organizations. We believe that the basic approach is of considerable generality, extending to numerous recorded instances:

Politicians catching themselves lying by changing their story, indicating they were telling an untruth on at least one of those occasions (O’Toole, 2022 , p. 427).

Attorneys of conspiracy theorist Alex Jones—who was sued for his claims that the Sandy Hook massacre never happened by parents of the victims—seeking to defend him by calling him a performance artist who should not be taken seriously (Borchers, 2017 ).

Alex Jones himself admitting in court that the Sandy Hook shooting was “100% real” after having misled millions of people for many years (Associated Press, 2022 ).

Fox News requiring their employees to be vaccinated against COVID-19 or submit to daily testing while the network routinely broadcast anti-vaccination content (Darcy, 2021 ).

Tucker Carlson, former Fox News host, openly admitting that he lies on air (Muzaffar, 2021 ).

Moving forward

Our work explored three fundamental premises: First, that democracy rests on a foundation of common knowledge (H. Farrell and Schneier, 2018 ) and that it is imperiled if citizens cannot agree on basic facts such as the integrity of elections (H. Farrell and Schneier, 2018 ; Tenove, 2020 ). Second, that while democratic debate—including evidence-informed policy-making—often involves contestation of facts (e.g., Kuklinski et al., 1998 ), this does not licence the use of outright lies and propaganda to willfully mislead the public (Lewandowsky, 2020 ). Third, that it is often possible to identify falsehoods, disinformation, and lies and differentiate them from good-faith political and policy-related argumentation.

At the time of this writing, Donald Trump is the Republican nominee for the 2024 presidential election. His campaign has rolled out an explicitly authoritarian agenda for his second term (Arnsdorf and Stein, 2023 ). The authoritarian agenda is likely to result in less free speech, rather than more, which is ironic in light of the fact that people such as Jim Jordan, who are attacking the idea of studying disinformation, do so under the banner of defending the First Amendment. Against this background, the question of how to address Donald Trump’s lies in particular and misinformation in general takes on particular importance.

At the more pessimistic end, Barkho ( 2023 ) posed three questions about the success of fact-checking Trump’s claims: first, have fact-checkers succeeded in persuading Trump to stop disseminating lies? Second, have the long inventories of falsehoods compiled by fact-checkers embarrassed or shamed Trump? Third, has fact-checking changed public perception of what constitutes truth? At first glance, the answer to all three questions might appear to be a resounding “no” (even though the counterfactual is, of course, unknown). However, at the more optimistic end of the spectrum, experimental studies in which election-fraud misinformation was corrected have found positive effects on trust in electoral processes (Bailard et al., 2022 ; Painter and Fernandes, 2022 ), including among Republican respondents and supporters of Trump. Those findings should give rise to a sliver of optimism that even partisans are receptive to corrective messages about election integrity, and therefore underscore the value of disinformation research.

Correcting lies about elections is arguably compatible with the spirit of a democracy. But what is the democratic legitimacy of broader countermeasures against misinformation and disinformation? It is straightforward to explore techniques with which to correct misconceptions in an experiment, in particular if the misinformation is introduced in the experiment itself (e.g., Ecker et al., 2011 ). It is less straightforward to deploy such techniques in the public sphere. Who determines what is “misinformation”, and what is “correct”? And how narrow is the gap between correcting misinformation and banning it? Several countries have recently outlawed “fake news” (e.g., Burkina Faso, Cambodia, Hungary, India, Malaysia, Singapore, and Vietnam) whose democratic credentials are at best questionable. In those cases, fake news can damage democracy not only by disinforming the public but also because countermeasures can be used to curb civil liberties and justify authoritarian crackdowns (Neo, 2022 ; Vese, 2022 ). Indeed, given that Donald Trump has routinely labeled any media coverage he did not like as “fake news”, perhaps the worst response to misinformation would be a law against fake news designed by Donald Trump and his allies.

There are, however, numerous ways in which the public can be better protected by the platforms—in particular if prodded into action by suitable regulations—against disinformation. One avenue involves content moderation and removal of unacceptable or problematic content, such as hate speech. The public is broadly supportive of moderation in certain cases (Kozyreva, Herzog, et al., 2023 ), and the European Union’s recent Digital Services Act (DSA) acknowledges a role for content moderation while highlighting the need for transparency of the underlying rules (for details, see Kozyreva, Smillie, et al., 2023 ). In addition, there are a number of alternative approaches that aim to inform or educate consumers rather than govern content directly. Those approaches have the advantage that they side-step concerns about censorship and that they are demonstrably scalable and readily deployable by the platforms.

One avenue involves the provision of “nutrition labels”, that is, indicators of the quality of a source. Reliable indicators of quality exist that are based on basic journalistic principles (Lin et al., 2023 ), and it is well-known that perceived source credibility can influence misinformation persuasiveness (Nadarevic et al., 2020 ; Prike et al., 2024 ). The effectiveness of source-quality indicators can be enhanced by introducing friction, for example, by requiring users to expend additional clicks to make information visible (L. Fazio, 2020 ; Pillai and Fazio, 2023 ). Naturally, such indicators cannot be perfect, and even sources of widely-acknowledged high quality can also publish dubious content. This makes it important to go beyond credibility and consider alternative approaches, such as those that boost users’ ability to spot deception and enhance their information-discernment skills. This can range from teaching “critical ignoring” (Kozyreva, Wineburg, et al., 2023 ), which enables people to ignore information that is unlikely to warrant expenditure of our limited attention, to psychological inoculation or “prebunking” (Lewandowsky and van der Linden, 2021 ; Roozenbeek et al., 2022 ), which involves refuting a lie in advance by explaining the rhetorical techniques that disinformers use to mislead consumers (e.g., scapegoating, false dichotomies, ad hominem attacks, and so on). Through short “edutainment” videos that are displayed as ads or public-service messages, this approach has been scaled on social media to empower millions of people to spot manipulation techniques (Goldberg, 2023 ). Meta-analyses have affirmed the efficacy of the inoculation approach (Banas and Rains, 2010 ; Lu et al., 2023 ). However, while standard debunking and prebunking interventions promise to be effective regardless of the cultural context in which they are applied (Blair et al., 2024 ; Pereira et al., 2023 ; Porter and Wood, 2021 ; but see Pereira et al., 2022 ), the effects of other interventions such as media-literacy training may be less robust in the Global South (Badrinathan, 2021 ). Some interventions developed and successfully applied in the Global North may also be less suitable in less-developed countries, if for example they target dissemination channels that have limited relevance locally (Badrinathan and Chauchard, 2024 ; de Freitas Melo et al., 2019 ).

Overall, much is now known about various cognitively-inspired countermeasures to correct misinformation or to protect people against being misled in the first place. For further extensive discussion of these countermeasures, see Ecker et al. ( 2022 ) and Kozyreva et al. ( 2024 ). Some of the cognitive science of misinformation has been reflected in European regulatory initiatives, such as the strengthened Code of Practice on Disinformation (Kozyreva, Smillie, et al., 2023 ). In addition, specific evidence-based recommendations for platforms have been developed by Roozenbeek et al. ( 2023 ) and Wardle and Derakhshan ( 2017 ).

Our work has also identified several important questions for future research. We consider the long-term consequences of misinformation on society to be a particularly pressing issue. We have a reasonably good understanding of the individual-level cognitive processes that are engaged when a person is exposed to a single piece of misinformation (Ecker et al., 2022 ). We know very little about the cognitive and social consequences for an individual who is inundated with information of dubious quality for prolonged periods of time. We do not know how societies are affected by epistemic uncertainty and chaos in the long run. Numerous indicators suggest that Western societies, in particular the United States, are ailing (e.g., Lewandowsky et al., 2017 ), but the attribution of those trends to misinformation or epistemic chaos is difficult. On those occasions where researchers have successfully isolated causal effects, they tend to implicate certain media organs (e.g., Fox News in particular) in compromising public health (Bursztyn et al., 2020 ; Simonov et al., 2020 ), and they have identified the role of social media in causing ethnic hate crimes and xenophobia (Bursztyn et al., 2019 ; Müller and Schwarz, 2021 ). However, it is unclear as yet how generalizable those findings are and much additional work remains to be done (for a review, see Lorenz-Spreen et al., 2022 ).

Future research should also address some of the limitations of fact-checking, such as the difficulties of verifying statements about the future (Nieminen and Sankari, 2021 ) or arguments that employ the rhetorical technique of “paltering” — that is, the use of truthful statements to convey a misleading impression (Lewandowsky et al., 2016 ; Rogers et al., 2017 ). One approach is to focus on what is pragmatically useful for people to make informed decisions, such as whether a claim is misleading (Birks, 2019 ), with critical thinking methods offering a means of identifying the presence of logical fallacies (Cook et al., 2018 ).

Increasing research attention is being paid to the concept of discernment; that is, the extent to which accurate misinformation is believed more than misinformation (Pennycook and Rand, 2021 ). Focusing on discernment rather than acceptance of misinformation guards against inadvertently developing interventions that reduce belief in facts and misinformation equally. A general cynicism and disbelief of everything does not solve the misinformation problem. Instead, we must boost people’s ability to distinguish between facts and falsehoods.

We began the paper with a quote from Hannah Arendt, one of the foremost analysts of 20th century totalitarianism. It is worth here revisiting the same quotation in its extended form, which underscores the urgency of finding a solution to the epistemic crisis affecting democracy in the U.S. and beyond:

“If everybody always lies to you, the consequence is not that you believe the lies, but rather that nobody believes anything any longer…. And a people that no longer can believe anything cannot make up its mind. It is deprived not only of its capacity to act but also of its capacity to think and to judge. And with such a people you can then do what you please .” (our emphasis)

Further detailed debunkings of election disinformation are provided by the Cybersecurity and Infrastructure Security Agency at https://www.cisa.gov/topics/election-security/rumor-vs-reality .

We focus here on the activities of Jim Jordan because he is the acknowledged leader of a political counter movement aimed at misinformation research. This must not be taken to imply that Jordan is the only political actor involved in this effort.

https://www.nationalacademies.org/documents/embed/link/LF2255DA3DD1C41C0A42D3BEF0989ACAECE3053A6A9B/file/DC4CDD2AC5D4B2DB08255A7EA6244AA9D7CA6F951C22?noSaveAs=1

One ruling that was initially in Trump’s favor was later overturned by the Pennsylvania Supreme Court. Canon and Sherman ( 2021 ) provide a list of cases.

Search conducted on 10 April 2024.

Acerbi A, Altay S, Mercier H (2022) Research note: Fighting misinformation or fighting for information? Harv Kennedy School Misinform Rev 3. https://doi.org/10.37016/mr-2020-87

Adams Z, Osman M, Bechlivanidis C, Meder B (2023) (Why) Is Misinformation a Problem? Perspect Psychol Sci 17456916221141344. https://doi.org/10.1177/17456916221141344

Agiesta J, Edwards-Levy A (2023) CNN poll: Percentage of Republicans who think Biden’s 2020 win was illegitimate ticks back up near 70%. CNN. https://edition.cnn.com/2023/08/03/politics/cnn-poll-republicans-think-2020-election-illegitimate/index.html

Allen J, Martel C, Rand DG (2022) Birds of a feather don’t fact-check each other: Partisanship and the evaluation of news in Twitter’s Birdwatch crowdsourced fact-checking program. CHI Conf Hum Factors Comp Syst 1–19. https://doi.org/10.1145/3491102.3502040

Allen MR, Desai N, Namazi A, Leas E, Dredze M, Smith DM, Ayers JW (2024) Characteristics of X (formerly Twitter) Community Notes addressing COVID-19 vaccine misinformation. JAMA 331:1670. https://doi.org/10.1001/jama.2024.4800

Article   PubMed   Google Scholar  

Angwin J, Larson J, Mattu S, Kirchner L (2016) Machine bias: There’s software used across the country to predict future criminals and it’s biased against blacks. ProPublica. https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing

Aratani L, Oladipo G (2023) Giuliani files for bankruptcy after judge rules Georgia election workers can collect $148m. The Guardian. https://www.theguardian.com/us-news/2023/dec/21/giuliani-148-million-damages-georgia-lawsuit

Arceneaux K, Truex R (2022) Donald Trump and the Lie. Perspect Polit 1–17. https://doi.org/10.1017/S1537592722000901

Arnsdorf I, Stein J (2023) Trump touts authoritarian vision for second term: ‘I am your justice’. Washington Post. https://www.washingtonpost.com/elections/2023/04/21/trump-agenda-policies-2024/

Arun C (2019) On WhatsApp, rumours, and lynchings. Econ Polit Wkly 54(6):30–35

Google Scholar  

Ash E, Galletta S, Hangartner D, Margalit Y, Pinna M (2023) The effect of Fox News on health behavior during COVID-19. Polit Anal 1–10. https://doi.org/10.1017/pan.2023.21

Associated Press (2022) Alex Jones concedes that the Sandy Hook attack was ’100% real’. NPR. https://www.npr.org/2022/08/03/1115414563/alex-jones-sandy-hook-case

Badrinathan S (2021) Educative interventions to combat misinformation: evidence from a field experiment in India. Am Polit Sci Rev 1–17. https://doi.org/10.1017/S0003055421000459

Badrinathan S, Chauchard S (2024) Researching and countering misinformation in the Global South. Curr Opin Psychol 55:101733. https://doi.org/10.1016/j.copsyc.2023.101733

Bailard, CS, Porter, E, & Gross, K (2022). Fact-checking Trump’s election lies can improve confidence in U.S. elections: Experimental evidence. Harv Kennedy School Misinform Rev. https://doi.org/10.37016/mr-2020-109

Baker E, Oreskes N (2017) Science as a game, marketplace or both: a reply to Steve Fuller. Soc Epistemol Rev Reply Collect 6:65–69

Banas JA, Rains SA (2010) A meta-analysis of research on inoculation theory. Commun Monogr 77:281–311

Article   Google Scholar  

Barkho L (2023) A critical inquiry into US media’s fact-checking and compendiums of Donald Trump’s falsehoods and “lies”. In A Akande (Ed.) The perils of populism: The end of the American century? (pp. 259–278). Springer

Barrett PM, Sims JG (2021) False accusation: The unfounded claim that social media companies censor conservatives (tech. rep.). New York University Stern Center for Business and Human Rights

Barry D (2017). In a swirl of ‘untruths’ and ‘falsehoods,’ calling a lie a lie. New York Times. https://www.nytimes.com/2017/01/25/business/media/donald-trump-lie-media.html

Barsever D, Singh S, Neftci E (2020) Building a better lie detector with BERT: The difference between truth and lies. 2020 International Joint Conference on Neural Networks (IJCNN). https://doi.org/10.1109/ijcnn48605.2020.9206937

Bean SJ (2011) Emerging and continuing trends in vaccine opposition website content. Vaccine 29:1874–1880. https://doi.org/10.1016/j.vaccine.2011.01.003

Benkler Y, Faris R, Roberts H (2018) Network propaganda: Manipulation, disinformation, and radicalization in American politics. Oxford University Press

Berk R, Heidari H, Jabbari S, Kearns M, Roth A (2021) Fairness in criminal justice risk assessments: The state of the art. Sociol Methods Res 50:3–44. https://doi.org/10.1177/0049124118782533

Article   MathSciNet   Google Scholar  

Berlinski N, Doyle M, Guess AM, Levy G, Lyons B, Montgomery JM, Nyhan B, Reifler J (2021) The effects of unsubstantiated claims of voter fraud on confidence in elections. J Exp Polit Sci 10(1), 34–49. https://doi.org/10.1017/xps.2021.18

Bernstein A (2023) Republican Rep. Jim Jordan issues sweeping information requests to universities researching disinformation. Pro Publica. https://www.propublica.org/article/jim-jordan-disinformation-subpoena-universities

Birks J (2019) Fact-checking journalism and political argumentation: A British perspective. Palgrave Macmillan. https://doi.org/10.1007/978-3-030-30573-4

Blair RA, Gottlieb J, Nyhan B, Paler L, Argote P, Stainfield CJ (2024) Interventions to counter misinformation: Lessons from the Global North and applications to the Global South. Curr Opin Psychol 55:101732. https://doi.org/10.1016/j.copsyc.2023.101732

Blake A (2023) Jenna Ellis’s tearful guilty plea should worry Rudy Giuliani. Washington Post. https://www.washingtonpost.com/politics/2023/10/24/jenna-ellis-guilty-plea-georgia-giuliani-trump/

Blassnig S, Büchel F, Ernst N, Engesser S (2019) Populism and informal fallacies: an analysis of right-wing populist rhetoric in election campaigns. Argumentation 33:107–136. https://doi.org/10.1007/s10503-018-9461-2

Blitzer J (2023) Jim Jordan’s conspiratorial quest for power. The New Yorker. https://www.newyorker.com/magazine/2023/10/30/jim-jordans-conspiratorial-quest-for-power

Bolin JL, Hamilton LC (2018) The news you choose: News media preferences amplify views on climate change. Environ Polit. https://doi.org/10.1080/09644016.2018.1423909

Bond CF, DePaulo BM (2006) Accuracy of deception judgments. Personal Soc Psychol Rev 10:214–234. https://doi.org/10.1207/s15327957pspr1003_2

Borchers C (2016) Why the New York Times decided it is now okay to call Donald Trump a liar. Washington Post. https://www.washingtonpost.com/news/the-fix/wp/2016/09/22/why-the-new-york-times-decided-it-is-now-okay-to-call-donald-trump-a-liar/

Borchers C (2017) Alex Jones should not be taken seriously, according to Alex Jones’s lawyers. Washington Post. https://www.washingtonpost.com/news/the-fix/wp/2017/04/17/trump-called-alex-jones-amazing-joness-own-lawyer-calls-him-a-performance-artist/

Braun MT, Swol LMV, Vang L (2015) His lips are moving: Pinocchio effect and other lexical indicators of political deceptions. Discourse Process 52:1–20. https://doi.org/10.1080/0163853X.2014.942833

Article   ADS   Google Scholar  

Brown TR (2022) Demystifying mindreading for the law. Wisconsin Law Review Forward, 1–11

Brulle RJ (2018) The climate lobby: A sectoral analysis of lobbying spending on climate change in the USA, 2000 to 2016. Climatic Change. https://doi.org/10.1007/s10584-018-2241-z

Brumback K (2023) Giuliani concedes he made public comments falsely claiming Georgia election workers committed fraud. Associated Press. https://apnews.com/article/giuliani-georgia-election-workers-lawsuit-false-statements-afc64a565ee778c6914a1a69dc756064

Bursztyn L, Egorov G, Enikolopov R, Petrova M (2019) Social media and xenophobia: Evidence from Russia (tech. rep.). National Bureau of Economic Research. https://doi.org/10.3386/w26567

Bursztyn L, Rao A, Roth C, Yanagizawa-Drott D (2020) Misinformation during a pandemic (tech. rep.). National Bureau of Economic Research. https://doi.org/10.3386/w27417

Canon DT, Sherman O (2021) Debunking the “Big Lie”: Election Administration in the 2020 Presidential Election. Pres Stud Q 51:546–581. https://doi.org/10.1111/psq.12721

Carrasco-Farré C (2022) The fingerprints of misinformation: How deceptive content differs from reliable sources in terms of cognitive effort and appeal to emotions. Hum Soc Sci Commun 9:1–18. https://doi.org/10.1057/s41599-022-01174-9

Cataldo JK, Bero LA, Malone RE (2010) “A delicate diplomatic situation”: Tobacco industry efforts to gain control of the Framingham study. J Clin Epidemiol 63:841–853. https://doi.org/10.1016/j.jclinepi.2010.01.021

Article   PubMed   PubMed Central   Google Scholar  

Ceccarelli L (2011) Manufactured scientific controversy: Science, rhetoric, and public debate. Rhetor Public Aff 14:195–228

Constâncio AS, Tsunoda DF, Silva HDFN, Silveira JMD, Carvalho DR (2023) Deception detection with machine learning: a systematic review and statistical analysis. PLoS One 18:e0281323. https://doi.org/10.1371/journal.pone.0281323

Article   CAS   PubMed   PubMed Central   Google Scholar  

Cook J, Ellerton P, Kinkead D (2018) Deconstructing climate misinformation to identify reasoning errors. Environ Res Lett 13:024018

Cooley A, Nexon DH (2022) The real crisis of global order: Illiberalism on the rise. Foreign Aff 101:103–118

Darcy O (2021) Fox has quietly implemented its own version of a vaccine passport while its top personalities attack them. CNN. https://edition.cnn.com/2021/07/19/media/fox-vaccine-passport/index.html

Davis D, Sinnreich A (2020) Beyond fact-checking: Lexical patterns as lie detectors in Donald Trump’s tweets. Int J Commun 14:5237–5260

de Freitas Melo P, Vieira CC, Garimella K, de Melo POV, Benevenuto F (2019) Can WhatsApp counter misinformation by limiting message forwarding? International Conference on Complex Networks and Their Applications, 372–384. https://doi.org/10.1007/978-3-030-36687-2_31

DellaVigna S, Kaplan E (2007) The fox news effect: media bias and voting. Q J Econ 122:1187–1234

Desikan A, MacKinney T, Kalman C, Carter JM, Reed G, Goldman GT (2023) An equity and environmental justice assessment of anti-science actions during the Trump administration. J Public Health Policy 44:147–162. https://doi.org/10.1057/s41271-022-00390-6

Dieterich W, Mendoza C, Brennan T (2016) COMPAS risk scales: Demonstrating accuracy equity and predictive parity. (tech. rep.). Northpoint, Inc

Dixit P, Mac R (2018) How WhatsApp destroyed a village. BuzzFeed News. https://www.buzzfeednews.com/article/pranavdixit/whatsapp-destroyed-village-lynchings-rainpada-india

Douglis A (2018) Disentangling perjury and lying. Yale J Law Hum 29:339–374

Dourado T, Salgado S (2021) Disinformation in the Brazilian pre-election context: Probing the content, spread and implications of fake news about Lula da Silva. Commun Rev 24:297–319. https://doi.org/10.1080/10714421.2021.1981705

Dressel J, Farid H (2018) The accuracy, fairness, and limits of predicting recidivism. Sci Adv 4:eaao5580

Article   ADS   PubMed   PubMed Central   Google Scholar  

Ecker UKH, Lewandowsky S, Apai J (2011) Terrorists brought down the plane!—No, actually it was a technical fault: Processing corrections of emotive information. Q J Exp Psychol 64:283–310. https://doi.org/10.1080/17470218.2010.497927

Ecker UKH, Lewandowsky S, Cook J, Schmid P, Fazio LK, Brashier N, Kendeou P, Vraga EK, Amazeen MA (2022) The psychological drivers of misinformation belief and its resistance to correction. Nat Rev Psychol 1:13–29. https://doi.org/10.1038/s44159-021-00006-y

Eggers AC, Garro H, Grimmer J (2021) No evidence for systematic voter fraud: a guide to statistical claims about the 2020 election. Proc Natl Acad Sci USA 118:e2103619118. https://doi.org/10.1073/pnas.2103619118

Enders A, Farhart C, Miller J, Uscinski J, Saunders K, Drochon H (2022) Are republicans and conservatives more likely to believe conspiracy theories? Polit Behav 1–24. https://doi.org/10.1007/s11109-022-09812-3

Enders AM, Uscinski JE (2021) Are misinformation, antiscientific claims, and conspiracy theories for political extremists? Group Processes & Intergroup Relations

Fallin A, Grana R, Glantz SA (2013) ‘to quarterback behind the scenes, third-party efforts’: the tobacco industry and the tea party. Tob Control 0:1–10. https://doi.org/10.1136/tobaccocontrol-2012-050815

Farber HJ, Neptune ER, Ewart GW (2018) Corrective statements from the tobacco industry: more evidence for why we need effective tobacco control. Ann Am Thorac Soc 15:127–130. https://doi.org/10.1513/annalsats.201711-845gh

Farrell H, Schneier B (2018) Common-knowledge attacks on democracy (tech. rep.). Berkman Klein Center for Internet & Society

Farrell J (2016) Network structure and influence of the climate change counter-movement. Nat Clim Change 6:370–374. https://doi.org/10.1038/nclimate2875

Fausset R, Hakim D (2023) Sidney Powell pleads guilty in Georgia Trump case. New York Times. https://www.nytimes.com/2023/10/19/us/sidney-powell-guilty-plea-trump-georgia.html

Fazio LK, Brashier NM, Payne BK, Marsh EJ (2015) Knowledge does not protect against illusory truth. J Exp Psychol General. https://doi.org/10.1037/xge0000098

Fazio L (2020) Pausing to consider why a headline is true or false can help reduce the sharing of false news. Harv Kennedy School Misinform Rev. https://doi.org/10.37016/mr-2020-009

Feldman L, Maibach EW, Roser-Renouf C, Leiserowitz A (2012) Climate on cable: the nature and impact of global warming coverage on Fox News, CNN, and MSNBC. Int J Press/Polit 17:3–31

Field H, Vanian J (2023) Tech layoffs ravage the teams that fight online misinformation and hate speech. CNBC. https://www.cnbc.com/2023/05/26/tech-companies-are-laying-off-their-ethics-and-safety-teams-.html

Fong A, Roozenbeek J, Goldwert D, Rathje S, van der Linden S (2021) The language of conspiracy: a psychological analysis of speech used by conspiracy theorists and their followers on Twitter. Group Process Intergroup Relat 24:606–623. https://doi.org/10.1177/1368430220987596

Francey N, Chapman S (2000) “operation Berkshire”: the international tobacco companies’ conspiracy. Br Med J 321:371–374. https://doi.org/10.1136/bmj.321.7257.371

Article   CAS   Google Scholar  

Garrett RK, Bond RM (2021) Conservatives’ susceptibility to political misperceptions. Sci Adv 7(23):eabf1234. https://doi.org/10.1126/sciadv.abf1234

Ghanem B, Rosso P, Rangel F (2020) An emotional analysis of false information in social media and news articles. ACM Trans Internet Technol 20:19:1–19:18. https://doi.org/10.1145/3381750

Goldberg B (2023) Defanging disinformation’s threat to Ukrainian refugees. Jigsaw. https://medium.com/jigsaw/defanging-disinformations-threat-to-ukrainian-refugees-b164dbbc1c60

González-Bailón S, Lazer D, Barberá P, Zhang M, Allcott H, Brown T, Crespo-Tenorio A, Freelon D, Gentzkow M, Guess AM, Iyengar S, Kim YM, Malhotra N, Moehler D, Nyhan B, Pan J, Rivera CV, Settle J, Thorson E, Tucker JA (2023) Asymmetric ideological segregation in exposure to political news on Facebook. Science 381:392–398. https://doi.org/10.1126/science.ade7138

Article   ADS   CAS   PubMed   Google Scholar  

Graham MH, Yair O (2023) Expressive responding and trump’s big lie. Polit Behav. https://doi.org/10.1007/s11109-023-09875-w

Greene KT (2024) Partisan differences in the sharing of low-quality news sources by U.S. political elites. Polit Commun 1–20. https://doi.org/10.1080/10584609.2024.2306214

Grice HP (1975) Logic and conversation. In P Cole & JL Morgan (Eds.), Syntax and semantics, vol. 3: Speech acts (pp. 41–58). Academic Press

Grinberg N, Joseph K, Friedland L, Swire-Thompson B, Lazer D (2019) Fake news on Twitter during the 2016 U.S. presidential election. Science 363:374–378. https://doi.org/10.1126/science.aau2706

Grofman B, Cervas J (2023) Statistical fallacies in claims about ‘massive and widespread fraud’ in the 2020 presidential election: examining claims based on aggregate election results 1,2. Stat Public Policy 1–36. https://doi.org/10.1080/2330443X.2023.2289529

Guess AM, Nyhan B, Reifler J (2020a) Exposure to untrustworthy websites in the 2016 U.S. election. Nat Hum Behav 4:472–480. https://doi.org/10.1038/s41562-020-0833-x

Guess AM, Lockett D, Lyons B, Montgomery JM, Nyhan B, Reifler J (2020b) “Fake news” may have limited effects on political participation beyond increasing beliefs in false claims. Harv Kennedy School Misinform Rev 1(1). https://doi.org/10.37016/mr-2020-004

Guess AM, Nagler J, Tucker J (2019) Less than you think: prevalence and predictors of fake news dissemination on Facebook. Sci Adv 5:eaau4586. https://doi.org/10.1126/sciadv.aau4586

Harris KR (2022) Real fakes: The epistemology of online misinformation. Philos Technol 35. https://doi.org/10.1007/s13347-022-00581-9

Henricksen W, Betz B (2023) The stolen election lie and the freedom of speech. Penn State Law Review. https://doi.org/10.2139/ssrn.4354211

Hotez P (2023) Anti-science conspiracies pose new threats to US biomedicine in 2023. FASEB BioAdvances. https://doi.org/10.1096/fba.2023-00032

Hruschka TMJ, Appel M (2023) Learning about informal fallacies and the detection of fake news: An experimental intervention. PLoS One 18:e0283238

Hsu SS, Weiner R (2023) Defamed Georgia poll workers who won $148M from Giuliani sue him again. Washington Post. https://www.washingtonpost.com/dc-md-va/2023/12/18/giuliani-defamation-lawsuit-georgia/

Hurley L (2023) Supreme Court blocks restrictions on Biden administration efforts to get platforms to remove social media posts. NBC News. https://www.nbcnews.com/politics/supreme-court/supreme-court-blocks-biden-social-media-curbs-rcna105785

Huszár F, Ktena SI, O’Brien C, Belli L, Schlaikjer A, Hardt M (2022) Algorithmic amplification of politics on Twitter. Proc Natl Acad Sci 119:e2025334119. https://doi.org/10.1073/pnas.2025334119

Article   CAS   PubMed   Google Scholar  

Jacobson GC (2021) Donald Trump’s big lie and the future of the republican party. Pres Stud Q 51:273–289. https://doi.org/10.1111/psq.12716

Jacobson GC (2023) The dimensions, origins, and consequences of belief in Donald Trump’s Big Lie. Polit Sci Q 138:133–166. https://doi.org/10.1093/psquar/qqac030

Jalli N, Idris I (2019) Fake news and elections in two Southeast Asian nations: A comparative study of Malaysia general election 2018 and Indonesia presidential election 2019. Proceedings of the International Conference of Democratisation in Southeast Asia (ICDeSA 2019). https://doi.org/10.2991/icdesa-19.2019.30

Jung Y, Lee S (2023) Trump vs. the GOP: Political Determinants of COVID-19 Vaccination. Polit Behav. https://doi.org/10.1007/s11109-023-09882-x

Kellow CL, Steeves HL (1998) The role of radio in the Rwandan genocide. https://doi.org/10.1111/j.1460-2466.1998.tb02762.x

Kinser S (2020) Science in an age of scrutiny: How scientists can respond to criticism and personal attacks. Union of Concerned Scientists. https://www.ucsusa.org/sites/default/files/2020-09/science-in-an-age-of-scrutiny-2020.pdf

Kozyreva A, Herzog SM, Lewandowsky S, Hertwig R, Lorenz-Spreen P, Leiser M, Reifler J (2023) Resolving content moderation dilemmas between free speech and harmful misinformation. Proc Natl Acad Sci USA 120:e2210666120. https://doi.org/10.1073/pnas.2210666120

Kozyreva A, Lorenz-Spreen P, Herzog SM, Ecker UKH, Lewandowsky S, Hertwig R, Ali A, Bak-Coleman JB, Barzilai S, Basol M, Berinsky A, Betsch C, Cook J, Fazio LK, Geers M, Guess AM, Huang H, Larreguy H, Maertens R, … Wineburg S (2024) Toolbox of interventions against online misinformation. Nat Hum Behav. https://doi.org/10.31234/osf.io/x8ejt

Kozyreva A, Smillie L, Lewandowsky S (2023) Incorporating psychological science into policy making. Eur Psychol 28:206–224. https://doi.org/10.1027/1016-9040/a000493

Kozyreva A, Wineburg S, Lewandowsky S, Hertwig R (2023) Critical ignoring as a core competence for digital citizens. Curr Dir Psychol Sci 32:81–88. https://doi.org/10.1177/09637214221121570

Kuklinski JH, Quirk PJ, Schwieder DW, Rich RF (1998) “Just the facts, ma’am”: political facts and public opinion. Ann Am Acad Political Soc Sci 560:143–154. https://doi.org/10.1177/0002716298560001011

Kull S, Ramsay C, Lewis E (2003) Misperceptions, the media, and the Iraq war. Political Sci Q 118:569–598

Kumari R, Ashok N, Ghosal T, Ekbal A (2022) What the fake? Probing misinformation detection standing on the shoulder of novelty and emotion. Inf Process Manag 59:102740. https://doi.org/10.1016/j.ipm.2021.102740

Lackey J (2013) Lies and deception: an unhappy divorce. Analysis. https://doi.org/10.1093/analys/ant006

Lagioia F, Rovatti R, Sartor G (2023) Algorithmic fairness through group parities? The case of COMPAS-SAPMOC. AI & SOCIETY, 38, 459–478. https://doi.org/10.1007/s00146-022-01441-y

Landman A, Glantz SA (2009) Tobacco industry efforts to undermine policy-relevant research. Am J Public Health 99:45–58. https://doi.org/10.2105/AJPH.2004.050963

Lasser J, Aroyehun ST, Simchon A, Carrella F, Garcia D, Lewandowsky S (2022) Social media sharing of low quality news sources by political elites. PNAS Nexus, pgac186. https://doi.org/10.1093/pnasnexus/pgac186

Latour B (2004) Why has critique run out of steam? From matters of fact to matters of concern. Crit Inq 30:225–248

Lebernegg N, Eberl J-M, Tolochko P, Boomgaarden H (2024) Do you speak disinformation? Computational detection of deceptive news-like content using linguistic and stylistic features. Digit J. https://doi.org/10.1080/21670811.2024.2305792

Leonhardt D (2021) Red Covid. New York Times. https://www.nytimes.com/2021/09/27/briefing/covid-red-states-vaccinations.html

Lerer L (2020) Giuliani in public: ‘it’s a fraud.’ Giuliani in court: ‘This is not a fraud case.’ New York Times https://www.nytimes.com/2020/11/18/us/politics/trump-giuliani-voter-fraud.html

Levine S (2023) Angry Fox News chief said fact-checks of Trump’s election lies ‘bad for business’. The Guardian. https://www.theguardian.com/media/2023/mar/29/fox-news-trump-fact-check-election-lies-dominion

Lewandowsky S (2020) Willful construction of ignorance: A tale of two ontologies. In R Hertwig & C Engel (Eds.), Deliberate ignorance: Choosing not to know (pp. 101–117). MIT Press

Lewandowsky S, Ballard T, Oberauer K, Benestad R (2016) A blind expert test of contrarian claims about climate data. Glob Environ Change 39:91–97. https://doi.org/10.1016/j.gloenvcha.2016.04.013

Lewandowsky S, Ecker UKH, Cook J (2017) Beyond misinformation: understanding and coping with the post-truth era. J Appl Res Mem Cogn 6:353–369. https://doi.org/10.1016/j.jarmac.2017.07.008

Lewandowsky S, Kalish ML, Ngang S (2002) Simplified learning in complex situations: Knowledge partitioning in function learning. J Exp Psychol Gen 131:163–193. https://doi.org/10.1037/0096-3445.131.2.163

Lewandowsky S, Robertson RE, DiResta R (2023a) Challenges in understanding human-algorithm entanglement during online information consumption. Perspect Psychol Sci. https://doi.org/10.1177/17456916231180809

Lewandowsky S, Stritzke WGK, Freund AM, Oberauer K, Krueger JI (2013) Misinformation, disinformation, and violent conflict: From Iraq and the “War on Terror” to future threats to peace. Am Psychol 68:487–501. https://doi.org/10.1037/a0034515

Lewandowsky S (2022) Fake news and participatory propaganda. In R Pohl (Ed.), Cogn illusions (pp. 324–340). Routledge https://doi.org/10.4324/9781003154730-23

Lewandowsky S, Ecker UKH, Cook J, van der Linden S, Roozenbeek J, Oreskes N (2023b) Misinformation and the epistemic integrity of democracy. Curr Opin Psychol 101711. https://doi.org/10.1016/j.copsyc.2023.101711

Lewandowsky S, Pomerantsev P (2022) Technology and democracy: a paradox wrapped in a contradiction inside an irony. Memory Mind Media 1. https://doi.org/10.1017/mem.2021.7

Lewandowsky S, van der Linden S (2021) Countering misinformation and fake news through inoculation and prebunking. Eur Rev Soc Psychol 32:348–384. https://doi.org/10.1080/10463283.2021.1876983

Li D (2004) Echoes of violence: considerations on radio and genocide in Rwanda. J Genocide Res 6:9–27. https://doi.org/10.1080/1462352042000194683

Lin H, Lasser J, Lewandowsky S, Cole R, Gully A, Rand DG, Pennycook G (2023) High level of correspondence across different news domain quality rating sets. PNAS Nexus 2:pgad286. https://doi.org/10.1093/pnasnexus/pgad286

Lorenz-Spreen P, Oswald L, Lewandowsky S, Hertwig R (2022) A systematic review of worldwide causal and correlational evidence on digital media and democracy. Nat Hum Behav 1–28. https://doi.org/10.1038/s41562-022-01460-1

Lu C, Hu B, Li Q, Bi C, Ju X-D (2023) Psychological inoculation for credibility assessment, sharing intention, and discernment of misinformation: systematic review and meta-analysis. J Med Internet Res 25:e49255. https://doi.org/10.2196/49255

Martel C, Allen J, Pennycook G, Rand DG (2024) Crowds can effectively identify misinformation at scale. Perspect Psychol Sci 19:477–488. https://doi.org/10.1177/17456916231190388

Martel C, Pennycook G, Rand DG (2020) Reliance on emotion promotes belief in fake news. Cogn Res Princ Implic 5:47. https://doi.org/10.1186/s41235-020-00252-3

Mattes K, Popova V, Evans JR (2023) Deception detection in politics: can voters tell when politicians are lying. Polit Behav 45:395–418. https://doi.org/10.1007/s11109-021-09747-1

McGraw KM (1998) Manipulating public opinion with moral justification. Ann Am Acad Political Soc Sci 560:129–142. https://doi.org/10.1177/0002716298560001010

McIntyre L (2018) Post-truth. MIT Press

McLauchlin T (2023) Tail risks for 2024: Prospects for a violent constitutional crisis in the United States (tech. rep. No. 28). Network for Strategic Analysis, Queen’s University, Canada

Mounk Y (2023) The identity trap. Penguin Random House

Müller K, Schwarz C (2021) Fanning the flames of hate: social media and hate crime. J Eur Econ Assoc 19:2131–2167. https://doi.org/10.1093/jeea/jvaa045

Musi E, Aloumpi M, Carmi E, Yates S, O’Halloran K (2022) Developing fake news immunity: Fallacies as misinformation triggers during the pandemic. Online J Commun Media Technol 12:e202217. https://doi.org/10.30935/ojcmt/12083

Musi E, Reed C (2022) From fallacies to semi-fake news: improving the identification of misinformation triggers across digital media. Discourse Soc 33:349–370. https://doi.org/10.1177/09579265221076609

Muzaffar M (2021) Tucker Carlson admits he lies on his show: ‘I really try not to… [but] I certainly do’. The Independent. https://www.independent.co.uk/news/world/americas/tucker-carlson-fox-news-dave-rubin-b1919738.html

Nadarevic L, Reber R, Helmecke AJ, Köse D (2020) Perceived truth of statements and simulated social media postings: An experimental investigation of source credibility, repeated exposure, and presentation format. Cogn Res Princ Implic 5. https://doi.org/10.1186/s41235-020-00251-4

Nan X, Wang Y, Thier K (2022) Why do people believe health misinformation and who is at risk? A systematic review of individual differences in susceptibility to health misinformation. Soc Sci Med 314:115398. https://doi.org/10.1016/j.socscimed.2022.115398

Neff A, Fredrickson C (2023) Trump’s lawyers face sanctions, discipline, and indictment – how should the legal profession respond? Just Security. https://www.justsecurity.org/90509/trumps-lawyers-face-sanctions-discipline-and-indictment-how-should-the-legal-profession-respond/

Neo R (2022) A cudgel of repression: analysing state instrumentalisation of the ‘fake news’ label in Southeast Asia. Journalism 23:1919–1938. https://doi.org/10.1177/1464884920984060

Nieminen S, Sankari V (2021) Checking PolitiFact’s fact-checks. J Stud 22:358–378. https://doi.org/10.1080/1461670x.2021.1873818

Nix N, Menn J (2023) These academics studied falsehoods spread by Trump. Now the GOP wants answers. Washington Post. https://www.washingtonpost.com/technology/2023/06/06/disinformation-researchers-congress-jim-jordan/

Nix N, Zakrzewski C, Menn J (2023) Misinformation research isbuckling under GOP legal attacks. Washington Post. https://www.washingtonpost.com/technology/2023/09/23/online-misinformation-jim-jordan/

Nyberg D (2023) The passive revolution is televised: The dominant ideology of media capitalism. Organization, 13505084231180288. https://doi.org/10.1177/13505084231180288

Ognyanova K, Lazer D, Robertson RE, Wilson C (2020) Misinformation in action: Fake news exposure is linked to lower trust in media, higher trust in government when your side is in power. Harv Kennedy School Misinform Rev. https://doi.org/10.37016/mr-2020-024

Oreskes N, Conway EM (2010) Merchants of doubt. Bloomsbury Publishing

Oreskes N, & Conway EM (2023) The big myth. New York City: Bloomsbury Publishing

O’Toole F (2022) We don’t know ourselves: A personal history of Ireland since 1958. Head of Zeus

Painter DL, Fernandes J (2022) “the big lie”: How fact checking influences support for insurrection. Am Behav Sci 000276422211031. https://doi.org/10.1177/00027642221103179

Papantoniou K, Papadakos P, Patkos T, Flouris G, Androutsopoulos I, Plexousakis D (2021) Deception detection in text and its relation to the cultural dimension of individualism/collectivism. Nat Lang Eng 28:545–606. https://doi.org/10.1017/s1351324921000152

Peltz M (2023) New details in Dominion suit reveal damning evidence of deception in Fox News’ 2020 election coverage. Mediamatters. https://www.mediamatters.org/foxdominion-lawsuit/new-details-dominion-suit-reveal-damning-evidence-deception-fox-news-2020

Peng W, Lim S, Meng J (2023) Persuasive strategies in online health misinformation: a systematic review. Inf Commun Soc 26:2131–2148. https://doi.org/10.1080/1369118X.2022.2085615

Pennycook G, Rand DG (2021) Research note: Examining false beliefs about voter fraud in the wake of the 2020 presidential election. Harvard Kennedy School (HKS) Misinform Rev 2 . https://doi.org/10.37016/mr-2020-51

Pennycook G, Rand DG (2019) Fighting misinformation on social media using crowdsourced judgments of news source quality. Proc Natl Acad Sci USA. https://doi.org/10.1073/pnas.1806781116

Pereira FB, Bueno NS, Nunes F, Pavão N (2022) Fake news, fact checking, and partisanship: the resilience of rumors in the 2018 brazilian elections. J Polit 84:2188–2201. https://doi.org/10.1086/719419

Pereira FB, Bueno NS, Nunes F, Pavão N (2023) Inoculation reduces misinformation: experimental evidence from multidimensional interventions in brazil. J Exp Polit Sci 1–12. https://doi.org/10.1017/xps.2023.11

Persad G, Emanuel EJ, Sangenito S, Glickman A, Phillips S, Largent EA (2021) Public perspectives on COVID-19 vaccine prioritization. JAMA Netw Open 4:e217943. https://doi.org/10.1001/jamanetworkopen.2021.7943

Pillai RM, Fazio LK (2023) Explaining why headlines are true or false reduces intentions to share false information. Collabra: Psychol 9. https://doi.org/10.1525/collabra.87617

Pinna M, Picard L, Goessmann C (2022) Cable news and COVID-19 vaccine uptake. Sci Rep 12:16804. https://doi.org/10.1038/s41598-022-20350-0

Article   ADS   CAS   PubMed   PubMed Central   Google Scholar  

Polantz K (2021) Lawyers sanctioned for ‘conspiracy theory’ election fraud lawsuit. CNN. https://edition.cnn.com/2021/08/04/politics/lawyers-colorado-2020-election/index.html

Porter E, Wood TJ (2021) The global effectiveness of fact-checking: evidence from simultaneous experiments in Argentina, Nigeria, South Africa, and the United Kingdom. Proc Natl Acad Sci USA 118:e2104235118. https://doi.org/10.1073/pnas.2104235118

Pothos EM, Lewandowsky S, Basieva I, Barque-Duran A, Tapper K, Khrennikov A (2021) Information overload for (bounded) rational agents. Proc R Soc B Biol Sci 288:20202957. https://doi.org/10.1098/rspb.2020.2957

Prike T, Butler LH, Ecker UKH (2024) Source-credibility information and social norms improve truth discernment and reduce engagement with misinformation online. Sci Rep 14:6900. https://doi.org/10.1038/s41598-024-57560-7

Proctor RN (2011) Golden holocaust: Origins of the cigarette catastrophe and the case for abolition. University of California Press

Proctor RN (2012) The history of the discovery of the cigarette–lung cancer link: evidentiary traditions, corporate denial, global toll. Tob Control 21(2):87–91. https://doi.org/10.1136/tobaccocontrol-2011-050338

Pröllochs N, Bär D, Feuerriegel S (2021) Emotions explain differences in the diffusion of true vs. false social media rumors. Sci Rep 11:22721. https://doi.org/10.1038/s41598-021-01813-2

Reid T (2022) Voting machine problems in Arizona seized on by Trump, election deniers. Reuters. https://www.reuters.com/world/us/voting-machine-problems-battleground-arizona-seized-by-trump-election-deniers-2022-11-08/

Rogers T, Zeckhauser R, Gino F, Norton MI, Schweitzer ME (2017) Artful paltering: the risks and rewards of using truthful statements to mislead others. J Personal Soc Psychol 112:456–473

Roozenbeek J, van der Linden S, Goldberg B, Rathje S, Lewandowsky S (2022) Psychological inoculation improves resilience against misinformation on social media. Sci Adv 8:eabo6254. https://doi.org/10.1126/sciadv.abo6254

Roozenbeek J, Suiter J, Culloty E (2023) Countering misinformation: evidence, knowledge gaps, and implications of current interventions. Eur Psychol. https://doi.org/10.31234/osf.io/b52um

Rumpler E, Feldman JM, Bassett MT, Lipsitch M (2023) Fairness and efficiency considerations in COVID-19 vaccine allocation strategies: A case study comparing front-line workers and 65–74 year olds in the United States. PLOS Glob Public Health 3:e0001378. https://doi.org/10.1371/journal.pgph.0001378

Rutenberg J, Myers SL (2024) How Trump’s allies are winning the war over disinformation. New York Times. https://www.nytimes.com/2024/03/17/us/politics/trump-disinformation-2024-social-media.html

Salek TA (2023) Deflecting deliberation through rhetorical nihilism: “Stop the Steal” as an unethical and intransigent rival public. Commun Democracy 57:94–118. https://doi.org/10.1080/27671127.2023.2202744

Scarcella M (2023) US Supreme Court rebuffs lawyers punished after ‘woeful’ suit backing Trump. Reuters. https://www.reuters.com/legal/us-supreme-court-rebuffs-lawyers-punished-after-woeful-suit-backing-trump-2023-10-02/

Shahid F, Vashistha A (2023) Decolonizing Content Moderation: Does Uniform Global Community Standard Resemble Utopian Equality or Western Power Hegemony? Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, 1–18. https://doi.org/10.1145/3544548.3581538

Simonov A, Sacher S, Dubé J-P, Biswas S (2020) The persuasive effect of Fox News: Non-compliance with social distancing during the Covid-19 pandemic (tech. rep.). National Bureau of Economic Research. https://doi.org/10.3386/w27237

Simonov A, Sacher S, Dubé J-P, Biswas S (2022) Frontiers: the persuasive effect of fox news: noncompliance with social distancing during the COVID-19 pandemic. Mark Sci 41:230–242. https://doi.org/10.1287/mksc.2021.1328

Smith P, Bansal-Travers M, O’Connor R, Brown A, Banthin C, Guardino-Colket S, Cummings K (2011) Correcting over 50 years of tobacco industry misinformation. Am J Prev Med 40:690–698

Soroka S, Fournier P, Nir L (2019) Cross-national evidence of a negativity bias in psychophysiological reactions to news. Proc Natl Acad Sci USA 116:18888–18892. https://doi.org/10.1073/pnas.1908369116

Stapleton A (2016) No, you can’t vote by text message. CNN. https://edition.cnn.com/2016/11/07/politics/vote-by-text-message-fake-news/index.html

Starbird K, DiResta R, DeButts M (2023) Influence and Improvisation: participatory disinformation during the 2020 US election. Soc Media Soc 9:20563051231177943. https://doi.org/10.1177/20563051231177943

Supran G, Rahmstorf S, Oreskes N (2023) Assessing ExxonMobil’s global warming projections. Science 379(6628):eabk0063. https://doi.org/10.1126/science.abk0063

Supran G, Oreskes N (2017) Assessing ExxonMobil’s climate change communications (1977–2014). Environ Res Lett 12:084019. https://doi.org/10.1088/1748-9326/aa815f

Supran G, Oreskes N (2021) Rhetoric and frame analysis of ExxonMobil’s climate change communications. One Earth. https://doi.org/10.1016/j.oneear.2021.04.014

Swire-Thompson B, Ecker UKH, Lewandowsky S, Berinsky AJ (2020) They might be a liar but they’re my liar: Source evaluation and the prevalence of misinformation. Polit Psychol 41:21–34. https://doi.org/10.1111/pops.12586

Takhshid Z (2021) Regulating social media in the global south. Vanderbilt J Entertain Technol Law 24:1–56

Tenove C (2020) Protecting democracy from disinformation: Normative threats and policy responses. Int J Press/Polit 25:517–537. https://doi.org/10.1177/1940161220918740

Terkel A, Timm JC, Gregorian D (2023) Here’s what fox news was trying to hide in its dominion lawsuit redactions. NBC News. https://www.nbcnews.com/politics/elections/dominion-releases-previously-redacted-slides-fox-news-lawsuit-rcna77257

U.S. House of Representatives Judiciary Committee. (2023). News release: Jim Jordan on why the select subcommittee on the weaponization of the federal government is necessary | House Judiciary Committee Republicans. http://judiciary.house.gov/media/press-releases/jim-jordan-on-why-the-select-subcommittee-on-the-weaponization-of-the-federal

Uscinski JE, Parent JM (2014) American conspiracy theories. Oxford University Press

Uscinski JE (2015) The epistemology of fact checking (is still naìve): Rejoinder to Amazeen. Crit Rev 27:243–252. https://doi.org/10.1080/08913811.2015.1055892

Van Der Zee S, Poppe R, Havrileck A, Baillon A (2021) A personal model of trumpery: linguistic deception detection in a real-world high-stakes setting. Psychol Sci. https://doi.org/10.1177/09567976211015941

van Doorn M (2023) Advancing the debate on the consequences of misinformation: clarifying why it’s not (just) about false beliefs. Inquiry 0:1–27. https://doi.org/10.1080/0020174X.2023.2289137

Vese D (2022) Governing fake news: the regulation of social media and the right to freedom of expression in the era of emergency. Eur J Risk Regul 13:477–513. https://doi.org/10.1017/err.2021.48

Vosoughi S, Roy D, Aral S (2018) The spread of true and false news online. Science 359:1146–1151. https://doi.org/10.1126/science.aap9559

Walasek L, Brown GDA (2023) Incomparability and incommensurability in choice: No common currency of value? Perspect Psychol Sci. https://doi.org/10.1177/17456916231192828

Wallace J, Goldsmith-Pinkham P, Schwartz JL (2023) Excess death rates for republican and democratic registered voters in Florida and Ohio during the COVID-19 pandemic. JAMA Internal Med. https://doi.org/10.1001/jamainternmed.2023.1154

Wanless A, Berk M (2019) The audience is the amplifier: Participatory propaganda. In P Baines, N O’Shaughnessy, & N Snow (Eds.), The sage handbook of propaganda (pp. 85–104). Sage

Wardle C, Derakhshan H (2017) Information disorder: Toward an interdisciplinary framework for research and policymaking (tech. rep.). Council of Europe. https://rm.coe.int/information-disorder-report-version-august-2018/16808c9c77

Weinschenk AC, Panagopoulos C, van der Linden S (2021) Democratic norms, social projection, and false consensus in the 2020 U.S. presidential election. J Polit Mark 20:255–268. https://doi.org/10.1080/15377857.2021.1939568

West D (2023) We shouldn’t turn disinformation into a constitutional right. Brookings Institution. https://www.brookings.edu/articles/we-shouldnt-turn-disinformation-into-a-constitutional-right/

Williams D (2021) Motivated ignorance, rationality, and democratic politics. Synthese 198:7807–7827. https://doi.org/10.1007/s11229-020-02549-8

Williams D (2022) The marketplace of rationalizations. Econ Philosophy, 1–25. https://doi.org/10.1017/S0266267121000389

Wu T (2017) The attention merchants. Atlantic Books

Yee AK (2023a) Information deprivation and democratic engagement. Philos Sci 90:1110–1119. https://doi.org/10.1017/psa.2023.9

Yee AK (2023b) Machine learning, misinformation, and citizen science. Eur J Philosophy Sci 13. https://doi.org/10.1007/s13194-023-00558-1

Zakrzewski C, Lima C, Harwell D (2023) What the Jan. 6 probe found out about social media, but didn’t report. Washington Post. https://www.washingtonpost.com/technology/2023/01/17/jan6-committee-report-social-media/

Download references

Acknowledgements

SL acknowledges financial support from the European Research Council (ERC Advanced Grant 101020961 PRODEMINFO), the Humboldt Foundation through a research award, the Volkswagen Foundation (grant “Reclaiming individual autonomy and democratic discourse online: How to rebalance human and algorithmic decision making”), and the European Commission (Horizon 2020 grants 964728 JITSUVAX and 101094752 SoMe4Dem). SL also receives funding from Jigsaw (a technology incubator created by Google) and from UK Research and Innovation through EU Horizon replacement funding grant number 10049415. UKHE acknowledges support from the Australian Research Council (grant FT190100708). For the purpose of open access, the author(s) has applied a Creative Commons Attribution (CC BY) licence to any Author Accepted Manuscript version arising from this submission.

Author information

Authors and affiliations.

University of Bristol, Bristol, UK

Stephan Lewandowsky

University of Potsdam, Potsdam, Germany

University of Western Australia, Crawley, WA, Australia

Ullrich K. H. Ecker

University of Melbourne, Melbourne, VIC, Australia

University of Cambridge, Cambridge, UK

Sander van der Linden

Kings College London, London, UK

Jon Roozenbeek

Harvard University, Cambridge, UK

Naomi Oreskes

Boston University, Boston, MA, USA

Lee C. McIntyre

You can also search for this author in PubMed   Google Scholar

Contributions

The first author created the first draft and all other authors contributed additional material and comments and suggestions and participated jointly in the editing and revision process.

Corresponding author

Correspondence to Stephan Lewandowsky .

Ethics declarations

Competing interests.

SL, JR, and SvdL have received funding from Google Jigsaw for empirical work on inoculation against misinformation and continue to collaborate with Jigsaw. NO has received funding from the Rockefeller Family Fund to support research on fossil fuel industry disinformation. She has also served as a consultant to the law firm Sher-Edling, who are representing several counties in California suing the fossil fuel industry, and as an expert witness in the defamation case of climate scientist Michael Mann. The remaining authors declare no competing interests.

Ethical approval

This article does not contain any studies with human participants performed by any of the authors.

Informed consent

This article does not contain any studies with human participants performed by any of the authors that would require consent.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Lewandowsky, S., Ecker, U.K.H., Cook, J. et al. Liars know they are lying: differentiating disinformation from disagreement. Humanit Soc Sci Commun 11 , 986 (2024). https://doi.org/10.1057/s41599-024-03503-6

Download citation

Received : 25 January 2024

Accepted : 22 July 2024

Published : 31 July 2024

DOI : https://doi.org/10.1057/s41599-024-03503-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

A Prediction Method for Surface Subsidence at Deep Mining Areas with Thin Bedrock and Thick Soil Layer Considering Consolidation Behavior

• Original Paper
• Published: 03 August 2024

Cite this article

• Jiachen Wang 1 , 2 , 3 ,
• Shanxi Wu 1 , 2 , 3 ,
• Zhaohui Wang 1 , 2 , 3 ,
• Shenyi Zhang 1 , 2 , 3 ,
• Boyuan Cheng 1 , 2 , 3 &
• Huashun Xie 1 , 2 , 3  

Among the various hazards induced by underground coal mining, surface subsidence tends to cause structural damage to the ground. Therefore, accurate prediction and evaluation of surface subsidence are significant for ensuring mining security and sustainable development. Traditional methods like the probability integral method provide effective predictions. However, these methods do not take into account the consolidation behavior of thick soil layers. In this study, based on the principle of superposition, an improved probability integral method that includes surface subsidence caused by rock layer movement and the consolidation behavior of thick soil layers is developed. The proposed method was applied in the Zhaogu No. 2 coal mine, located in the Jiaozuo mining area. Utilizing unmanned surface vehicle measurement technology, it was found that the maximum subsidence values of the two survey lines were 5.441 m and 4.842 m, with maximum subsidence rate of 62.9 mm/day at observation points. Experimental tests have shown that surface subsidence in deep mining areas with thin bedrock and thick soil layers exhibited a large subsidence coefficient and a wide range of subsidence, closely related to the consolidation behavior of thick soil layers. After verification, compared to the probability integral method, the improved probability integral method incorporating soil consolidation showed a 14.7% reduction in average error and a 22% reduction in maximum error. Therefore, the improved probability integral method proposed can be a very promising tool for forecasting and evaluating potential geohazards in coal mining areas.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save.

• Get 10 units per month
• Download Article/Chapter or eBook
• 1 Unit = 1 Article or 1 Chapter
• Cancel anytime

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Data availability

The original contributions presented in the study are included in the article/Supplementary Material, and further inquiries can be directed to the corresponding author.

Adelsohn, E., Iannacchione, A., & Winn, R. (2020). Investigations on longwall mining subsidence impacts on Pennsylvania highway alignments. International Journal of Mining Science and Technology, 30 (1), 85–92.

Article   Google Scholar  

Aksoy, C. O., Kucuk, K., & Uyar, G. G. (2016). Long-term time-dependent consolidation analysis by numerical modelling to determine subsidence effect area induced by longwall top coal caving method. International Journal of Oil Gas and Coal Technology, 12 (1), 18–37.

Bell, F. G., & Genske, D. D. (2001). The influence of subsidence attributable to coal mining on the environment, development and restoration: Some examples from western Europe and south Africa. Environmental & Engineering Geoscience, 7 (1), 81–99.

Bru, G., Herrera, G., Tomás, R., Duro, J., De la Vega, R., & Mulas, J. (2013). Control of deformation of buildings affected by subsidence using persistent scatterer interferometry. Structure and Infrastructure Engineering, 9 (2), 188–200.

Google Scholar  

Budhu, M. (2010). Soil Mechanics and Foundations (3rd ed.). Wiley.

Budhu, M., & Adiyaman, I. B. (2010). Mechanics of land subsidence due to groundwater pumping. International Journal for Numerical and Analytical Methods in Geomechanics, 34 (14), 1459–1478.

Chai, H. B., Xu, M. T., Guan, P. J., Ding, Y. H., Xu, H., & Zhao, Y. Q. (2023). Research on mining subsidence prediction parameter inversion based on improved modular vector method. Applied Sciences-Basel, 13 (24), 13272.

Article   CAS   Google Scholar  

Chen, H., Xue, Y. G., & Qiu, D. H. (2023). Numerical simulation of the land subsidence induced by groundwater mining. Cluster Computing, 26 (6), 3647–3656.

Deck, O., Verdel, T., & Salmon, R. (2009). Vulnerability assessment of mining subsidence hazards. Risk Analysis, 29 (10), 1381–1394.

Ding, L. J., & Liu, Y. H. (2017). Three-dimensional physical simulation on overlying strata’s motion rule in shallow seam. Fresenius Environmental Bulletin, 26 (8), 5314–5322.

CAS   Google Scholar  

Galaviz-González, J. R., Horta-Rangel, J., Limón-Covarrubias, P., Avalos-Cueva, D., Cabello-Suárez, L. Y., López-Lara, T., & Hernández-Zaragoza, J. B. (2022). Elastoplastic coupled model of saturated soil consolidation under effective stress. Water, 14 (19), 2958.

Ghabraie, B., Ren, G., Barbato, J., & Smith, J. V. (2017). A predictive methodology for multi-seam mining induced subsidence. International journal of rock mechanics and mining sciences. International Journal of Rock Mechanics and Mining Sciences, 93 , 280–294.

Gruszczynski, W., Niedojadlo, Z., & Mrochen, D. (2018). Influence of model parameter uncertainties on forecasted subsidence. Acta Geodynamica Et Geomaterialia, 15 (3), 211–228.

Gruszczynski, W., Niedojadlo, Z., & Mrochen, D. (2019). A comparison of methods for propagating surface deformation uncertainties in model parameters. Acta Geodynamica Et Geomaterialia, 16 (4), 349–357.

Guo, Z. Z., Xie, H. P., & Wang, J. Z. (2004). Applying probability distribution density function to predict the surface subsidence caused by subcritical extraction. Journal of China Coal Society, 29 (2), 155–158.

Hall, K. M., & Fox, P. J. (2018). Large strain consolidation model for land subsidence. International Journal of Geomechanics., 18 (11), 06018028.

Hossain, M. I. S., Alam, M. S., Biswas, P. K., Rana, M. S., Sultana, M. S., Zaman, M. N., Samad, M. A., Rahman, M. J., & Woobaidullah, A. S. M. (2023). Integrated satellite imagery and electrical resistivity analysis of underground mine-induced subsidence and associated risk assessment of Barapukuria coal mine, Bangladesh. Environmental Earth Sciences, 82 (22), 537.

Hou, D. F., Li, D. H., Xu, G. S., & Zhang, Y. B. (2018). Superposition model for analyzing the dynamic ground subsidence in mining area of thick loose layer. International Journal of Mining Science and Technology, 28 (4), 663–668.

Intui, S., Inazumi, S., & Soralump, S. (2022). Evaluation of land subsidence during groundwater recovery. Applied Sciences-Basel, 12 (15), 7904.

Jiang, S. Y., Fan, G. W., Li, Q. Z., Zhang, S. Z., & Chen, L. (2021). Effect of mining parameters on surface deformation and coal pillar stability under customized shortwall mining of deep extra-thick coal seams. Energy Reports, 7 , 2138–2154. https://doi.org/10.1016/j.egyr.2021.04.008

Lanes, R. M., Greco, M., & Almeida, V. D. (2023). Viscoelastic soil-structure interaction procedure for building on footing foundations considering consolidation settlements. Buildings, 13 (3), 813. https://doi.org/10.3390/buildings13030813

Li, G. X., Zhang, B. Y., & Yu, Y. Z. (2022). Soil Mechanics (3rd ed.). Tsinghua University Press.

Li, J. X., Yu, X. X., Chen, D. S., & Fang, X. J. (2021). Research on the establishment of a mining subsidence prediction model under thick loose layer and its parameter inversion method. Earth Sciences Research Journal, 25 (2), 215–223.

Liu, B. C., & Dai, H. Y. (2016). Research development and origin of probability integral method. Coal Mining Technology, 21 (2), 1–3.

Malinowska, A., & Hejmanowski, R. (2010). Building damage risk assessment on mining terrains in Poland with GIS application. International Journal of Rock Mechanics and Mining Sciences, 47 (2), 238–245.

Pal, A., Roser, J., & Vulic, M. (2020). Surface subsidence prognosis above an underground longwall excavation and based on 3d point cloud analysis. Minerals, 10 (1), 82.

Pan, R., Li, Y., Wang, H., Chen, J., Xu, Y., Yang, H., & Cao, S. (2021). A new model for the identification of subcritical surface subsidence in deep pillarless mining. Engineering Failure Analysis, 129 , 105631.

Prakash, A., Kumar, A., Verma, A., Mandal, S. K., & Singh, P. K. (2021). Trait of subsidence under high rate of coal extraction by longwall mining: Some inferences. Sadhana-Academy Proceedings in Engineering Sciences, 46 (4), 216.

Shi, W. P., Li, K. X., Yu, S. W., Zhang, C. Z., & Li, J. K. (2021a). Analysis on subsidence law of bedrock and ultrathick loose layer caused by coal mining. Geofluids, 2021 , 8849955.

Shi, W. P., Qu, X. C., Jiang, C. T., & Li, K. X. (2021b). Study on numerical simulation test of mining surface subsidence law under ultrathick loose layer. Geofluids, 2021 , 6655827.

Sillerico, E., Marchamalo, M., Rejas, G. J., & Martínez, R. (2010). DInSAR technique: basis and applications to terrain subsidence monitoring in construction works. Informes De La Construccion, 62 (519), 47–53.

Strozik, G., Jendrus, R., Manowska, A., & Popczyk, M. (2016). Mine subsidence as a post-mining effect in the upper silesia coal basin. Polish Journal of Environmental Studies, 25 (2), 777–785.

Wang, F., Jiang, B. Y., Chen, S. J., & Ren, M. Z. (2019a). Surface collapse control under thick unconsolidated layers by backfilling strip mining in coal mines. International Journal of Rock Mechanics and Mining Sciences, 113 , 268–277.

Wang, F., Xu, J. L., Chen, S. J., & Ren, M. Z. (2019b). Method to predict the height of the water conducting fractured zone based on bearing structures in the overlying strata. Mine Water and the Environment, 38 (4), 767–779.

Wang, F., Zhu, W. H., Jie, Z. Q., Lu, L., & Chen, Z. T. (2023a). Load bearing capacity of arch structure in unconsolidated layers. Scientific Reports, 13 (1), 4232.

Wang, J. C., Wang, Z. H., Tang, Y. S., Li, M., Chang, K. L., Gong, H., & Xu, G. L. (2021). Experimental study on mining-induced dynamic impact effect of main roofs in deeply buried thick coal seams with weakly consolidated thin bed rock. Chinese Journal of Rock Mechanics and Rock Engineering, 40 (12), 2377–2391.

Wang, Z. H., Tang, Y. S., Li, M., Wu, S. X., Sun, W. C., & Shui, Y. T. (2023b). Development and application of overburden structure composed of caving arch and towering roof beam in deep longwall panel with thin bedrock. Journal of China Coal Society, 48 (2), 563–575.

Wei, T., Guo, G. L., Li, H. Z., Wang, L., Jiang, Q., & Jiang, C. M. (2023). A novel probability integral method segmental modified model for subsidence prediction applicable to thick loose layer mining areas. Environmental Science and Pollution Research, 30 (18), 52049–52061.

Wu, S. X., Wang, Z. H., Li, J. L., Hu, H. Y., An, B. C., He, J. Q., & Zhang, S. Y. (2024). Research on the mechanical characteristics of thick alluvium on the surface subsidence features of thin bedrock deposits at depth. Mining Metallurgy & Exploration, 41 , 1281–1298.

Yan, W. T., Guo, J. T., Yan, S. G., Yan, Y. G., & Tang, W. (2023). A novel surface subsidence prediction model based on stochastic medium theory for inclined coal seam mining. Advances in Civil Engineering . https://doi.org/10.1155/2023/4640471

Yan, Y. G., Zhang, Y. J., Zhu, Y. H., Cai, J. C., & Wang, J. Y. (2022). Quantitative study on the law of surface subsidence zoning in steeply inclined extra-thick coal seam mining. Sustainability, 14 (11), 6758.

Yang, S. L., Wu, S. X., Wang, Z. H., Tang, Y. S., Li, J. L., & Sun, W. C. (2023). Surface subsidence and its prediction method of mining deep-buried seam with thick alluvial layer and thin bedrock. Journal of China Coal Society, 48 (2), 523–537.

Zhang, B., Ye, J. C., Zhang, Z. J., Xu, L., & Xu, N. X. (2019a). A Comprehensive Method for Subsidence Prediction on Two-Seam Longwall Mining. Energies, 12 (16), 3139.

Zhang, C., Bai, Q. S., & Han, P. H. (2023). A review of water rock interaction in underground coal mining: Problems and analysis. Bulletin of Engineering Geology and the Environment, 82 (5), 157.

Zhang, C., Tu, S. H., & Zhao, Y. X. (2019b). Compaction characteristics of the caving zone in a longwall goaf: A review. Environmental Earth Sciences, 78 (1), 27.

Zhang, J. H., Chen, Z. H., Tian, Z. M., Jiang, S. Y., & Li, C. H. (2015). One-dimensional compression tests and deformation rules of unsaturated soils. Chinese Journal of Geotechnical Engineering, 37 (01), 61–66.

Zhou, D. W., Wu, K., & Li, L. (2018). Combined prediction model for mining subsidence in coal mining areas covered with thick alluvial soil layer. Bulletin of Engineering Geology and the Environment, 77 (01), 283–304.

Zhu, D. F., Yu, B. B., Wang, D. Y., & Zhang, Y. J. (2024). Fusion of finite element and machine learning methods to predict rock shear strength parameters. Journal of Geophysics and Engineering . https://doi.org/10.1093/jge/gxae064

Zhu, X. X., Zhang, W. Q., Wang, Z. Y., Wang, C. H., Li, W., & Wang, C. H. (2020). Simulation analysis of influencing factors of subsidence based on mining under huge loose strata: A case study of Heze mining area. China. Geofluids, 2020 , 6357683.

Download references

Acknowledgements

This research was supported by the National Natural Science Foundation of China (Grant Nos. 51934008; 52374106), the Fundamental Research Funds for the Central Universities (Grant Nos. 2024ZKPYNY04; 2023ZKPYNY01; No. 2023YQTD02). The second author would give special thanks to the China Scholarship Council, China (Grant No. 202306430056), and also to the Chair of Mining Engineering and Mineral Economics, Montanuniversität Leoben for hosting during his Austria visit. The authors acknowledge the above funds for supporting this research and editors and reviewers for their comments and suggestions.

Author information

Authors and affiliations.

School of Energy and Mining Engineering, China University of Mining and Technology-Beijing, Beijing, 100083, China

Jiachen Wang, Shanxi Wu, Zhaohui Wang, Shenyi Zhang, Boyuan Cheng & Huashun Xie

Coal Industry Engineering Research Center of Top Coal, Beijing, 100083, China

Engineering Research Center of Green and Intelligent Mining for Thick Coal Seam, Ministry of Education, Beijing, 100083, China

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Zhaohui Wang .

Ethics declarations

Competing interests.

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Wang, J., Wu, S., Wang, Z. et al. A Prediction Method for Surface Subsidence at Deep Mining Areas with Thin Bedrock and Thick Soil Layer Considering Consolidation Behavior. Nat Resour Res (2024). https://doi.org/10.1007/s11053-024-10395-5

Download citation

Received : 13 May 2024

Accepted : 21 July 2024

Published : 03 August 2024

DOI : https://doi.org/10.1007/s11053-024-10395-5

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Mining subsidence
• Thick soil layer
• Probability integral method
• Consolidation of behavior soil layer
• Subsidence prediction
• Find a journal
• Publish with us
• Track your research

Comparison the Influence of Two Types of Black Cumin Oil (Nigella sativa L.) on Some Physicochemical, Microbiological and Sensory Evaluation of Processed Cheese

• Alhelli, Amaal Mohammed
• Kanaem Al-Ibady, Qater Al-Nada Ali
• Obaidi, Hadel

During storage of processed cheese, structure and flavor slowly change. The destination of the current research was to examine the influence of two types of Nigella sativa L. oil (NSO) extracted by supercritical fluid extraction (SFNSO) and cold press (CPNSO) via determine the physicochemical, microbiological and sensory attributes of processed cheese. Seven batches of processed cheese were included in the current experiment: control cheese without any addition and cheese samples incorporated separately with 0.1, 0.2 and 0.3% v/w of SFNSO and CPNSO. Experimental cheese samples were analyzed in triplicate for total coliforms, Escherichia coli (E.coli), total bacteria count (TBc), and yeast & moulds. Moreover, moisture content (%), fat (%), pH, soluble nitrogen (SN%), and total nitrogen (TN%). All treated cheeses without addition and with CPNSO appeared more significant effect on physicochemical properties than the other samples. In addition, sensory evaluation at 0, 1, 2, 3 and 4 months of storage treated with SFNSO shows more impact on microbiological content than CPNSO and control cheeses. Employment SFNSO on processed cheese enhanced the stability of sensory evaluation throughout storage time.

• Processed cheese;
• Nigella sativa L;
• Supercritical fluid extraction;
• Cold press Nigella sativa oil;
• Sensory evaluation