natural experiment variable

Physical Review A

Covering atomic, molecular, and optical physics and quantum science.

• Collections
• Editorial Team

Effects of topological boundary conditions on Bell nonlocality

Patrick emonts, mengyao hu, albert aloy, and jordi tura, phys. rev. a 110 , 032201 – published 3 september 2024.

• No Citing Articles
• INTRODUCTION
• PRELIMINARIES—BELL INEQUALITIES
• CONCLUSIONS AND OUTLOOK
• ACKNOWLEDGMENTS

Bell nonlocality is the resource that enables device-independent quantum information processing tasks. It is revealed through the violation of so-called Bell inequalities, indicating that the observed correlations cannot be reproduced by any local hidden-variable model. While well explored in few-body settings, the question of which Bell inequalities are best suited for a given task remains quite open in the many-body scenario. One natural approach is to assign Bell inequalities to physical Hamiltonians, mapping their interaction graph to two-body, nearest-neighbor terms. Here, we investigate the effect of boundary conditions in a two-dimensional square lattice, which can induce different topologies in lattice systems. We find a relation between the induced topology and the Bell inequality's effectiveness in revealing nonlocal correlations. By using a combination of tropical algebra and tensor networks, we quantify their detection capacity for nonlocality. Our work can act as a guide to certify Bell nonlocality in many-qubit devices by choosing a suitable Hamiltonian and measuring its ground-state energy, a task that many quantum experiments are purposely built for.

• Received 14 June 2024
• Accepted 2 August 2024

DOI: https://doi.org/10.1103/PhysRevA.110.032201

©2024 American Physical Society

Physics Subject Headings (PhySH)

• Research Areas
• Physical Systems

Authors & Affiliations

• 1 Instituut-Lorentz, Universiteit Leiden , P.O. Box 9506, 2300 RA Leiden, The Netherlands
• 2 Institute for Quantum Optics and Quantum Information, Austrian Academy of Sciences, Boltzmanngasse 3, A-1090 Vienna, Austria
• 3 Vienna Center for Quantum Science and Technology , Faculty of Physics, University of Vienna, Boltzmanngasse 5, A-1090 Vienna, Austria
• * These authors contributed equally to this work.

Article Text (Subscription Required)

References (subscription required).

Vol. 110, Iss. 3 — September 2024

Access Options

• Buy Article »
• Log in with individual APS Journal Account »
• Log in with a username/password provided by your institution »
• Get access through a U.S. public or high school library »

Authorization Required

Other options.

• Buy Article »
• Find an Institution with the Article »

Download & Share

Dimer configurations in one and two dimensions with periodic boundary conditions. In the one-dimensional case, there are only two distinct dimer coverings (links on even or odd links). In two spatial dimensions, multiple dimer configurations are possible. Dimer configurations related by spatial symmetries (see gray box) yield the quantum and classical bounds.

Possible violation of a Bell inequality. Depending on ε , the classical bound β C (gray line) and the quantum value β Q (black line) of the inequality vary. The region where a violation is possible (in blue) depends on the size of the system, the dimer configuration, and the boundary conditions. The intersections between the classical bound and the quantum value are the critical values of the coupling ε l * and ε h * .

Graph orbits of a 3 × 3 torus. The dimers form three distinct classes, visible as three connected subgraphs. Each vertex corresponds to one dimer covering. The labels on the edges represent the symmetry operations that connect the different coverings. The shaded region in blue is a zoomed-in version of class 0.

Example of successive contractions in one dimension. In each step, one variable is eliminated by minimizing over it.

The two-dimensional system can be transformed into a one-dimensional system by blocking the columns to enlarged sites. The dimension of the variables s i is exponentially bigger than the original variables s i .

Comparison of exact results (vertical lines) and MPS results (dots) for ε * on a torus of size 3 × 3 . The left (right) panel shows value of ε l * ( ε h * ) smaller (greater) than 1. Each dot represents one dimer configuration on the lattice. The vertical axis only enumerates the different dimer configurations.

Ranges of critical values for different boundary conditions and system sizes. From left to right, the three figures indicate the results for 3 × 3 , 4 × 4 , and 5 × 5 systems. The blue (orange) regions shows the ranges of ε * for a system on a torus (Klein bottle). The bars close to the axis connect the value of the minimal and the maximal critical epsilon for a given system. The bars span across multiple classes. Insets: A dimer covering of the class with maximal (minimal) ε * .

Computation of ε * for a 5 × 5 lattice with periodic boundary conditions. The points are the median of 10 representative dimer configurations from each class. The asymmetric error bars show the minimal and maximal deviations among all considered realizations.

Sketch of the topology of a torus (top) and a Klein bottle (bottom). The arrows, from left to right, indicate the successive merging of boundaries.

Statistics of the dimer configurations. Top: Statistics for a system on a torus of size 4 × 4 . Bottom: Statistics for a system on a Klein bottle of the same size.

Sign up to receive regular email alerts from Physical Review A

• Forgot your username/password?
• Create an account

Article Lookup

Paste a citation or doi, enter a citation.

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Recommended Articles
• Google Scholar
• on Google Scholar
• Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

A study on the impact of china’s prefabricated building policy on the carbon reduction benefits of china’s construction industry based on a difference-in-differences method.

1. Introduction

2. literature review, 3. methods and data, 3.1. difference-in-differences model, 3.2. mechanism analysis, 3.3. variables and date, 3.3.1. dependent variable, 3.3.2. core explanatory variables, 3.3.3. mechanism variables, 3.3.4. control variable, 4. experimental analysis, 4.1. benchmark regression results, 4.2. parallel trend test, 5. robustness testing, 5.1. placebo testing, 5.2. heterogeneity analysis, 5.3. psm-did, 6. mechanism variables analysis, 7. discussion on variable factors, 7.1. discussion on core variables, 7.2. discussion on control variable, 7.3. discussion on mechanism variables, 8. conclusions and policy recommendations, 8.1. research conclusions, 8.2. research limitations, 8.3. policy recommendations, author contributions, institutional review board statement, informed consent statement, data availability statement, conflicts of interest.

• Gu, G.; Zheng, H.; Tong, L.; Dai, Y. Does carbon financial market as an environmental regulation policy tool promote regional energy conservation and emission reduction? Empirical evidence from China. Energy Policy 2022 , 163 , 112826. [ Google Scholar ] [ CrossRef ]
• Weng, Q.; Xu, H. A review of China’s carbon trading market. Renew. Sustain. Energy Rev. 2018 , 91 , 613–619. [ Google Scholar ] [ CrossRef ]
• Qi, S.; Cheng, S.; Cui, J. Environmental and economic effects of China’s carbon market pilots: Empirical evidence based on a DID model. J. Clean. Prod. 2021 , 279 , 123720. [ Google Scholar ] [ CrossRef ]
• Liang, X.; Lin, S.; Bi, X.; Lu, E.; Li, Z. Chinese construction industry energy efficiency analysis with undesirable carbon emissions and construction waste outputs. Environ. Sci. Pollut. Res. 2021 , 28 , 15838–15852. [ Google Scholar ] [ CrossRef ]
• Zhang, Y.; Yan, D.; Hu, S.; Guo, S. Modelling of energy consumption and carbon emission from the building construction sector in China, a process-based LCA approach. Energy Policy 2019 , 134 , 110949. [ Google Scholar ] [ CrossRef ]
• Mao, C.; Shen, Q.; Pan, W.; Ye, K. Major Barriers to Off-Site Construction: The Developers’ Perspective in China. J. Manag. Eng. 2013 , 31 , 04014043. [ Google Scholar ] [ CrossRef ]
• Huang, L.; Krigsvoll, G.; Johansen, F.; Liu, Y.; Zhang, X. Carbon emission of global construction sector. Renew. Sustain. Energy Rev. 2018 , 81 , 1906–1916. [ Google Scholar ] [ CrossRef ]
• Tang, T.-Z. A Comparative Study of Environmental Management Systems and Policies between China and Japan. Contemp. Econ. Jpn. 2007 , 156 , 1–6. [ Google Scholar ]
• Dou, Y.; Xue, X.; Wang, Y.; Luo, X.; Shang, S. New media data-driven measurement for the development level of prefabricated construction in China. J. Clean. Prod. 2019 , 241 , 118353. [ Google Scholar ] [ CrossRef ]
• Liu, G.; Gu, T.; Xu, P.; Hong, J.; Shrestha, A.; Martek, I. A production line-based carbon emission assessment model for prefabricated components in China. J. Clean. Prod. 2019 , 209 , 30–39. [ Google Scholar ] [ CrossRef ]
• Jiang, L.; Li, Z.F.; Li, L.; Gao, Y.L. Constraints on the Promotion of Prefabricated Construction in China. Sustainability 2018 , 10 , 2516. [ Google Scholar ] [ CrossRef ]
• Wang, Y.; Xue, X.; Yu, T.; Wang, Y. Mapping the dynamics of China’s prefabricated building policies from 1956 to 2019: A bibliometric analysis. Build. Res. Inf. 2020 , 49 , 216–233. [ Google Scholar ] [ CrossRef ]
• Guo, M.; Hu, Y. The Impact of Financial Development on Carbon Emission: Evidence from China. Sustainability 2020 , 12 , 6959. [ Google Scholar ] [ CrossRef ]
• Kais, S.; Sami, H. An econometric study of the impact of economic growth and energy use on carbon emissions: Panel data evidence from fifty eight countries. Renew. Sustain. Energy Rev. 2016 , 59 , 1101–1110. [ Google Scholar ] [ CrossRef ]
• Qu, F.Q.; Yan, W.; Chen, C.Y. Research on Construction of Coordination Performance Evaluation Index System of Prefabricated Construction Supply Chain. Constr. Econ. 2019 , 40 , 97–102. [ Google Scholar ]
• Barlow, J.; Childerhouse, P.; Gann, D.; Hong-Minh, S.; Naim, M.; Ozaki, R. Choice and delivery in housebuilding: Lessons from Japan for UK housebuilders. Build. Res. Inf. 2003 , 31 , 134–145. [ Google Scholar ] [ CrossRef ]
• Coutts, J. The Building Regulations and building control. In Loft Conversions , 2nd ed.; Wiley-Blackwell: Hoboken, NJ, USA, 2013. [ Google Scholar ]
• Garrone, P.; Grilli, L. Is There a Relationship between Public Expenditures in Energy R&D and Carbon Emissions per GDP? An Empirical Investigation. Energy Policy 2010 , 38 , 5600–5613. [ Google Scholar ]
• Fedorczak-Cisak, M.; Bomberg, M.; Yarbrough, D.W.; Lingo, L.E.; Romanska-Zapala, A. Position Paper Introducing a Sustainable, Universal Approach to Retrofitting Residential Buildings. Buildings 2022 , 12 , 846. [ Google Scholar ] [ CrossRef ]
• Zhang, X.; Skitmore, M. Industrialized Housing in China: A Coin with Two Sides. Int. J. Strateg. Prop. Manag. 2012 , 16 , 143–157. [ Google Scholar ] [ CrossRef ]
• Teng, Y.; Li, K.J.; Pan, W.; Ng, T. Reducing building life cycle carbon emissions through prefabrication: Evidence from and gaps in empirical studies. Build. Environ. 2018 , 132 , 125–136. [ Google Scholar ] [ CrossRef ]
• Li, X.-J.; Lai, J.-Y.; Ma, C.-Y.; Wang, C. Using BIM to research carbon footprint during the materialization phase of prefabricated concrete buildings: A China study. J. Clean. Prod. 2021 , 279 , 123454. [ Google Scholar ] [ CrossRef ]
• You, F.; Hu, D.; Zhang, H.T.; Guo, Z.; Zhao, Y.H.; Wang, B.N.; Yuan, Y. Carbon emissions in the life cycle of urban building system in China-A case study of residential buildings. Ecol. Complex. 2011 , 8 , 201–212. [ Google Scholar ] [ CrossRef ]
• Cao, X.; Li, X.; Zhu, Y.; Zhang, Z. A comparative study of environmental performance between prefabricated and traditional residential buildings in China. J. Clean. Prod. 2015 , 109 , 131–143. [ Google Scholar ] [ CrossRef ]
• Sebaibi, N.; Boutouil, M. Reducing energy consumption of prefabricated building elements and lowering the environmental impact of concrete. Eng. Struct. 2020 , 213 , 8. [ Google Scholar ] [ CrossRef ]
• Yin, X.; Dong, Q.; Zhou, S.; Yu, J.; Huang, L.; Sun, C. Energy-Saving Potential of Applying Prefabricated Straw Bale Construction (PSBC) in Domestic Buildings in Northern China. Sustainability 2020 , 12 , 3464. [ Google Scholar ] [ CrossRef ]
• Liu, Z.Y.; Ying, H.Q. Prefabrication Construction in Residential Building of Vanke Real Estate Company China. In Proceedings of the 2009 International Conference on Management and Service Science, Wuhan, China, 16–18 September 2009. [ Google Scholar ]
• Senbel, M.; Mcdaniels, T.; Dowlatabadi, H. The ecological footprint: A non-monetary metric of human consumption applied to North America. Glob. Environ. Chang. Hum. Policy Dimens. 2003 , 13 , 83–100. [ Google Scholar ] [ CrossRef ]
• Shigeto, S.; Yamagata, Y.; Ii, R.; Hidaka, M.; Horio, M. An easily traceable scenario for 80% CO 2 emission reduction in Japan through the final consumption-based CO 2 emission approach: A case study of Kyoto-city. Appl. Energy 2012 , 90 , 201–205. [ Google Scholar ] [ CrossRef ]
• Zhang, B.; Qiao, H.; Chen, Z.M.; Chen, B. Growth in embodied energy transfers via China’s domestic trade: Evidence from multi-regional input-output analysis. Appl. Energy 2016 , 184 , 1093–1105. [ Google Scholar ] [ CrossRef ]
• Steininger, K.; Lininger, C.; Droege, S.; Roser, D.; Tomlinson, L.; Meyer, L. Justice and cost effectiveness of consumption-based versus production-based approaches in the case of unilateral climate policies. Glob. Environ. Chang.-Hum. Policy Dimens. 2014 , 24 , 75–87. [ Google Scholar ] [ CrossRef ]
• Atkinson, G.; Hamilton, K.; Ruta, G.; Van der Mensbrugghe, D. Trade in ‘virtual carbon’: Empirical results and implications for policy. Glob. Environ. Chang.-Hum. Policy Dimens. 2011 , 21 , 563–574. [ Google Scholar ] [ CrossRef ]
• Mi, Z.; Zhang, Y.; Guan, D.; Shan, Y.; Liu, Z.; Cong, R.; Yuan, X.-C.; Wei, Y.-M. Consumption-based emission accounting for Chinese cities. Appl. Energy 2016 , 184 , 1073–1081. [ Google Scholar ] [ CrossRef ]
• Guan, D.; Lin, J.; Davis, S.J.; Pan, D.; He, K.; Wang, C.; Wuebbles, D.J.; Streets, D.G.; Zhang, Q. Reply to Lopez et al.: Consumption-based accounting helps mitigate global air pollution. Proc. Natl. Acad. Sci. USA 2014 , 111 , E2631. [ Google Scholar ] [ CrossRef ]
• Suh, S.; Lenzen, M.; Treloar, G.J.; Hondo, H.; Horvath, A.; Huppes, G.; Jolliet, O.; Klann, U.; Krewitt, W.; Moriguchi, Y.; et al. System Boundary Selection in Life-Cycle Inventories Using Hybrid Approaches. Environ. Sci. Technol. 2003 , 38 , 657–664. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Liu, G.; Chen, R.; Xu, P.; Fu, Y.; Mao, C.; Hong, J. Real-time carbon emission monitoring in prefabricated construction. Autom. Constr. 2020 , 110 , 102945. [ Google Scholar ] [ CrossRef ]
• Monahan, J.; Powell, J.C. An embodied carbon and energy analysis of modern methods of construction in housing: A case study using a lifecycle assessment framework. Energy Build. 2011 , 43 , 179–188. [ Google Scholar ] [ CrossRef ]
• Jia Wen, T.; Chin Siong, H.; Noor, Z.Z. Assessment of embodied energy and global warming potential of building construction using life cycle analysis approach: Case studies of residential buildings in Iskandar Malaysia. Energy Build. 2015 , 93 , 295–302. [ Google Scholar ] [ CrossRef ]
• Acquaye, A.A.; Duffy, A.P. Input-output analysis of Irish construction sector greenhouse gas emissions. Build. Environ. 2010 , 45 , 784–791. [ Google Scholar ] [ CrossRef ]
• Liang, H.W.; Dong, L.; Luo, X.; Ren, J.Z.; Zhang, N.; Gao, Z.Q.; Dou, Y. Balancing regional industrial development: Analysis on regional disparity of China’s industrial emissions and policy implications. J. Clean. Prod. 2016 , 126 , 223–235. [ Google Scholar ] [ CrossRef ]
• Liu, B.S.; Yang, X.D.; Huo, T.F.; Shen, G.Q.; Wang, X.Q. A linguistic group decision-making framework for bid evaluation in mega public projects considering carbon dioxide emissions reduction. J. Clean. Prod. 2017 , 148 , 811–825. [ Google Scholar ] [ CrossRef ]
• Jenne, C.A.; Cattell, R.K. Structural change and energy efficiency in industry. Energy Econ. 1983 , 5 , 114–123. [ Google Scholar ] [ CrossRef ]
• Hu, J.L.; Wang, S.C. Total-factor energy efficiency of regions in China. Energy Policy 2006 , 34 , 3206–3217. [ Google Scholar ] [ CrossRef ]
• Wang, L.J.; Song, X.J.; Song, X.J. Research on the measurement and spatial-temporal difference analysis of energy efficiency in China’s construction industry based on a game cross-efficiency model. J. Clean. Prod. 2021 , 278 , 13. [ Google Scholar ] [ CrossRef ]
• Xie, S.S.; Wang, J.J. Evaluating the Effect of China’s Carbon Emission Trading Policy on Energy Efficiency of the Construction Industry Based on a Difference-in-Differences Method. Adv. Civ. Eng. 2022 , 2022 , 12. [ Google Scholar ] [ CrossRef ]
• Li, J.; Li, J.-F.; Yu, Q.; Chen, Q.N.; Xie, S. Strain-based scanning probe microscopies for functional materials, biological structures, and electrochemical systems. J. Mater. 2015 , 1 , 3–21. [ Google Scholar ] [ CrossRef ]
• Agi, M.A.N.; Nishant, R. Understanding influential factors on implementing green supply chain management practices: An interpretive structural modelling analysis. J. Environ. Manag. 2017 , 188 , 351–363. [ Google Scholar ] [ CrossRef ]
• Diabat, A.; Govindan, K. An analysis of the drivers affecting the implementation of green supply chain management. Resour. Conserv. Recycl. 2011 , 55 , 659–667. [ Google Scholar ] [ CrossRef ]
• Raut, R.D.; Narkhede, B.; Gardas, B.B. To identify the critical success factors of sustainable supply chain management practices in the context of oil and gas industries: ISM approach. Renew. Sustain. Energy Rev. 2017 , 68 , 33–47. [ Google Scholar ] [ CrossRef ]
• Sun, S.; Chen, Y.; Wang, A.; Liu, X. An Evaluation Model of Carbon Emission Reduction Effect of Prefabricated Buildings Based on Cloud Model from the Perspective of Construction Supply Chain. Buildings 2022 , 12 , 1534. [ Google Scholar ] [ CrossRef ]
• Gao, J.X.; Ren, H.; Ma, X.R.; Cai, W.G.; Shi, Q.W. A total energy efficiency evaluation framework based on embodied energy for the construction industry and the spatio-temporal evolution analysis. Eng. Constr. Archit. Manag. 2019 , 26 , 1652–1671. [ Google Scholar ] [ CrossRef ]
• Gong, Y.Y.; Song, D.Y. Life Cycle Building Carbon Emissions Assessment and Driving Factors Decomposition Analysis Based on LMDI-A Case Study of Wuhan City in China. Sustainability 2015 , 7 , 16670–16686. [ Google Scholar ] [ CrossRef ]
• Zhang, X.; Wang, F. Life-cycle assessment and control measures for carbon emissions of typical buildings in China. Build. Environ. 2015 , 86 , 89–97. [ Google Scholar ] [ CrossRef ]
• Li, B.; Han, S.W.; Wang, Y.F.; Li, J.Y.; Wang, Y. Feasibility assessment of the carbon emissions peak in China’s construction industry: Factor decomposition and peak forecast. Sci. Total Environ. 2020 , 706 , 135716. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Wei-Ke, C.; Fang, L. Research on Building Energy Consumption based on Whole Life Cycle Theory. Build. Sci. 2008 , 24 , 23–27. [ Google Scholar ]
• Fan, Y.; Liu, L.C.; Wu, G.; Tsai, H.T.; Wei, Y.M. Changes in carbon intensity in China: Empirical findings from 1980–2003. Ecol. Econ. 2007 , 62 , 683–691. [ Google Scholar ] [ CrossRef ]
• Masood, R.; Lim, J.B.P.; González, V.A. Performance of the supply chains for New Zealand prefabricated house-building. Sustain. Cities Soc. 2021 , 64 , 102537. [ Google Scholar ] [ CrossRef ]
• Qiu, S.; Wang, Z.; Liu, S. The policy outcomes of low-carbon city construction on urban green development: Evidence from a quasi-natural experiment conducted in China. Sustain. Cities Soc. 2021 , 66 , 102699. [ Google Scholar ] [ CrossRef ]

Click here to enlarge figure

Share and Cite

Wang, X.; Xie, S.; Wei, Z.; Wang, J. A Study on the Impact of China’s Prefabricated Building Policy on the Carbon Reduction Benefits of China’s Construction Industry Based on a Difference-in-Differences Method. Sustainability 2024 , 16 , 7606. https://doi.org/10.3390/su16177606

Wang X, Xie S, Wei Z, Wang J. A Study on the Impact of China’s Prefabricated Building Policy on the Carbon Reduction Benefits of China’s Construction Industry Based on a Difference-in-Differences Method. Sustainability . 2024; 16(17):7606. https://doi.org/10.3390/su16177606

Wang, Xiangxiang, Shasha Xie, Zhe Wei, and Jinjing Wang. 2024. "A Study on the Impact of China’s Prefabricated Building Policy on the Carbon Reduction Benefits of China’s Construction Industry Based on a Difference-in-Differences Method" Sustainability 16, no. 17: 7606. https://doi.org/10.3390/su16177606

Article Metrics

Article access statistics, further information, mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals

• Chrome for Developers

Gemini Nano language detection API available for early preview

A language detection API is now available for local experimentation to our early preview program (EPP) participants. With this API, you can determine what language is being used on a web page.

The language detection APIs explainer is available as a proposal for the future development of this exploratory API and other APIs, including a translation API.

Language detection is the first step for translation. Browsers often already have language detection capabilities, and this API will allow web developers to access this technology with a JavaScript API.

As with our other APIs, we'll take your feedback to update the way language detection works, to ensure it meets the needs of developers and users. We hope to learn about the detection quality of summarization, feedback on the API design, and the impact of the current implementation in Chrome Canary.

Once you've signed up and been accepted to the EPP, you'll have access to a demo so you can experiment with this API.

Join the early preview program

As of now, the Prompt API, summarization API, and the language detection API are available for prototyping .

Sign up for the early preview program to gain access to the documentation and demos, stay up-to-date with the latest changes, and discover new APIs.

Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License , and code samples are licensed under the Apache 2.0 License . For details, see the Google Developers Site Policies . Java is a registered trademark of Oracle and/or its affiliates.

Last updated 2024-08-27 UTC.

• History & Society
• Science & Tech
• Biographies
• Animals & Nature
• Geography & Travel
• Arts & Culture
• Games & Quizzes
• On This Day
• One Good Fact
• New Articles
• Lifestyles & Social Issues
• Philosophy & Religion
• Politics, Law & Government
• World History
• Health & Medicine
• Browse Biographies
• Birds, Reptiles & Other Vertebrates
• Bugs, Mollusks & Other Invertebrates
• Environment
• Fossils & Geologic Time
• Entertainment & Pop Culture
• Sports & Recreation
• Visual Arts
• Demystified
• Image Galleries
• Infographics
• Top Questions
• Britannica Kids
• Saving Earth
• Space Next 50
• Student Center
• Introduction

Comparison with controlled study design

Natural experiments as quasi experiments, instrumental variables.

• Where was science invented?
• When did science begin?
• Is Internet technology "making us stupid"?
• What is the impact of artificial intelligence (AI) technology on society?

natural experiment

Our editors will review what you’ve submitted and determine whether to revise the article.

• Table Of Contents

natural experiment , observational study in which an event or a situation that allows for the random or seemingly random assignment of study subjects to different groups is exploited to answer a particular question. Natural experiments are often used to study situations in which controlled experimentation is not possible, such as when an exposure of interest cannot be practically or ethically assigned to research subjects. Situations that may create appropriate circumstances for a natural experiment include policy changes, weather events, and natural disasters. Natural experiments are used most commonly in the fields of epidemiology , political science , psychology , and social science .

Key features of experimental study design include manipulation and control. Manipulation, in this context , means that the experimenter can control which research subjects receive which exposures. For instance, subjects randomized to the treatment arm of an experiment typically receive treatment with the drug or therapy that is the focus of the experiment, while those in the control group receive no treatment or a different treatment. Control is most readily accomplished through random assignment, which means that the procedures by which participants are assigned to a treatment and control condition ensure that each has equal probability of assignment to either group. Random assignment ensures that individual characteristics or experiences that might confound the treatment results are, on average, evenly distributed between the two groups. In this way, at least one variable can be manipulated, and units are randomly assigned to the different levels or categories of the manipulated variables.

In epidemiology, the gold standard in research design generally is considered to be the randomized control trial (RCT). RCTs, however, can answer only certain types of epidemiologic questions, and they are not useful in the investigation of questions for which random assignment is either impracticable or unethical. The bulk of epidemiologic research relies on observational data, which raises issues in drawing causal inferences from the results. A core assumption for drawing causal inference is that the average outcome of the group exposed to one treatment regimen represents the average outcome the other group would have had if they had been exposed to the same treatment regimen. If treatment is not randomly assigned, as in the case of observational studies, the assumption that the two groups are exchangeable (on both known and unknown confounders) cannot be assumed to be true.

As an example, suppose that an investigator is interested in the effect of poor housing on health. Because it is neither practical nor ethical to randomize people to variable housing conditions, this subject is difficult to study using an experimental approach. However, if a housing policy change, such as a lottery for subsidized mortgages, was enacted that enabled some people to move to more desirable housing while leaving other similar people in their previous substandard housing, it might be possible to use that policy change to study the effect of housing change on health outcomes. In another example, a well-known natural experiment in Helena , Montana, smoking was banned from all public places for a six-month period. Investigators later reported a 60-percent drop in heart attacks for the study area during the time the ban was in effect.

Because natural experiments do not randomize participants into exposure groups, the assumptions and analytical techniques customarily applied to experimental designs are not valid for them. Rather, natural experiments are quasi experiments and must be thought about and analyzed as such. The lack of random assignment means multiple threats to causal inference , including attrition , history, testing, regression , instrumentation, and maturation, may influence observed study outcomes. For this reason, natural experiments will never unequivocally determine causation in a given situation. Nevertheless, they are a useful method for researchers, and if used with care they can provide additional data that may help with a research question and that may not be obtainable in any other way.

The major limitation in inferring causation from natural experiments is the presence of unmeasured confounding. One class of methods designed to control confounding and measurement error is based on instrumental variables (IV). While useful in a variety of applications, the validity and interpretation of IV estimates depend on strong assumptions, the plausibility of which must be considered with regard to the causal relation in question.

In particular, IV analyses depend on the assumption that subjects were effectively randomized, even if the randomization was accidental (in the case of an administrative policy change or exposure to a natural disaster) and adherence to random assignment was low. IV methods can be used to control for confounding in observational studies, to control for confounding due to noncompliance, and to correct for misclassification.

IV analysis, however, can produce serious biases in effect estimates. It can also be difficult to identify the particular subpopulation to which the causal effect IV estimate applies. Moreover, IV analysis can add considerable imprecision to causal effect estimates. Small sample size poses an additional challenge in applying IV methods.

Experimental Method In Psychology

Saul McLeod, PhD

Editor-in-Chief for Simply Psychology

BSc (Hons) Psychology, MRes, PhD, University of Manchester

Saul McLeod, PhD., is a qualified psychology teacher with over 18 years of experience in further and higher education. He has been published in peer-reviewed journals, including the Journal of Clinical Psychology.

Learn about our Editorial Process

Olivia Guy-Evans, MSc

Associate Editor for Simply Psychology

BSc (Hons) Psychology, MSc Psychology of Education

Olivia Guy-Evans is a writer and associate editor for Simply Psychology. She has previously worked in healthcare and educational sectors.

On This Page:

The experimental method involves the manipulation of variables to establish cause-and-effect relationships. The key features are controlled methods and the random allocation of participants into controlled and experimental groups .

What is an Experiment?

An experiment is an investigation in which a hypothesis is scientifically tested. An independent variable (the cause) is manipulated in an experiment, and the dependent variable (the effect) is measured; any extraneous variables are controlled.

An advantage is that experiments should be objective. The researcher’s views and opinions should not affect a study’s results. This is good as it makes the data more valid  and less biased.

There are three types of experiments you need to know:

1. Lab Experiment

A laboratory experiment in psychology is a research method in which the experimenter manipulates one or more independent variables and measures the effects on the dependent variable under controlled conditions.

A laboratory experiment is conducted under highly controlled conditions (not necessarily a laboratory) where accurate measurements are possible.

The researcher uses a standardized procedure to determine where the experiment will take place, at what time, with which participants, and in what circumstances.

Participants are randomly allocated to each independent variable group.

Examples are Milgram’s experiment on obedience and  Loftus and Palmer’s car crash study .

• Strength : It is easier to replicate (i.e., copy) a laboratory experiment. This is because a standardized procedure is used.
• Strength : They allow for precise control of extraneous and independent variables. This allows a cause-and-effect relationship to be established.
• Limitation : The artificiality of the setting may produce unnatural behavior that does not reflect real life, i.e., low ecological validity. This means it would not be possible to generalize the findings to a real-life setting.
• Limitation : Demand characteristics or experimenter effects may bias the results and become confounding variables .

2. Field Experiment

A field experiment is a research method in psychology that takes place in a natural, real-world setting. It is similar to a laboratory experiment in that the experimenter manipulates one or more independent variables and measures the effects on the dependent variable.

However, in a field experiment, the participants are unaware they are being studied, and the experimenter has less control over the extraneous variables .

Field experiments are often used to study social phenomena, such as altruism, obedience, and persuasion. They are also used to test the effectiveness of interventions in real-world settings, such as educational programs and public health campaigns.

An example is Holfing’s hospital study on obedience .

• Strength : behavior in a field experiment is more likely to reflect real life because of its natural setting, i.e., higher ecological validity than a lab experiment.
• Strength : Demand characteristics are less likely to affect the results, as participants may not know they are being studied. This occurs when the study is covert.
• Limitation : There is less control over extraneous variables that might bias the results. This makes it difficult for another researcher to replicate the study in exactly the same way.

3. Natural Experiment

A natural experiment in psychology is a research method in which the experimenter observes the effects of a naturally occurring event or situation on the dependent variable without manipulating any variables.

Natural experiments are conducted in the day (i.e., real life) environment of the participants, but here, the experimenter has no control over the independent variable as it occurs naturally in real life.

Natural experiments are often used to study psychological phenomena that would be difficult or unethical to study in a laboratory setting, such as the effects of natural disasters, policy changes, or social movements.

For example, Hodges and Tizard’s attachment research (1989) compared the long-term development of children who have been adopted, fostered, or returned to their mothers with a control group of children who had spent all their lives in their biological families.

Here is a fictional example of a natural experiment in psychology:

Researchers might compare academic achievement rates among students born before and after a major policy change that increased funding for education.

In this case, the independent variable is the timing of the policy change, and the dependent variable is academic achievement. The researchers would not be able to manipulate the independent variable, but they could observe its effects on the dependent variable.

• Strength : behavior in a natural experiment is more likely to reflect real life because of its natural setting, i.e., very high ecological validity.
• Strength : Demand characteristics are less likely to affect the results, as participants may not know they are being studied.
• Strength : It can be used in situations in which it would be ethically unacceptable to manipulate the independent variable, e.g., researching stress .
• Limitation : They may be more expensive and time-consuming than lab experiments.
• Limitation : There is no control over extraneous variables that might bias the results. This makes it difficult for another researcher to replicate the study in exactly the same way.

Key Terminology

Ecological validity.

The degree to which an investigation represents real-life experiences.

Experimenter effects

These are the ways that the experimenter can accidentally influence the participant through their appearance or behavior.

Demand characteristics

The clues in an experiment lead the participants to think they know what the researcher is looking for (e.g., the experimenter’s body language).

Independent variable (IV)

The variable the experimenter manipulates (i.e., changes) is assumed to have a direct effect on the dependent variable.

Dependent variable (DV)

Variable the experimenter measures. This is the outcome (i.e., the result) of a study.

Extraneous variables (EV)

All variables which are not independent variables but could affect the results (DV) of the experiment. EVs should be controlled where possible.

Confounding variables

Variable(s) that have affected the results (DV), apart from the IV. A confounding variable could be an extraneous variable that has not been controlled.

Random Allocation

Randomly allocating participants to independent variable conditions means that all participants should have an equal chance of participating in each condition.

The principle of random allocation is to avoid bias in how the experiment is carried out and limit the effects of participant variables.

Order effects

Changes in participants’ performance due to their repeating the same or similar test more than once. Examples of order effects include:

(i) practice effect: an improvement in performance on a task due to repetition, for example, because of familiarity with the task;

(ii) fatigue effect: a decrease in performance of a task due to repetition, for example, because of boredom or tiredness.

What Are Natural Experiments and How Do Economists Use Them?

Natural Experiments vs. Controlled Experiments

Cultura RM Exclusive/Matt Lincoln/Getty Images

• U.S. Economy
• Supply & Demand
• Archaeology
• Ph.D., Business Administration, Richard Ivey School of Business
• M.A., Economics, University of Rochester
• B.A., Economics and Political Science, University of Western Ontario

A natural experiment is an empirical or observational study in which the control and experimental variables of interest are not artificially manipulated by researchers but instead are allowed to be influenced by nature or factors outside of the researchers' control. Unlike traditional randomized experiments, natural experiments are not controlled by researchers but rather observed and analyzed.

Natural Experiments Versus Observational Studies

So if natural experiments are not controlled but rather observed by researchers, what is there to distinguish them from purely observational studies? The answer is that natural experiments still follow the primary principles of experimental study. Natural experiments are most effective when they mimic as closely as possible the existence of test and control groups of controlled experiments, which is to say that there is a clearly defined exposure to some condition in a clearly defined population and the absence of that exposure in another similar population for comparison. When such groups are present, the processes behind natural experiments are said to resemble randomization even when researchers do not interfere.

Under these conditions, observed outcomes of natural experiments can feasibly be credited to the exposure meaning that there is some cause for belief in a causal relationship as opposed to simple correlation. It is this characteristic of natural experiments — the effective comparison that makes a case for the existence of a causal relationship — that distinguishes natural experiments from purely non-experimental observational studies. But that is not to say that natural experiments aren't without their critics and validation difficulties. In practice, the circumstances surrounding a natural experiment are often complex and their observations will never unequivocally prove causation. Instead, they provide an important inferential method through which researchers can gather information about a research question upon which data might otherwise not be available.

Natural Experiments in Economics

In the social sciences, particularly economics, the expensive nature and limitations of traditionally controlled experiments involving human subjects has long been recognized as a limitation for the development and progress of the field. As such, natural experiments provide a rare testing ground for economists and their colleagues. Natural experiments are used when such controlled experimentation would be too difficult, expensive, or unethical as is the case with many human experiments. Opportunities for natural experimentation are of the utmost importance to subjects like epidemiology or the study of health and disease conditions in defined populations in which experimental study would problematic, to say the least. But natural experiments are also used by researchers in the field of economics to study otherwise difficult to test subjects and are often possible when there is some change in law, policy, or practice in a defined space like a nation, jurisdiction, or even social group. Some examples of economics research questions that have been studied through natural experimentation include:

• The "return on investment" of higher education in American adults
• The effect of military service on lifetime earning 
• The effect of public smoking bans on hospital admissions

Journal Articles on Natural Experiment:

• The Economic Consequences of Unwed Motherhood: Using Twin Births as a Natural Experiment
• Natural and Quasi-Experiments in Economics
• A Natural Experiment in "Jeopardy!"
• Cost-Push Inflation vs. Demand-Pull Inflation
• What Is International Economics?
• Definition and Use of Instrumental Variables in Econometrics
• So What Exactly Do Economists Do?
• What Is Microeconomics?
• What Is a Commodity in Economics?
• What Are the Various Subfields of Economics?
• Overview of Cost Curves in Economics
• What Is Classical Liberalism? Definition and Examples
• Online Microeconomics Textbook
• White Noise Process Definition
• Understand the Economic Concept of a Budget Line
• What Is a Plant in the Study of Economics?
• Quasiconcave Utility Functions
• Learn the Definition What Is Okun's Law in Economics

Reference Library

Collections

• See what's new
• All Resources
• Student Resources
• Assessment Resources
• Teaching Resources
• CPD Courses
• Livestreams

Study notes, videos, interactive activities and more!

Psychology news, insights and enrichment

Currated collections of free resources

Browse resources by topic

• All Psychology Resources

Resource Selections

Currated lists of resources

Study Notes

Natural Experiments

Last updated 22 Mar 2021

• Share on Facebook
• Share on Twitter
• Share by Email

Experiments look for the effect that manipulated variables (independent variables, or IVs) have on measured variables (dependent variables, or DVs), i.e. causal effects.

Natural experiments are studies where the experimenter cannot manipulate the IV, so the DV is simply measured and judged as the effect of an IV. For this reason, participants cannot be randomly allocated to experimental groups as they are already pre-set, making them quasi-experiments . For instance, an experiment might investigate the relative levels of aggression observed in boys and girls in a primary school (the experimenter cannot manipulate who belongs to the ‘boy’ and ‘girl’ groups).

Evaluation of natural experiments:

- The natural settings where such experiments take place mean that results will have high ecological validity (i.e. they should relate well to real life behaviour).

- Demand characteristics are often not a problem, unlike laboratory experiments (i.e. participants are less likely to adjust their natural behaviour according to their interpretation of the study’s purpose, as they might not know they are taking part in a study).

- Being unable to randomly allocate participants to conditions means that sample bias may be an issue (e.g. other extraneous variables that change with the pre-set IV group differences may confound the results, meaning a causal IV-DV effect is unlikely).

- Ethical issues such as lack of informed consent commonly arise, as deception is often required; debriefing, once the observation/experiment has ended, is necessary.

• Natural Experiment

You might also like

Types of experiment: overview, a level psychology topic quiz - research methods.

Quizzes & Activities

Research Methods: MCQ Revision Test 1 for AQA A Level Psychology

Topic Videos

Example Answers for Research Methods: A Level Psychology, Paper 2, June 2018 (AQA)

Exam Support

Example Answers for Research Methods: A Level Psychology, Paper 2, June 2019 (AQA)

Our subjects.

• › Criminology
• › Economics
• › Geography
• › Health & Social Care
• › Psychology
• › Sociology
• › Teaching & learning resources
• › Student revision workshops
• › Online student courses
• › CPD for teachers
• › Livestreams
• › Teaching jobs

Boston House, 214 High Street, Boston Spa, West Yorkshire, LS23 6AD Tel: 01937 848885

• › Contact us
• › Terms of use
• › Privacy & cookies

© 2002-2024 Tutor2u Limited. Company Reg no: 04489574. VAT reg no 816865400.

• A-Z Publications

Annual Review of Public Health

Volume 38, 2017, review article, open access, natural experiments: an overview of methods, approaches, and contributions to public health intervention research.

• Peter Craig 1 , Srinivasa Vittal Katikireddi 1 , Alastair Leyland 1 , and Frank Popham 1
• View Affiliations Hide Affiliations Affiliations: MRC/CSO Social and Public Health Sciences Unit, University of Glasgow, Glasgow G2 3QB, United Kingdom; email: [email protected] , [email protected] , [email protected] , [email protected]
• Vol. 38:39-56 (Volume publication date March 2017) https://doi.org/10.1146/annurev-publhealth-031816-044327
• First published as a Review in Advance on January 11, 2017
• Copyright © 2017 Annual Reviews. This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 (CC-BY-SA) International License, which permits unrestricted use, distribution, and reproduction in any medium and any derivative work is made available under the same, similar, or a compatible license. See credit lines of images or other third-party material in this article for license information.

Population health interventions are essential to reduce health inequalities and tackle other public health priorities, but they are not always amenable to experimental manipulation. Natural experiment (NE) approaches are attracting growing interest as a way of providing evidence in such circumstances. One key challenge in evaluating NEs is selective exposure to the intervention. Studies should be based on a clear theoretical understanding of the processes that determine exposure. Even if the observed effects are large and rapidly follow implementation, confidence in attributing these effects to the intervention can be improved by carefully considering alternative explanations. Causal inference can be strengthened by including additional design features alongside the principal method of effect estimation. NE studies often rely on existing (including routinely collected) data. Investment in such data sources and the infrastructure for linking exposure and outcome data is essential if the potential for such studies to inform decision making is to be realized.

Article metrics loading...

Full text loading...

Literature Cited

• Abadie A , Diamond A , Hainmueller J . 1.  2010 . Synthetic control methods for comparative case studies: estimating the effect of California's Tobacco Control Program. J. Am. Stat. Assoc. 105 : 493– 505 [Google Scholar]
• Abadie A , Diamond A , Hainmueller J . 2.  2011 . Synth: an R package for synthetic control methods in comparative case studies. J. Stat. Softw. 42 : 1– 17 [Google Scholar]
• Abadie A , Diamond A , Hainmueller J . 3.  2015 . Comparative politics and the synthetic control method. Am. J. Polit. Sci. 50 : 495– 510 [Google Scholar]
• Abadie A , Gardeazabal J . 4.  2003 . The economic costs of conflict: a case study of the Basque Country. Am. Econ. Rev. 93 : 113– 32 [Google Scholar]
• 5.  Acad. Med. Sci. 2007 . Identifying the Environmental Causes of Disease: How Should We Decide What to Believe and When to Take Action. London: Acad. Med. Sci. [Google Scholar]
• Andalon M . 6.  2011 . Impact of Oportunidades in overweight and obesity in Mexico. Health Econ 20 : Suppl. 1 1– 18 [Google Scholar]
• Austin PC . 7.  2011 . An introduction to propensity score methods for reducing the effects of confounding in observational studies. Multivar. Behav. Res. 46 : 399– 424 [Google Scholar]
• Basu S , Rehkopf DH , Siddiqi A , Glymour MM , Kawachi I . 8.  2016 . Health behaviors, mental health, and health care utilization among single mothers after welfare reforms in the 1990s. Am. J. Epidemiol. 83 : 531– 38 [Google Scholar]
• Bauhoff S . 9.  2014 . The effect of school district nutrition policies on dietary intake and overweight: a synthetic control approach. Econ. Hum. Biol. 12 : 45– 55 [Google Scholar]
• Bonell C , Fletcher A , Morton M , Lorenc T , Moore L . 10.  2012 . Realist randomised controlled trials: a new approach to evaluating complex public health interventions. Soc. Sci. Med. 75 : 2299– 306 [Google Scholar]
• Bor J , Moscoe E , Mutevedzi P , Newell M-L , Barnighausen T . 11.  2014 . Regression discontinuity designs in epidemiology: causal inference without randomized trials. Epidemiology 25 : 729– 37 [Google Scholar]
• Boyd J , Ferrante AM , O'Keefe C , Bass AJ , Randall AM . 12.  et al. 2012 . Data linkage infrastructure for cross-jurisdictional health-related research in Australia. BMC Health Serv. Res. 2 : 480 [Google Scholar]
• Brown J , Neary J , Katikireddi SV , Thomson H , McQuaid RW . 13.  et al. 2015 . Protocol for a mixed-methods longitudinal study to identify factors influencing return to work in the over 50s participating in the UK Work Programme: Supporting Older People into Employment (SOPIE). BMJ Open 5 : e010525 [Google Scholar]
• Chattopadhyay R , Duflo E . 14.  2004 . Women as policy makers: evidence from a randomised policy experiment in India. Econometrica 72 : 1409– 43 [Google Scholar]
• 15.  Comm. Soc. Determinants Health. 2008 . Closing the Gap in a Generation: Health Equity Through Action on the Social Determinants of Health. Final Report of the Commission on Social Determinants of Health. Geneva: World Health Organ. [Google Scholar]
• Craig P , Cooper C , Gunnell D , Macintyre S , Petticrew M . 16.  et al. 2012 . Using natural experiments to evaluate population health interventions: new Medical Research Council guidance. J. Epidemiol. Community Health 66 : 1182– 86 [Google Scholar]
• Crifasi CK , Meyers JS , Vernick JS , Webster DW . 17.  2015 . Effects of changes in permit-to-purchase handgun laws in Connecticut and Missouri on suicide rates. Prev. Med. 79 : 43– 49 [Google Scholar]
• D'Agostino RB . 18.  1998 . Tutorial in biostatistics. Propensity score methods for bias reduction in the comparison of a treatment to a non-randomized control group. Stat. Med. 17 : 2265– 81 [Google Scholar]
• De Angelo G , Hansen B . 19.  2014 . Life and death in the fast lane: police enforcement and traffic fatalities. Am. Econ. J. Econ. Policy 6 : 231– 57 [Google Scholar]
• Deaton A . 20.  2010 . Instruments, randomisation and learning about development. J. Econ. Lit. 48 : 424– 55 [Google Scholar]
• Dundas R , Ouédraogo S , Bond L , Briggs AH , Chalmers J . 21.  et al. 2014 . Evaluation of health in pregnancy grants in Scotland: a protocol for a natural experiment. BMJ Open 4 : e006547 [Google Scholar]
• Dunning T . 22.  2012 . Natural Experiments in the Social Sciences: A Design-Based Approach Cambridge, UK: Cambridge Univ. Press [Google Scholar]
• Dusheiko M , Gravelle H , Jacobs R , Smith P . 23.  2006 . The effect of financial incentives on gatekeeping doctors: evidence from a natural experiment. J. Health Econ. 25 : 449– 78 [Google Scholar]
• Eadie D , Heim D , MacAskill S , Ross A , Hastings G , Davies J . 24.  2008 . A qualitative analysis of compliance with smoke-free legislation in community bars in Scotland: implications for public health. Addiction 103 : 1019– 26 [Google Scholar]
• Fall T , Hägg S , Mägi R , Ploner A , Fischer K . 25.  et al. 2013 . The role of adiposity in cardiometabolic traits: a Mendelian randomization analysis. PLOS Med 10 : e1001474 [Google Scholar]
• 26.  Farr Inst. Health Inf. Res. 2016 . Environmental and public health research. Farr Inst. Health Inf. Res. Dundee, UK: http://www.farrinstitute.org/research-education/research/environmental-and-public-health [Google Scholar]
• 27.  Foresight 2007 . Tackling Obesities: Future Choices. Challenges for Research and Research Management. London: Gov. Off. Sci. [Google Scholar]
• Fuller T , Peters J , Pearson M , Anderson R . 28.  2014 . Impact of the transparent reporting of evaluations with nonrandomized designs reporting guideline: ten years on. Am. J. Public Health 104 : e110– 17 [Google Scholar]
• Goodman A , van Sluijs EMF , Ogilvie D . 29.  2016 . Impact of offering cycle training in schools upon cycling behaviour: a natural experimental study. Int. J. Behav. Nutr. Phys. Act. 13 : 34 [Google Scholar]
• Green CP , Heywood JS , Navarro M . 30.  2014 . Did liberalising bar hours decrease traffic accidents?. J. Health Econ. 35 : 189– 98 [Google Scholar]
• Grundy C , Steinbach R , Edwards P , Green J , Armstrong B . 31.  et al. 2009 . Effect of 20 mph traffic speed zones on road injuries in London, 1986–2006: controlled interrupted time series analysis. BMJ 339 : b4469 [Google Scholar]
• Gunnell D , Fernando R , Hewagama M , Priyangika W , Konradsen F , Eddleston M . 32.  2007 . The impact of pesticide regulations on suicide in Sri Lanka. Int. J. Epidemiol. 36 : 1235– 42 [Google Scholar]
• Heckman JJ . 33.  1995 . Randomization as an instrumental variable. Rev. Econ. Stat. 78 : 336– 41 [Google Scholar]
• Hernán M , Robins J . 34.  2017 . Causal Inference Boca Raton, FL: Chapman Hall/CRC In press [Google Scholar]
• Hernán M , Robins JM . 35.  2006 . Instruments for causal inference. An epidemiologist's dream. Epidemiology 17 : 360– 72 [Google Scholar]
• Holmes MV , Dale CE , Zuccolo L , Silverwood RJ , Guo Y . 36.  et al. 2014 . Association between alcohol and cardiovascular disease: Mendelian randomisation analysis based on individual participant data. BMJ 349 g4164 [Google Scholar]
• 37.  House Commons Sci. Technol. Comm 2016 . The Big Data Dilemma. Fourth Report of Session 2015–16. HC 468 London: Station. Off. Ltd. [Google Scholar]
• Humphreys DK , Panter J , Sahlqvist S , Goodman A , Ogilvie D . 38.  2016 . Changing the environment to improve population health: a framework for considering exposure in natural experimental studies. J. Epidemiol. Community Health. doi: 10.1136/jech-2015-206381 [Google Scholar]
• Ichida Y , Hirai H , Kondo K , Kawachi I , Takeda T , Endo H . 39.  2013 . Does social participation improve self-rated health in the older population? A quasi-experimental intervention study. Soc. Sci. Med. 94 : 83– 90 [Google Scholar]
• 40.  IOM (Inst. Med.) 2010 . Bridging the Evidence Gap in Obesity Prevention: A Framework to Inform Decision Making Washington, DC: Natl. Acad. Press [Google Scholar]
• Jones A , Rice N . 41.  2009 . Econometric evaluation of health policies HEDG Work. Pap. 09/09 Univ. York [Google Scholar]
• Katikireddi SV , Bond L , Hilton S . 42.  2014 . Changing policy framing as a deliberate strategy for public health advocacy: a qualitative policy case study of minimum unit pricing of alcohol. Milbank Q 92 : 250– 83 [Google Scholar]
• Katikireddi SV , Der G , Roberts C , Haw S . 43.  2016 . Has childhood smoking reduced following smoke-free public places legislation? A segmented regression analysis of cross-sectional UK school-based surveys. Nicotine Tob. Res. 18 : 1670– 74 [Google Scholar]
• Kontopantelis E , Doran T , Springate DA , Buchan I , Reeves D . 44.  2015 . Regression based quasi-experimental approach when randomisation is not an option: interrupted time series analysis. BMJ 350 : h2750 [Google Scholar]
• Kreif N , Grieve R , Hangartner D , Nikolova S , Turner AJ , Sutton M . 45.  2015 . Examination of the synthetic control method for evaluating health policies with multiple treated units. Health Econ doi: 10.1002/hec.3258 [Google Scholar]
• Labrecque JA , Kaufman JS . 46.  2016 . Can a quasi-experimental design be a better idea than an experimental one?. Epidemiology 27 : 500– 2 [Google Scholar]
• Lee DS , Lemieux T . 47.  2010 . Regression discontinuity designs in economics. J. Econ. Lit. 48 : 281– 355 [Google Scholar]
• Lewis SJ , Araya R , Davey Smith G , Freathy R , Gunnell D . 48.  et al. 2011 . Smoking is associated with, but does not cause, depressed mood in pregnancy—a Mendelian randomization study. PLOS ONE 6 : e21689 [Google Scholar]
• Linden A , Adams JL . 49.  2011 . Applying a propensity score-based weighting model to interrupted time series data: improving causal inference in programme evaluation. J. Eval. Clin. Pract. 17 : 1231– 38 [Google Scholar]
• Linden A , Adams JL . 50.  2012 . Combining the regression discontinuity design and propensity score-based weighting to improve causal inference in program evaluation. J. Eval. Clin. Pract. 18 : 317– 25 [Google Scholar]
• Little RJ , Rubin DB . 51.  2000 . Causal effects in clinical and epidemiological studies via potential outcomes: concepts and analytical approaches. Annu. Rev. Public Health 21 : 121– 45 [Google Scholar]
• Ludwig J , Miller D . 52.  2007 . Does Head Start improve children's life chances? Evidence from an RD design. Q. J. Econ. 122 : 159– 208 [Google Scholar]
• Mcleod AI , Vingilis ER . 53.  2008 . Power computations in time series analyses for traffic safety interventions. Accid. Anal. Prev. 40 : 1244– 48 [Google Scholar]
• Melhuish E , Belsky J , Leyland AH , Barnes J . 54.  Natl. Eval. Sure Start Res. Team. 2008 . Effects of fully-established Sure Start Local Programmes on 3-year-old children and their families living in England: a quasi-experimental observational study. Lancet 372 : 1641– 47 [Google Scholar]
• Messer LC , Oakes JM , Mason S . 55.  2010 . Effects of socioeconomic and racial residential segregation on preterm birth: a cautionary tale of structural confounding. Am. J. Epidemiol. 171 : 664– 73 [Google Scholar]
• Moore G , Audrey S , Barker M , Bond L , Bonell C . 56.  et al. 2015 . MRC process evaluation of complex intervention. Medical Research Council guidance. BMJ 350 : h1258 [Google Scholar]
• Moscoe E , Bor J , Barnighausen T . 57.  2015 . Regression discontinuity designs are under-used in medicine, epidemiology and public health: a review of current and best practice. J. Clin. Epidemiol. 68 : 132– 43 [Google Scholar]
• Nandi A , Hajizadeh M , Harper S , Koski A , Strumpf EC , Heymann J . 58.  2016 . Increased duration of paid maternity leave lowers infant mortality in low and middle-income countries: a quasi-experimental study. PLOS Med. 13 : e1001985 [Google Scholar]
• Pega F , Blakely T , Glymour MM , Carter KN , Kawachi I . 59.  2016 . Using marginal structural modelling to estimate the cumulative impact of an unconditional tax credit on self-rated health. Am. J. Epidemiol. 183 : 315– 24 [Google Scholar]
• Phillips R , Amos A , Ritchie D , Cunningham-Burley S , Martin C . 60.  2007 . Smoking in the home after the smoke-free legislation in Scotland: qualitative study. BMJ 335 : 553 [Google Scholar]
• Ramsay CR , Matowe L , Grilli R , Grimshaw JM , Thomas RE . 61.  2005 . Interrupted time series designs in health technology assessment: lessons from two systematic reviews of behaviour change strategies. Int. J. Technol. Assess. Health Care 19 : 613– 23 [Google Scholar]
• Restrepo BJ , Rieger M . 62.  2016 . Denmark's policy on artificial trans fat and cardiovascular disease. Am. J. Prev. Med. 50 : 69– 76 [Google Scholar]
• Rihoux B , Ragin C . 63.  2009 . Configurational Comparative Methods: Qualitative Comparative Analysis (QCA) and Related Techniques London: Sage [Google Scholar]
• Robinson M , Geue C , Lewsey J , Mackay D , McCartney G . 64.  et al. 2014 . Evaluating the impact of the Alcohol Act on off-trade alcohol sales: a natural experiment in Scotland. Addiction 109 : 2035– 43 [Google Scholar]
• Rosenbaum PR , Rubin DB . 65.  1983 . The central role of the propensity score in observational studies for causal effects. Biometrika 70 : 41– 55 [Google Scholar]
• Rubin DB . 66.  2008 . For objective causal inference, design trumps analysis. Ann. Appl. Stat. 2 : 808– 40 [Google Scholar]
• Ryan AM , Krinsky S , Kontopantelis E , Doran T . 67.  2016 . Long-term evidence for the effect of pay-for-performance in primary care on mortality in the UK: a population study. Lancet 388 : 268– 74 [Google Scholar]
• Sanson-Fisher RW , D'Este CS , Carey ML , Noble N , Paul CL . 68.  2014 . Evaluation of systems-oriented public health interventions: alternative research designs. Annu. Rev. Public Health 35 : 9– 27 [Google Scholar]
• Shadish WR , Cook TD , Campbell DT . 69.  2002 . Experimental and Quasi-Experimental Designs for Generalized Causal Inference New York: Houghton Mifflin [Google Scholar]
• Shah BR , Laupacis A , Hux JE , Austin PC . 70.  2005 . Propensity score methods gave similar results to traditional regression modeling in observational studies: a systematic review. J. Clin. Epidemiol. 58 : 550– 59 [Google Scholar]
• Swanson SA , Hernán MA . 71.  2013 . Commentary: how to report instrumental variable analyses (suggestions welcome). Epidemiology 24 : 370– 74 [Google Scholar]
• Thomas J , O'Mara-Eves A , Brunton G . 72.  2014 . Using qualitative comparative analysis (QCA) in systematic reviews of complex interventions: a worked example. Syst. Rev. 3 : 67 [Google Scholar]
• van Leeuwen N , Lingsma HF , de Craen T , Nieboer D , Mooijaart S . 73.  et al. 2016 . Regression discontinuity design: simulation and application in two cardiovascular trials with continuous outcomes. Epidemiology 27 : 503– 11 [Google Scholar]
• von Elm E , Altman DG , Egger M , Pocock SJ , Gøtzsche PC . 74.  et al. 2008 . Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. J. Clin. Epidemiol. 61 : 344– 49 [Google Scholar]
• Wagner AK , Soumerai SB , Zhang F , Ross-Degnan D . 75.  2002 . Segmented regression analysis of interrupted time series studies in medication use research. J. Clin. Pharm. Ther. 27 : 299– 309 [Google Scholar]
• Wanless D . 76.  2004 . Securing Good Health for the Whole Population London: HM Treas. [Google Scholar]
• Warren J , Wistow J , Bambra C . 77.  2013 . Applying qualitative comparative analysis (QCA) to evaluate a public health policy initiative in the North East of England. Policy Soc 32 : 289– 301 [Google Scholar]
• Yen ST , Andrews M , Chen Z , Eastwood DB . 78.  2008 . Food stamp program participation and food insecurity: an instrumental variables approach. Am. J. Agric. Econ. 90 : 117– 32 [Google Scholar]

Data & Media loading...

• Article Type: Review Article

Most Read This Month

Most cited most cited rss feed, review of community-based research: assessing partnership approaches to improve public health, nature and health, measuring social class in us public health research: concepts, methodologies, and guidelines, the epidemiology of depression across cultures, acute respiratory effects of particulate air pollution, racism and health: evidence and needed research, the social determinants of health: coming of age, the role of behavioral science theory in development and implementation of public health interventions, mediation analysis: a practitioner's guide, the prescription opioid and heroin crisis: a public health approach to an epidemic of addiction.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Cambridge Open

Do natural experiments have an important future in the study of mental disorders?

Anita thapar.

1 Child & Adolescent Psychiatry Section, Division of Psychological Medicine and Clinical Neurosciences, MRC Centre for Neuropsychiatric Genetics and Genomics, Cardiff University School of Medicine, Hadyn Ellis Building, Maindy Road, Cathays, Cardiff, CF24 4HQ, UK

Michael Rutter

2 MRC SGDP Centre, Institute of Psychiatry, Psychology and Neuroscience, King's College, London, UK

There is an enormous interest in identifying the causes of psychiatric disorders but there are considerable challenges in identifying which risks are genuinely causal. Traditionally risk factors have been inferred from observational designs. However, association with psychiatric outcome does not equate to causation. There are a number of threats that clinicians and researchers face in making causal inferences from traditional observational designs because adversities or exposures are not randomly allocated to individuals. Natural experiments provide an alternative strategy to randomized controlled trials as they take advantage of situations whereby links between exposure and other variables are separated by naturally occurring events or situations. In this review, we describe a growing range of different types of natural experiment and highlight that there is a greater confidence about findings where there is a convergence of findings across different designs. For example, exposure to hostile parenting is consistently found to be associated with conduct problems using different natural experiment designs providing support for this being a causal risk factor. Different genetically informative designs have repeatedly found that exposure to negative life events and being bullied are linked to later depression. However, for exposure to prenatal cigarette smoking, while findings from natural experiment designs are consistent with a causal effect on offspring lower birth weight, they do not support the hypothesis that intra-uterine cigarette smoking has a causal effect on attention-deficit/hyperactivity disorder and conduct problems and emerging findings highlight caution about inferring causal effects on bipolar disorder and schizophrenia.

Introduction

Psychiatric disorders have a complex etiology; influenced by multiple genetic as well as environmental risk factors. Although most are heritable, in the shorter term, environmental factors are more tractable to modification. If environmental exposures are causal, then their modification should lead to improvements in population health or the psychiatric disorder being treated (see Table 1 : meaning of key terms). Thus it is important to assess which of our selected environmental exposures are genuinely causal – but this is challenging.

Key terms and what they mean

Kraemer et al ., 1997 ; Sedgwick, 2013 ; Arnold and Ercumen, 2016 .

Traditionally many of the exposures that we believe to be risk factors for psychiatric disorder have been implicated through observational designs. These infer causation from observations of association. However, association is not causation. Threats to causal inference include reverse causation, confounding, and selection bias (Rutter, 2007 ; Thapar and Rutter, 2015 ). For example, has the supposed risk factor of family discord arisen as the result of the individual's psychiatric disorder – reverse causation? Has the common factor of social disadvantage contributed to both the outcome of psychiatric disorder and family discord – confounding? Does cannabis have a causal risk effect on schizophrenia or is it that those with a higher propensity to develop schizophrenia are more likely to use cannabis – selection bias?

These are important threats because if they lead to misleading and inconsistent conclusions, this confuses clinicians, researchers, the general public, and patients. At worst it leads to wasted resources. The challenges in inferring causality are not just restricted to psychiatry. For example, observational studies suggested vitamin E had a protective effect on cardiovascular disease until randomized controlled trials (RCTs) suggested that this was not the case (Eidelman et al ., 2004 ). RCTs are often considered as the ‘gold standard’ for assessing causal effects. However, given that RCTs of many environmental exposures relevant to psychopathology are not going to be feasible or ethical, what is the alternative?

‘Natural experiments’ provide an alternative strategy. We refer to designs that take advantage of situations whereby links between the exposure and other variables are separated by naturally occurring events or situations. Unlike RCTs, the manipulation is not undertaken by the researcher. Some involve the design and others the statistical methods.

In this review, we will consider some types of natural experiments and describe how they have been applied in the field of psychiatry. The aim is not to provide an exhaustive account of different methods but rather to focus on the principles, design, and limitations. There are a number of other methods that in the interests of space will not be covered in this review but are discussed elsewhere (Rutter and Thapar, 2018 ). Although there are other reviews (e.g. Pingault et al ., 2018 ), we aim to describe a broad range of designs and will provide examples of findings that would be relevant to a clinician.

There is a growing trend toward viewing causal inference as a single approach based on considering what would have occurred if an individual had not been exposed to the risk? [see Krieger and Davey Smith ( 2016 ) for an excellent discussion]. However, we agree with Krieger and Davey Smith ( 2016 ) for taking a broader view; one that emphasizes convergence or ‘triangulation’ of findings across diverse types of designs that have different types of biases and assumptions. When the same finding is observed using different approaches, it provides greater confidence in inferring causality especially when such studies are conducted in different populations.

Genetically informative designs that remove familial and genetic confounding

Many of the most important risk factors for psychopathology, such as life events and inter-personal discord, are person-dependent; they are not randomly allocated. Thus, it is unsurprising that decades of research have shown that many types of adversities run in families and are heritable (e.g. McGuffin et al ., 1988 ; Plomin, 2018 ). This raises the possibility that an association between exposure and psychiatric outcome could arise through familial or genetic confounding (Thapar and Rutter, 2009 , 2015 ).

It is for this reason that genetically informative designs such as twin studies have been invaluable for testing whether links between environmental exposures and psychopathology remain associated once genetic or familial confounds are taken into account.

Some designs, such as the discordant sib pair and in vitro fertilization (IVF) design (Thapar et al ., 2007 ), enable removal of genetic or familial confounds for prenatal exposures. For example, prenatal exposure to cigarette smoke has been linked with later risk for offspring attention-deficit/hyperactivity disorder (ADHD), conduct disorder, bipolar disorder, and schizophrenia. The effects could potentially be causal; for example, mediated by effects of nicotine on the developing brain. However, unmeasured confounds and selection biases are a concern, meaning that natural experiment designs have proved very useful here (Quinn et al ., 2017 ; Rice et al ., 2018 ).

Twin and adoption studies are not able to separate genetic confounds for prenatal exposures. That is because twins share their prenatal exposures and varying degrees of genetic liability and for adopted offspring, it is their biological mother who provides both the prenatal environmental and half of their genetic makeup. However, such designs are well-suited for assessing post-natal exposures. Some designs such as the children-of-twins design (D'Onofrio et al ., 2003 ) and adoption designs are especially well-suited for examining cross-generational environmental as well as genetic transmission (see Table 2 ).

Genetically informative designs and what they can be used to assess

Maternal v. paternal exposure during pregnancy

One method that has been used to disaggregate intra-uterine and genetic or house-hold/familial-level influences involves testing associations between maternal v. paternal exposures during pregnancy and offspring outcomes ( Fig. 1 ). If the link is mediated by an intra-uterine effect, a stronger association would be expected for the maternal exposure. For example, in a UK population-birth cohort ALSPAC, strong associations were observed between maternal smoking in pregnancy and shorter birth length (Howe et al ., 2016 ) and lower birth weight in offspring (Langley et al ., 2012 ) that were not observed when exposure to paternal smoking was examined (see Table 1 ; this is an example of a negative control exposure). However, in this same cohort, associations between exposure to smoking in pregnancy and ADHD were as strong for maternal exposures as they were for paternal exposures even in the case of mothers who did not smoke. These results held when the contribution of additional passive smoking was considered. There are limitations to this design including the fact that parents will show similarities in exposures due to genetic (assortative mating) and social reasons and it is restricted to the sorts of exposures that both parents could feasibly experience in pregnancy.

Maternal v. paternal exposure.

Discordant sibling pair design

Full biologic siblings share on average 50% of their genome. Thus differences between them can be used to assess family-level confounds that include genetic and shared environmental contributions.

(i) Prenatal exposures. As they share the same mother, they become of special interest when they have been differentially exposed to prenatal factors. For example, taking the example of maternal smoking in pregnancy and ADHD, eight studies of discordant sibling pairs have now found that the siblings who were unexposed to smoking in utero showed elevated levels of ADHD (Rice et al ., 2018 ). Similar findings were observed for conduct problems. Birth weight provided the negative control as the studies that examined this outcome found that the association with cigarette smoking remained strong. A recent, large discordant sibling study also failed to find support for a causal effect of exposure to prenatal smoking on severe mental illness (bipolar disorder and schizophrenia) suggesting the contribution of family-level confounders to previously observed associations (Quinn et al ., 2017 ). There are many limitations to this design that have been described elsewhere. These include the issue of selection as mothers are behaving differently in different pregnancies. For example, the sample consists of a group of mothers who are able to quit smoking in one pregnancy but not the other. Also, there is the problem that siblings will be born at different times and thus will be exposed to different family-level and population-level risks.

(ii) Assessing later adversities using a sibling pair design and its extension, the co-relative study. The discordant sibling pair design and its extension involving pairs of relatives from the same generation such as half-siblings and cousins have also been used to assess causal links between adolescent and adult exposures and psychiatric disorders. For example, the observed association between cannabis use and schizophrenia has been well-established. However, the causal relationship could be subject to question given that those who are at elevated familial or genetic liability or with prodromal symptoms could be more likely to use cannabis (confounding, selection bias, and reverse causation). In one large, Swedish study, the authors used an extended sibling pair design to investigate the causal relationship between cannabis and schizophrenia (Giordano et al ., 2015 ). The association was much attenuated once familial confounding was taken into account; the effect size also was diminished when potential prodromal effects were considered that was assessed by increasing the temporal delay between cannabis abuse and admission for schizophrenia (odds ratio 1.67). The findings suggested that there is a likely causal link between cannabis use and schizophrenia for some but that the effect size is not as strong as previously reported because of the contribution of familial confounding and reverse causation.

An alternative design that enables separation of prenatal exposures from genetic ones is based on individuals who have been conceived through assisted reproductive technologies. Some of these individuals are genetically related to the woman who undergoes the pregnancy and others are genetically unrelated (see Fig. 2 ). If a prenatal exposure has causal effects, then association with the offspring outcome should be observed regardless of whether mother–offspring dyads are genetically related or unrelated. That was the case for maternal smoking in pregnancy and lower birth weight (Thapar et al ., 2009 ) and also for associations between maternal reports of stress in pregnancy and lower birth weight and preterm birth (Rice et al ., 2010 ).

In-vitro fertilisation design.

However, for association between maternal smoking in pregnancy and a trait measure of ADHD in offspring (Thapar et al ., 2009 ) as well as conduct problems (Rice et al ., 2009 ), association was only observed in genetically related mother–offspring dyads not in the unrelated pairs, suggesting genetic confounding. The finding converges with those from the maternal v. paternal exposure and discordant sibling pair designs. Interestingly, the magnitude of association in the related pairs was similar to that observed in other observational studies and including measured confounders of the sort including in observational designs, such as parental psychopathology, social class did not remove the genetic confound.

That is, findings from this and other studies suggest that residual confounds remain a problem for observational studies and that including multiple confounders is not a substitute for an informative design.

The IVF design (Thapar et al ., 2007 ) has also been used to assess inter-generational transmission of psychopathology and to examine post-natal adversity. For example, using this approach, depression symptoms were found to be environmentally transmitted and environmental links were observed between hostile parenting and antisocial behavior in offspring (Harold et al ., 2011 ).

The IVF design does have a number of limitations however. These include the representativeness of the families who have undergone IVF treatment and the low prevalence of certain types of risk factors (e.g. maternal smoking in pregnancy).

Twin designs

Twin designs utilize the fact that monozygotic (MZ) twins share on average 100% of their genes (DNA sequence) and dizygotic (DZ) twins share on average 50% of their genome.

The twin design allows variation in any given measure to be partitioned into genetic and environmental variance. Where both exposure and psychiatric outcome are assessed, ideally longitudinally to avoid the problem of reverse causation, the association between exposure (e.g. life events) and outcome (e.g. depression) can be decomposed into genetic and environmental components. As the genetic covariance between exposure and outcome is explicitly modelled, essentially the genetic confound is removed. Here, the investigator is interested in whether there is an environmental link that remains between the exposure and outcome. This design has been invaluable in demonstrating a number of potentially causal environmental risk factors for psychopathology.

For example, family and twin studies of depression in childhood, adolescence, and adult life have observed a familial and genetic contribution to life events, mainly those that are person-dependent (e.g. losing a job) rather than ones that are independent (death of a relative), as well as to depression (McGuffin et al ., 1988 ; Plomin, 2018 ).

Twin studies that have investigated the link between life events and depression suggest that the association between independent life events and depression appears to be mainly or entirely environmental; that is consistent with a causal explanation (Kendler et al ., 1999 ). For dependent life events, there is a stronger genetic contribution to the link with depression. This seems to be partly explained by self-selection into risk exposure by those predisposed to depression (Kendler et al ., 1999 ) and becomes more prominent from adolescence onwards (Rice et al ., 2003 ).

Another example is the link between harsh parenting and antisocial behavior in children. One twin study found that the association with corporal punishment was primarily explained by genetic factors (Jaffee et al ., 2004 ). This could arise, for example, through parental response to the child's behavior which is genetically influenced. However, the findings for physical abuse were different. Here, the link with antisocial behavior was environmentally mediated, and consistent with a causal explanation.

Twin designs, their uses, strengths, and limitations have been described in detail elsewhere (State and Thapar, 2015 ). When genetic contributions are identified through bivariate twin analyses that we have described, it can index selection bias and potential threats to causal inference. However even with longitudinal twin designs, an environmentally mediated link does not prove a causal link between an exposure and outcome as there could be alternative pathways that explain the association including measurement artifacts.

Discordant MZ twin pairs

This design utilizes the fact that MZ twins are considered to share 100% of their genes and means that differences in their phenotype are attributed to non-genetic contributions that include non-shared environment as well as measurement error and stochastic effects. The approach involves assessing whether MZ twins who are differentially exposed to a stressor or adversity (e.g. discordant for victimization) show differences in a given outcome (e.g. depression).

For example, in the UK E-risk twin study of 7–10 years old, 110 MZ twin pairs who were discordant for bullying victimization were assessed (Arseneault et al ., 2008 ). The co-twins who were bullied showed higher internalizing (anxiety/depression) symptom scores than those who were not exposed to bullying. A more recent US longitudinal twin study also investigated 145 MZ twins who were discordant for bullying victimization in childhood (Silberg et al ., 2016 ). Although being bullied showed a genetic link with social anxiety; there were also environmental links with social anxiety, separation anxiety, and young adult suicidal ideation. The findings from both of these studies are consistent with a causal effect of bullying victimization on emotional/anxiety symptoms and are important given the interest in reports from longitudinal observational designs.

In another longitudinal MZ discordant twin study (Caspi et al ., 2004 ), Caspi et al . assessed maternal hostility and warmth. This was achieved by conducting independent ratings from a recorded 5 min speech sample from the mother when talking about the child (expressed emotion EE). Maternal expressed emotion was found to be environmentally associated with later teacher-reported behavioral problems.

Although the MZ discordant pair design is useful because it controls for genetic confounding there are some drawbacks. For example, we now know that MZ twins are not per se 100% genetically identical, for example, through non-inherited genetic differences. Also discordant MZ twin pairs could be considered as atypical and rare especially for very highly heritable disorders such as autism or ADHD or schizophrenia. The exposure could be behaving as a proxy for some other risk factor that impacted on one twin and not the other.

Children of Twins design and extensions

The Children of Twins (CoT) design allow investigation of cross-generational links between parent and offspring psychopathology or parentally provided exposures and offspring outcomes. It takes advantage of the fact that the offspring of MZ and DZ twins are socially cousins (DZ twins are also genetically cousins) but the MZ twin offspring are genetically half siblings.

This type of design, for example, has been used to assess the cross-generational transmission of depression. In an Australian study of twins, their spouse, and offspring, environmental factors were found to explain the link between parents and offspring depression even when accounting for depression in spouses (Singh et al ., 2011 ). Similar findings had been found in an earlier US study (Silberg et al ., 2010 ). Another CoT study from Sweden found that depression symptoms in parents showed concurrent environmental but not genetic links with offspring internalizing symptoms (McAdams et al ., 2015 ). The findings accord with those from the IVF study (Harold et al ., 2011 ). A more recent Swedish CoT design observed only environmental transmission between parents and offspring for anxiety and neuroticism; again with no genetic contribution (Eley et al ., 2015 ).

These findings might appear puzzling in that while it is important to observe environmental transmission of depression and anxiety, there are no genetic contributions observed for either and this is inconsistent with twin studies (Sullivan et al ., 2000 ). Twin studies observe modest heritability for depression. One difficulty for cross-generational investigations is the assumption that the same genetic influences contribute across development when that is unlikely (e.g. Power et al ., 2017 ; Riglin et al ., 2017 ). Another issue is that twin study heritability estimates capture passive gene–environment correlation effects that would be reduced in CoT studies and eliminated in the IVF design.

The CoT design has also been used to assess postnatal adversities. One such study (Lynch et al ., 2006 ) found that harsh physical punishment remained associated with childhood behavioral problems even when genetic factors had been allowed for. These findings are in keeping with the twin study findings and taken together are consistent with harsh parenting having a causal effect on childhood antisocial behavior.

Adoption studies

Adoption studies allow genetic and prenatal influences to be separated from post-adoption experiences. They provide a powerful method for assessing the contribution of rearing influences because these are known to be affected by with genetically influenced parental attributes. Ordinarily these biological parental characteristics would in turn be correlated with child characteristics including psychopathology thereby introducing a potential genetic confound. The advantage of adoption studies is that they remove this confound, the so-called passive gene–environment correlation because the genotypes of the parents who are rearing the children are independent of the child's genotypes.

There are several examples where adoption studies have been able to demonstrate the contribution of the rearing environment. For example, a study of adopted away children showed that negative parenting provided by the adoptive parent was associated with their adoptive child's antisocial behavior (Ge et al ., 1996 ). The adoptive parent's negative parenting was also associated with substance abuse/dependency or antisocial personality in the child's biological parents; that association appeared to be mediated via the child's behavior. Overall the findings suggested causal effects of negative parenting on children's antisocial behavior but also showed that the children's genetically influenced antisocial behavior in turn affected the parenting of the adoptive parents. The observation that negative parenting has a causal effect on offspring antisocial behavior converges with the findings from twin studies showing a convergence of findings from different designs.

A more recent example is provided by a Swedish large-scale adoption study cross-generational study (Kendler et al ., 2018 ). The authors were able to assess the contribution of genetic and rearing influences to parent–offspring resemblance for treated major depressive disorder. They found that both genetic and rearing influences contributed equally to parent–offspring resemblance in major depressive disorder. The adoptive families enabled the authors to further show that genetic and rearing influences acted additively rather than having an interactive effect. The authors highlighted that there had been four previous adoption studies of depression; although genetic contributions had previously been observed, only one had observed an environmental contribution to depression. However, now there have been two adoption studies that have showed an environmental contribution to inter-generational transmission of depression. Also the same findings have been observed in three children of twin designs and in the IVF design, although here some of these find environmental contributions only with no genetic transmission.

Overall the findings from different genetically informative studies of depression are converging on the suggestion that environmental/social factors contribute to the cross-generational transmission of depression. That of course has important clinical treatment and prevention implications.

Designs involving the introduction or removal of risks to a population: potentially removing selection or allocation bias

Given a serious challenge to causal inference is selection or allocation bias, a number of studies have taken advantage of situations where risks have been introduced to or removed from an entire population.

Universal introduction of risk

Here, the best known studies are the Dutch Hunger Winter (Susser et al ., 1996 ) and Chinese famine studies (St Clair et al ., 2005 ) that examined the consequences of intra-uterine exposure to famine. These studies focused on populations that were exposed to universal time-limited famines that affected some individuals during the intra-uterine period. Exposed individuals in both studies showed around a twofold elevated risk of schizophrenia as well as congenital anomalies of the central nervous system. As there was no evidence for selection for exposure to either of these famines, the findings suggest that extreme nutritional deficiency in early pregnancy likely has a causal risk effect for schizophrenia. However, the conditions in both of these studies was extreme and atypical so whether the findings have relevance for the etiology of schizophrenia as a whole is unknown.

Universal removal of risk

In this design, the strength is that it again removes selection or allocation bias whereby the person or some external agent influences the removal of risk. One good example is provided by the Great Smoky Mountains Study that is a longitudinal epidemiological study. During the course of this study of over 1000 children, a casino opened on a Native American reserve and provided a substantial increase to the family income for around a quarter of the original sample. The investigators were able to examine data before and after this happened. They showed that the relief of poverty led to decreased levels of oppositional defiant disorder and conduct disorder but not anxiety or depression (Costello et al ., 2003 ). The effects appeared to be mediated via altered parenting that included increased levels of supervision and parental time. Later follow-up showed that family income supplementation provided in childhood continued to be associated with lower rates of psychiatric problems including alcohol and cannabis abuse, lower rates of convictions for minor offenses, and higher levels of education. There were no links with later behavioral disorders or depression or other drug use (Costello et al ., 2010 ).

Interrupted time series

This design takes advantage of multiple waves of data that have been collected before and after the introduction or removal of the putative causal variable. This could be used to assess the impact of a policy or a naturally occurring event.

For example, after the introduction of UK legislation to reduce paracetamol package sizes, there was an observed drop in deaths from paracetamol overdoses (Hawton et al ., 2013 ).

Another example comes from a study of gang membership that is known to be associated with higher rates of delinquency (Thornberry et al ., 1993 ). However, it is not known whether that is due to selection effects with those having a propensity to be delinquent choosing to be in a gang or whether it is the causal social effects of being in a gang. Thornberry et al . ( 2002 ) found as might be expected important selection effects; boys who joined gangs were more delinquent than those who did not. However, they also showed that once boys left the gang, their rates of delinquency dropped off though not back to the level they were prior to joining the gang. This observation suggested that gang membership had additional social influences on delinquency. However, reverse causation and unmeasured confounders are possible contributors because we do not know what affected the boys’ decisions to leave the gang.

Changes in policy

If these are applied to a whole nation and data are available before and after the introduction of the policy, then this can provide a useful natural experiment situation. One study in Sweden (Nilsson, 2008 ) focused on the effects of prenatal alcohol exposure in two regions that were subjected to an experimental policy change in alcohol sales. The intention was to shift the population away from drinking spirits to consuming drinks with a lower alcohol content. However, it inadvertently resulted in very marked increases in the consumption of strong beer especially amongst teenagers. The experimental policy started in 1967 but was terminated abruptly in mid-1968 once it was realized that alcohol consumption had increased. Using registry data, the researchers were able to assess a cohort of children who had been in utero during the exposed period. As the policy was time and geographically limited, the exposed cohort could be compared with unexposed cohorts in adjacent geographic regions and in adjacent time-unexposed cohorts. At around 30 years of age, the exposed group showed greatly reduced educational achievements, lower earnings, and greater welfare dependency than those born to the unexposed cohorts. The effects were strongest in males, those exposed for the longest in intrauterine life and those born in younger mothers. The results suggest that prenatal exposure to alcohol likely had intrauterine risk effects on offspring. However, the problem with this sort of policy study is that the results are obtained from analyses at a group rather than individual level.

Radical change in environment: adoption following profound institutional deprivation

One good example of a natural experiment was provided by the English and Romanian Adoptees Study that involved a very radical change in early environment. This is a longitudinal study of individuals who were exposed to institutional care and extreme privation from early infancy. The possibilities of selection bias and reverse causation were essentially removed because the children were admitted very early and virtually no children left care until the government regime fell in 1989. These children subsequently were exposed to a radical change in rearing environment after they were adopted into relatively advantaged homes in the UK. The findings from this study showed that although there was some recovery, early institutional care of the type experienced by these children for more than 6 months resulted in difficulties that persisted to adulthood including autistic-type symptoms, ADHD-like problems, disinhibited social engagement, and emotional symptoms but not cognitive impairment (Sonuga-Barke et al ., 2017 ).

As is the case for some of the other natural experiments, such as the famine studies, although selection bias is removed, the question is whether the findings apply to less severe and more common forms of deprivation.

Using instrumental variables as a statistical method to deal with unmeasured confounding

An instrumental variable is a measured variable that is associated with the exposure of interest but that is not associated with the same selection effects and confounds. If the exposure has a genuinely causal risk effect on the outcome, then we would expect the instrumental variable also to be associated with the outcome. Early use and misuse of alcohol have been considered as potential causal risks or exposures for the later outcomes of alcohol dependence and misuse in adult life. Early puberty has been used as an instrumental variable for early use and misuse of alcohol because it is strongly associated with these exposures yet is not subject to the same selection biases or confounds.

Three studies have found that while early alcohol use and misuse in adolescence is associated with later alcohol problems, early puberty does not predict alcohol problems (Stattin and Magnusson, 1990 ; Caspi and Moffitt, 1991 ; Pulkkinen et al ., 2006 ).

These findings suggest that early alcohol use is likely an early manifestation of later alcohol problems rather than a cause of it.

Mendelian randomization: a special type of instrumental variable

Mendelian randomization (MR) utilizes the random assortment of parental genotypes to offspring during meiosis. Here a genetic variant that is robustly associated with the exposure is used as the instrumental variable and provided certain assumptions are met should provide a means of controlling for confounding and reverse causation (see Fig. 3 ). As more genetic variants are being identified through genome-wide association studies, there is a growing interest in using MR to test causal hypotheses and many methodological extensions of this approach (Davey Smith and Hemani, 2014 ). One approach called two-sample MR takes advantage of already published large genome-wide association studies. It uses genetic variants for exposures [e.g. C reactive protein (CRP)] as instrumental variables and another set of genetic variants from another independent GWAS for the outcomes variants (e.g. cardiovascular disease). MR has been used most successfully in relation to cardiovascular disease. For example, MR has been used to show that CRP does not have a causal risk effect on cardiovascular disease (C Reactive Protein Coronary Heart Disease Genetics Collaboration (CCGC) et al ., 2011 ). More recently, MR has started to be used in psychiatry; for example, a recent study observed body mass index effects on depression but not the reverse (Nagel et al ., 2018 ). MR is challenging because of its assumptions. For example, there is a need for genetic variants that have a strong and robust association with the exposure in question, although there are methods that allow for combining multiple genome-wide significant variants. Also if the genetic variant (instrument) has pleiotropic effects, and that is often the case, or influences a confounder or affects the outcome via another mechanism other than via the exposure, then that poses problems. There are methods for assessing pleiotropy and again, like all the methods we have discussed, MR findings on their own need to be interpreted with caution. However, when findings converge with other designs, they can be helpful in inferring causation. They are also a helpful alternative to RCTs.

Mendelian randomization. ( a ) The instrument is associated with the outcome only through the exposure. ( b ) Limitations – if the instrument is associated with a confounder or there is a horizontal pleiotropy.

Conclusions

It is crucial that genuinely causal influences on psychopathology are identified if interventions and policies are going to be effective. In recent years, findings relevant to psychiatry have emerged from different natural experiment designs and some are consistent across different designs; this strengthens causal inference. For example, hostile parenting affects antisocial behavior and RCTs uphold this causal inference. Genetically informative studies converge in favor of life events and victimization being environmentally linked with depression and environmental cross-generational transmission for depression. However, although smoking cessation programs for pregnant women are clearly a priority as cigarette smoke is detrimental to offspring physical health, the natural experiment designs suggest these will not be a useful means for preventing ADHD or antisocial behavior. So do natural experiments have an important future in the study of mental disorders? The answer is a firm yes.

Acknowledgements

AT receives grant funding from the Wellcome Trust and MRC.

Natural Experiments and Quasi-Natural Experiments

• Living reference work entry
• First Online: 15 December 2016
• Cite this living reference work entry

• J. DiNardo 2  

3220 Accesses

14 Citations

34 Altmetric

Natural experiments or quasi-natural experiments in economics are serendipitous situations in which persons are assigned randomly to a treatment (or multiple treatments) and a control group, and outcomes are analysed for the purposes of putting a hypothesis to a severe test; they are also serendipitous situations where assignment to treatment ‘approximates’ randomized design or a well-controlled experiment.

This chapter was originally published in The New Palgrave Dictionary of Economics , 2nd edition, 2008. Edited by Steven N. Durlauf and Lawrence E. Blume

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Institutional subscriptions

Similar content being viewed by others

Experimental Design

Bibliography

Ashenfelter, O., and A.B. Krueger. 1994. Estimates of the economic returns to schooling from a new sample of identical twins. American Economic Review 84: 1157–1173.

Google Scholar  

Ashtekar, A., R.S. Cohen, D. Howard, J. Renn, S. Sarkear, and A. Shimony. 2003. Revisiting the foundations of relativistic physics: Festschrift in honor of john Stachel , Boston studies in the philosophy of science. Vol. 234. Dordrecht: Kluwer Academic.

Bastable, C.F. 1987. Experimental methods in economics (i). In The new palgrave: A dictionary of economics , ed. J. Eatwell, M. Milgate, and P. Newman, Vol. 2. London: Macmillan.

Bound, J., and G. Solon. 1999. Double trouble: On the value of twins-based estimation of the return to schooling. Economics of Education Review 18: 169–182.

Article   Google Scholar  

DiNardo, J. 2007. Interesting questions in freakonomics. Journal of Economic Literature .

DiNardo, J. and Lee, D.S.. 2002. The impact of unionization on establishment closure: A regression discontinuity analysis of representation elections. Working Paper No. 8993. Cambridge, MA: NBER.

DiNardo, J., and D.S. Lee. 2004. Economic impacts of new unionization on private sector employers: 1984–2001. Quarterly Journal of Economics 119: 1383–1441.

DiNardo, J., and T. Lemieux. 2001. Alcohol, marijuana, and American youth: The unintended consequences of government regulation. Journal of Health Economics 20: 991–1010.

Drake, S. 1981. Cause, experiment, and science: A galilean dialogue, incorporating a new english translation of Galileo’s bodies that Stay atop water, or move in it . Chicago: University of Chicago Press.

Fisher, R.A. 1935. Design of experiments . Edinburgh/London: Oliver & Boyd.

Hacking, I. 1983. Representing and intervening: Introductory topics in the philosophy of natural science . Cambridge: Cambridge University Press.

Book   Google Scholar  

Hacking, I. 1988. Telepathy: Origins of randomization in experimental design. Isis 79: 427–451.

Hacking, I. 2000. The social construction of what? Cambridge, MA: Harvard University Press.

Hearst, N., T.B. Newman, and S.B. Hulley. 1986. Delayed effects of the military draft on mortality: A randomized natural experiment. New England Journal of Medicine 314: 620–624.

Heckman, J.J. 2005. The scientific model of causality. Sociological Methodology 35: 1–97.

Heckman, J.J., and J.A. Smith. 1995. Assessing the case for social experiments. Journal of Economic Perspectives 9(2): 85–110.

Lee, D.S. 2008. Randomized experiments from non-random selection in U.S. house elections. Journal of Econometrics .

Magee, B. 2001. Talking philosophy: dialogues with fifteen leading philosphers . Oxford: Oxford University Press.

Mayo, D.G. 1996. Error and the growth of experimental knowledge science and its conceptual foundations . Chicago: University of Chicago Press.

Meyer, B. 1995. Natural and quasi-experiments in economics. Journal of Business and Economic Statistics 13: 151–161.

Morgan, M.S. 1987. Statistics without probability and Haavelmo’s revolution in econometrics. In The probabilistic revolution: Ideas in the sciences , ed. L. Krüger, G. Gigerenzer, and M.S. Morgan, Vol. 2. Cambridge, MA: MIT Press.

Nelson, A. 1990. Are economic kinds natural? In In Scientific Theories of Minnesota Studies in the Philosophy of Science , ed. C. Wade Savage, Vol. 14. Minneapolis: University of Minnesota Press.

Peirce, C.S.. 1958. In Collected Papers, vols. 7–8, ed. A. Burks. Cambridge, MA: Harvard University Press.

Rosenzweig, M.R., and K.I. Wolpin. 2000. Natural ‘natural experiments’ in economics. Journal of Economic Literature 38: 827–874.

Searle, J. 1995. The construction of social reality . New York: Free Press.

Shadish, W.R., T.D. Cook, and D.T. Campbell. 2002. Experimental and Quasi–Experimental designs for generalized causal inference . Boston: Houghton Mifflin.

Tribby, J. 1994. Club Medici: Natural experiment and the imagineering of ‘Tuscany’. Configurations 2: 215–235.

Voltaire. 1759. The history of candide; or all for the best , ed. C. Cooke. London, 1796.

Waller, R. 1684. Essayes of natural experiments made in the academie del cimento, under the protection of the most serene Prince Leopold of Tuscany . Facsimile edn, ed. R. Hall, trans. R. Waller. New York/London, 1964.

Wikipedia. 2006. Experiment. http://en.wikipedia.org . Accessed 28 Sept 2006.

Download references

Author information

Authors and affiliations.

http://link.springer.com/referencework/10.1057/978-1-349-95121-5

You can also search for this author in PubMed   Google Scholar

Editor information

Editors and affiliations, copyright information.

© 2008 The Author(s)

About this entry

Cite this entry.

DiNardo, J. (2008). Natural Experiments and Quasi-Natural Experiments. In: The New Palgrave Dictionary of Economics. Palgrave Macmillan, London. https://doi.org/10.1057/978-1-349-95121-5_2006-1

Download citation

DOI : https://doi.org/10.1057/978-1-349-95121-5_2006-1

Received : 12 September 2016

Accepted : 12 September 2016

Published : 15 December 2016

Publisher Name : Palgrave Macmillan, London

Online ISBN : 978-1-349-95121-5

eBook Packages : Springer Reference Economics and Finance Reference Module Humanities and Social Sciences Reference Module Business, Economics and Social Sciences

• Publish with us

Policies and ethics

• Find a journal
• Track your research

Natural Experiments in Management Research

Difference-in-Difference Design

The difference-in-difference (DID) design is probably the most frequently used design of natural experiments. The DID design aims to estimate an average treatment effect by comparing the pre-treatment to post-treatment changes in an outcome variable (e.g., job satisfaction, job performance) between a treatment group (i.e., individuals, groups or organizations that received a treatment) and a control group (i.e., individuals, groups or organizations that did not receive a treatment). The “simplest” form of the DID design is the two groups in two periods design ( Wing et al. 2018 ). For instance, company A may would like to know whether more delicious food in the company’s cafeteria is related to higher job satisfaction. To test the relationship, the company first collects data on employee job satisfaction at time t-0. Then, the company hires a renowed cook for its cafeteria in factory A (treatment group) but makes no changes in the cafeteria in factory B (control group). At time t-1, the company again collects data on employee job satisfaction.  The results of the survey are as follows:

Factory A : Job satisfaction in t-0 = 4.2; job satisfaction in t-1 = 4.9 Factory B : Job satisfaction in t-0 = 4.5; job satisfaction in t-1 = 4.7

The figure below visualizes the changes in job satisfaction from t-0 to t-1 for both groups.

To calculate the average effect of the treatment (more delicious food) on the outcome (job satisfaction), we can use the rather simple formula:

Average treatment effect (ATE) = (yt-1,T – yt-0, T) – (yt-1, C – yt-0, C) where yt-1, T is the average job satisfaction of the treatment group in t-1 (4.9), yt-o, T  is the average job satisfaction of the treatment group in t-0 (4.2), yt-1, C  is the average job satisfaction of the control group in t-1 (4.7), and yt-0, C  is the average job satisfaction of the control group in t-0 (4.5). Therefore, the ATE is (4.9-4.2) – (4.7-4.5) = 0.5, which indicates that more delicious food at the company’s cafeteria leads to an on average 0.5 points higher job satisfaction.

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• 13 October 2021

Nobel-winning ‘natural experiments’ approach made economics more robust

• Philip Ball 0

Philip Ball was a physics editor of Nature during the Benveniste saga. His books include H 2 O: A Biography of Water and The Devil’s Doctor: Paracelsus and the World of Renaissance Magic and Science .

You can also search for this author in PubMed   Google Scholar

The ‘natural experiments’ approach to economics that won three researchers the 2021 Sveriges Riksbank Prize in Economic Sciences has helped to make the field more robust, say economists.

Access options

Access Nature and 54 other Nature Portfolio journals

Get Nature+, our best-value online-access subscription

24,99 € / 30 days

cancel any time

Subscribe to this journal

Receive 51 print issues and online access

185,98 € per year

only 3,65 € per issue

Rent or buy this article

Prices vary by article type

Prices may be subject to local taxes which are calculated during checkout

doi: https://doi.org/10.1038/d41586-021-02799-7

Angrist, J. D. & Krueger, A. B. Quart. J. Econ. 106 , 976–1014 (1991).

Article   PubMed   Google Scholar  

Card, D. & Krueger, A. B. Am. Econ. Rev. 84 , 772–793 (1994).

Google Scholar  

Imbens, G. W. & Angrist, J. D. Econometrica 62 , 467–475 (1994).

Download references

Reprints and permissions

Division Director - Experimental Hematology and Cancer Biology

Cincinnati Children’s Hospital seeks the next Director for the Division of Experimental Hematology and Cancer Biology.

Cincinnati, Ohio

Cincinnati Children's Hospital & Medical Center

Faculty and Research Positions, Postdoctoral Recruitment

Jointly sponsored by the Hangzhou Municipal People's Government and the University of Chinese Academy of Sciences.

Hangzhou, Zhejiang, China

Hangzhou Institute of Advanced Study, UCAS

Associate or Senior Editor, Nature Energy

Job Title: Associate or Senior Editor, Nature Energy Location: New York, Jersey City, Philadelphia or London — Hybrid Working Application Deadline:...

New York City, New York (US)

Springer Nature Ltd

Faculty Positions & Postdocs at Institute of Physics (IOP), Chinese Academy of Sciences

IOP is the leading research institute in China in condensed matter physics and related fields. Through the steadfast efforts of generations of scie...

Beijing, China

Institute of Physics (IOP), Chinese Academy of Sciences (CAS)

FACULTY POSITION IN THE MOLECULAR BIOLOGY OF REPRODUCTION

Dallas, Texas (US)

The University of Texas Southwestern Medical Center (UT Southwestern Medical Center)

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Quickonomics

Natural Experiment

### Definition of Natural Experiment

A natural experiment refers to an empirical or observational study in which the conditions for the research are determined by nature or by other factors outside the control of the investigators. Unlike controlled experiments where researchers manipulate variables to determine cause and effect, natural experiments take advantage of real-world situations where a natural division or variation occurs, allowing for the observation of its impact on specific outcomes. These types of experiments are particularly valuable in fields like economics, sociology, and epidemiology, where ethical or practical considerations make controlled experiments difficult to carry out.

### Example

Consider a situation where a certain city introduces a significant public transportation subsidy, effectively reducing the cost of using public transit, while a neighboring city does not. Over time, researchers might observe changes in traffic congestion, air quality, and public transit usage between the two cities. This scenario can be viewed as a natural experiment because the subsidy’s implementation creates a naturally occurring division, facilitating the study of its effects on urban transportation patterns without needing a controlled, experimental design.

### Why Natural Experiments Matter

Natural experiments are crucial for several reasons. They provide a unique opportunity to study the effects of variables that cannot be ethically or practically manipulated by researchers. By observing naturally occurring variations, scientists and policymakers can infer causal relationships in complex, real-world settings. This ability to glean insights from uncontrolled environments helps bridge the gap between theory and practice, offering evidence that can inform public policy, economic strategies, and social interventions. Furthermore, because they rely on real-world occurrences, the findings from natural experiments often have a high degree of external validity, meaning they can be generalized to broader contexts.

### Frequently Asked Questions (FAQ)

#### How do natural experiments differ from randomized controlled trials (RCTs)?

Natural experiments differ from RCTs in that the researcher has no control over the assignment of the treatment or intervention. In RCTs, subjects are randomly assigned to either the treatment or control group to ensure that any differences observed are due to the treatment itself and not to pre-existing conditions. In contrast, natural experiments rely on external circumstances to create an “experimental” and a “control” group, which may lead to challenges in isolating the treatment effect from other confounding factors.

#### What are the limitations of natural experiments?

One of the key limitations of natural experiments is the potential for confounding variables that researchers cannot control. These variables may influence the outcome of the experiment, making it difficult to establish causality with the same certainty as in controlled experiments. Additionally, since natural experiments are observational, they can be susceptible to selection bias, where the characteristics of the groups being compared differ in significant ways other than the treatment of interest.

#### Can natural experiments provide conclusive evidence on causality?

While natural experiments can provide valuable insights and suggest causal relationships, they may not offer conclusive evidence of causality. The lack of control over assignment and potential confounding factors means that results should be interpreted with caution. However, when designed and analyzed correctly, and ideally combined with other sources of evidence, natural experiments can contribute significantly to our understanding of causal mechanisms in real-world settings.

#### How are natural experiments identified and analyzed?

Identifying natural experiments often involves recognizing opportunities where natural or policy-related variations mimic the conditions of a controlled experiment. This can include changes in laws, technological advancements, or natural disasters. Analysis typically requires sophisticated statistical methods to account for potential confounding variables and to isolate the effect of the treatment or intervention. Techniques such as difference-in-differences, regression discontinuity designs, and instrumental variable analysis are commonly used to analyze data from natural experiments.

To provide the best experiences, we and our partners use technologies like cookies to store and/or access device information. Consenting to these technologies will allow us and our partners to process personal data such as browsing behavior or unique IDs on this site and show (non-) personalized ads. Not consenting or withdrawing consent, may adversely affect certain features and functions.

Click below to consent to the above or make granular choices. Your choices will be applied to this site only. You can change your settings at any time, including withdrawing your consent, by using the toggles on the Cookie Policy, or by clicking on the manage consent button at the bottom of the screen.

• Science Notes Posts
• Contact Science Notes
• Todd Helmenstine Biography
• Anne Helmenstine Biography
• Free Printable Periodic Tables (PDF and PNG)
• Periodic Table Wallpapers
• Interactive Periodic Table
• Periodic Table Posters
• Science Experiments for Kids
• How to Grow Crystals
• Chemistry Projects
• Fire and Flames Projects
• Holiday Science
• Chemistry Problems With Answers
• Physics Problems
• Unit Conversion Example Problems
• Chemistry Worksheets
• Biology Worksheets
• Periodic Table Worksheets
• Physical Science Worksheets
• Science Lab Worksheets
• My Amazon Books

Independent and Dependent Variables Examples

The independent and dependent variables are key to any scientific experiment, but how do you tell them apart? Here are the definitions of independent and dependent variables, examples of each type, and tips for telling them apart and graphing them.

Independent Variable

The independent variable is the factor the researcher changes or controls in an experiment. It is called independent because it does not depend on any other variable. The independent variable may be called the “controlled variable” because it is the one that is changed or controlled. This is different from the “ control variable ,” which is variable that is held constant so it won’t influence the outcome of the experiment.

Dependent Variable

The dependent variable is the factor that changes in response to the independent variable. It is the variable that you measure in an experiment. The dependent variable may be called the “responding variable.”

Examples of Independent and Dependent Variables

Here are several examples of independent and dependent variables in experiments:

• In a study to determine whether how long a student sleeps affects test scores, the independent variable is the length of time spent sleeping while the dependent variable is the test score.
• You want to know which brand of fertilizer is best for your plants. The brand of fertilizer is the independent variable. The health of the plants (height, amount and size of flowers and fruit, color) is the dependent variable.
• You want to compare brands of paper towels, to see which holds the most liquid. The independent variable is the brand of paper towel. The dependent variable is the volume of liquid absorbed by the paper towel.
• You suspect the amount of television a person watches is related to their age. Age is the independent variable. How many minutes or hours of television a person watches is the dependent variable.
• You think rising sea temperatures might affect the amount of algae in the water. The water temperature is the independent variable. The mass of algae is the dependent variable.
• In an experiment to determine how far people can see into the infrared part of the spectrum, the wavelength of light is the independent variable and whether the light is observed is the dependent variable.
• If you want to know whether caffeine affects your appetite, the presence/absence or amount of caffeine is the independent variable. Appetite is the dependent variable.
• You want to know which brand of microwave popcorn pops the best. The brand of popcorn is the independent variable. The number of popped kernels is the dependent variable. Of course, you could also measure the number of unpopped kernels instead.
• You want to determine whether a chemical is essential for rat nutrition, so you design an experiment. The presence/absence of the chemical is the independent variable. The health of the rat (whether it lives and reproduces) is the dependent variable. A follow-up experiment might determine how much of the chemical is needed. Here, the amount of chemical is the independent variable and the rat health is the dependent variable.

How to Tell the Independent and Dependent Variable Apart

If you’re having trouble identifying the independent and dependent variable, here are a few ways to tell them apart. First, remember the dependent variable depends on the independent variable. It helps to write out the variables as an if-then or cause-and-effect sentence that shows the independent variable causes an effect on the dependent variable. If you mix up the variables, the sentence won’t make sense. Example : The amount of eat (independent variable) affects how much you weigh (dependent variable).

This makes sense, but if you write the sentence the other way, you can tell it’s incorrect: Example : How much you weigh affects how much you eat. (Well, it could make sense, but you can see it’s an entirely different experiment.) If-then statements also work: Example : If you change the color of light (independent variable), then it affects plant growth (dependent variable). Switching the variables makes no sense: Example : If plant growth rate changes, then it affects the color of light. Sometimes you don’t control either variable, like when you gather data to see if there is a relationship between two factors. This can make identifying the variables a bit trickier, but establishing a logical cause and effect relationship helps: Example : If you increase age (independent variable), then average salary increases (dependent variable). If you switch them, the statement doesn’t make sense: Example : If you increase salary, then age increases.

How to Graph Independent and Dependent Variables

Plot or graph independent and dependent variables using the standard method. The independent variable is the x-axis, while the dependent variable is the y-axis. Remember the acronym DRY MIX to keep the variables straight: D = Dependent variable R = Responding variable/ Y = Graph on the y-axis or vertical axis M = Manipulated variable I = Independent variable X = Graph on the x-axis or horizontal axis

• Babbie, Earl R. (2009). The Practice of Social Research (12th ed.) Wadsworth Publishing. ISBN 0-495-59841-0.
• di Francia, G. Toraldo (1981). The Investigation of the Physical World . Cambridge University Press. ISBN 978-0-521-29925-1.
• Gauch, Hugh G. Jr. (2003). Scientific Method in Practice . Cambridge University Press. ISBN 978-0-521-01708-4.
• Popper, Karl R. (2003). Conjectures and Refutations: The Growth of Scientific Knowledge . Routledge. ISBN 0-415-28594-1.

Related Posts

• Academic Medicine
• Acid Base, Electrolytes, Fluids
• Allergy and Clinical Immunology
• American Indian or Alaska Natives
• Anesthesiology
• Anticoagulation
• Art and Images in Psychiatry
• Artificial Intelligence
• Assisted Reproduction
• Bleeding and Transfusion
• Caring for the Critically Ill Patient
• Challenges in Clinical Electrocardiography
• Climate and Health
• Climate Change
• Clinical Challenge
• Clinical Decision Support
• Clinical Implications of Basic Neuroscience
• Clinical Pharmacy and Pharmacology
• Complementary and Alternative Medicine
• Consensus Statements
• Coronavirus (COVID-19)
• Critical Care Medicine
• Cultural Competency
• Dental Medicine
• Dermatology
• Diabetes and Endocrinology
• Diagnostic Test Interpretation
• Drug Development
• Electronic Health Records
• Emergency Medicine
• End of Life, Hospice, Palliative Care
• Environmental Health
• Equity, Diversity, and Inclusion
• Facial Plastic Surgery
• Gastroenterology and Hepatology
• Genetics and Genomics
• Genomics and Precision Health
• Global Health
• Guide to Statistics and Methods
• Hair Disorders
• Health Care Delivery Models
• Health Care Economics, Insurance, Payment
• Health Care Quality
• Health Care Reform
• Health Care Safety
• Health Care Workforce
• Health Disparities
• Health Inequities
• Health Policy
• Health Systems Science
• History of Medicine
• Hypertension
• Images in Neurology
• Implementation Science
• Infectious Diseases
• Innovations in Health Care Delivery
• JAMA Infographic
• Law and Medicine
• Leading Change
• Less is More
• LGBTQIA Medicine
• Lifestyle Behaviors
• Medical Coding
• Medical Devices and Equipment
• Medical Education
• Medical Education and Training
• Medical Journals and Publishing
• Mobile Health and Telemedicine
• Narrative Medicine
• Neuroscience and Psychiatry
• Notable Notes
• Nutrition, Obesity, Exercise
• Obstetrics and Gynecology
• Occupational Health
• Ophthalmology
• Orthopedics
• Otolaryngology
• Pain Medicine
• Palliative Care
• Pathology and Laboratory Medicine
• Patient Care
• Patient Information
• Performance Improvement
• Performance Measures
• Perioperative Care and Consultation
• Pharmacoeconomics
• Pharmacoepidemiology
• Pharmacogenetics
• Pharmacy and Clinical Pharmacology
• Physical Medicine and Rehabilitation
• Physical Therapy
• Physician Leadership
• Population Health
• Primary Care
• Professional Well-being
• Professionalism
• Psychiatry and Behavioral Health
• Public Health
• Pulmonary Medicine
• Regulatory Agencies
• Reproductive Health
• Research, Methods, Statistics
• Resuscitation
• Rheumatology
• Risk Management
• Scientific Discovery and the Future of Medicine
• Shared Decision Making and Communication
• Sleep Medicine
• Sports Medicine
• Stem Cell Transplantation
• Substance Use and Addiction Medicine
• Surgical Innovation
• Surgical Pearls
• Teachable Moment
• Technology and Finance
• The Art of JAMA
• The Arts and Medicine
• The Rational Clinical Examination
• Tobacco and e-Cigarettes
• Translational Medicine
• Trauma and Injury
• Treatment Adherence
• Ultrasonography
• Users' Guide to the Medical Literature
• Vaccination
• Venous Thromboembolism
• Veterans Health
• Women's Health
• Workflow and Process
• Wound Care, Infection, Healing
• Download PDF
• Share X Facebook Email LinkedIn
• Permissions

“Natural Experiments” in Health Care Research

• 1 Division of Health Policy and Economics, Department of Population Health Sciences, Weill Cornell Medical College, New York, New York
• 2 Division of General Internal Medicine, Department of Medicine, Weill Cornell Medical College, New York, New York
• 3 Department of Health Care Policy, Harvard Medical School, Boston, Massachusetts
• 4 Department of Medicine, Massachusetts General Hospital, Boston
• 5 National Bureau of Economic Research, Cambridge, Massachusetts
• Editorial Exploring Potential Causal Relationships Through Natural Experiments Alan M. Zaslavsky, PhD JAMA Health Forum

In “natural experiments,” the treatment or intervention is determined by variation not under the control of the researcher. These designs, used in economics and epidemiology to support inferences about causal relationships between interventions and outcomes, are useful tools to help improve the rigor of observational studies in health policy and medicine. Perhaps the first natural experiment in medicine was that of the English physician John Snow in the mid-nineteenth century. In 1854, a cholera outbreak struck Broad Street in London, killing hundreds. Studying case clusters, Snow discovered that neighborhoods supplied with water downstream of where sewage was discharged into the Thames River experienced high levels of disease, while neighborhoods receiving upstream water had low disease levels. 1 Snow described the populations as similar in age, occupation, income, and social rank, divided into groups without choice, illustrating an essential component of natural experiments: similar but distinct populations that are exposed to a condition outside the researchers’ control, allowing for reasonable conclusions about the potential causal link between exposure and outcome.

Randomized clinical trials (RCTs) have traditionally been viewed as the primary method for establishing causality in health care, but they have important limitations: they are expensive; it is not always possible to randomize patients; and their findings may not be generalizable to different patient populations or nonexperimental settings. When RCTs are not possible, medical and health policy researchers have turned to observational studies. In observational studies, however, individuals are not assigned to the intervention independently of potential confounding factors that could also influence outcomes, making it difficult to separate the treatment effect from other factors that may be associated with receiving the treatment.

By contrast, natural experiments rely on variation in treatment exposure that may be unrelated to other factors associated with the outcomes. Suppose researchers are interested in examining the likelihood of long-term use and adverse outcomes for patients after an initial opioid prescription. An observational analysis might be confounded if the factors that influence a clinician’s decision to prescribe opioids (eg, cancer-related pain) also affect long-term outcomes (eg, opioid dependence). An RCT might resolve this issue but would be ethically and practically challenging. Instead, researchers could examine how long-term opioid use varies among opioid-naive individuals who, by chance, are exposed to physicians with a high propensity vs low propensity to prescribe opioids (eg, when assigned to the next available physician in an emergency department). 2 In this scenario, the long-term outcomes following an initial opioid prescription could be identified by variation in the drug’s use associated with prescriber variation that is plausibly unrelated to variation in unobservable patient factors associated with both initial opioid use and long-term outcomes.

Natural experiments use quasi-randomization, a method of allocation to study groups that is not truly random and is not assigned by a researcher, such as a specific date, age, or event. These study designs have an important feature: the similarity of the groups can be measured. Treatment and control groups should be similar in sociodemographic characteristics, comorbidities, prior health care utilization, and any other factors that might be associated with outcomes, but often this is not the case and adjustments are needed based on these observed variables. Natural experiments attempt to control for unobserved variables. When well implemented, natural experiments may be more informative than traditional observational studies that do not control for unobservable confounders, but are less informative than RCTs in establishing true cause and effect. With natural experiments, the more closely the study design resembles an RCT, the more confidence we may have in the validity of the findings.

Five types of natural experiments are particularly relevant for observational studies in health policy and medicine: regression discontinuity designs (RDD), instrumental variable designs, difference-in-differences (DID) analyses, event-study analyses, and interrupted time-series 2 - 4 ( Table ). This is an overview of these types of studies with health policy examples and is not intended to provide a detailed assessment of these designs.

Regression discontinuity designs identify effect sizes associated with an intervention by studying individuals with treatment assignment that differs by position on either side of a specific, arbitrary cutoff (eg, a treatment threshold, policy implementation date, an age threshold, or a geographic discontinuity). 5 In this design, the probability of being exposed to the intervention changes discontinuously at this cutoff. Studies using RDD rely on the assumption that individuals on either side of the cutoff are similar, so their treatment assignment is nearly independent of their characteristics, both observed and unobserved. For example, a 2018 study evaluated the phased introduction of Medicare’s Value-Based Payment Modifier program. 3 Researchers used the program’s practice size thresholds (eg, 100 or more clinicians) to evaluate whether the program was associated with practice performance—with the assumption that practices just above and below the cutoff did not differ in important ways—and found that the program was not associated with improved practice performance and may have exacerbated health disparities.

Instrumental variable analyses using quasi-random variation in assignment to treatment or intervention have also been used to study clinical and health policy interventions. 6 For example, health policy researchers have been interested in whether higher spending hospitals achieve better outcomes, a relationship that is confounded by the fact that higher spending hospitals may treat patients that are disproportionately sicker, which could spuriously suggest that higher hospital spending leads to worse outcomes. To address this issue, a study examined the association between hospital spending and mortality by using quasi-random variation in ambulance dispatching patterns as an instrumental variable. 7 Ambulances may have preferences for which hospitals patients are taken to for reasons that are unrelated to patient clinical severity; this, in turn, may lead otherwise similar patients to be transported to (and treated at) higher vs lower spending hospitals.

Other natural experiments use different types of analyses to assess potential causal relationships. These include DID, event study, and interrupted time series analyses. In DID analysis, researchers compare outcomes in 2 groups that were similar before an intervention (natural or otherwise) that affected only 1 of the groups. 8 The DID analysis postulates that if the treatment had no effect, the differences between the groups would be unchanged after the treatment. One such study found lower long-term mortality rates after Hurricane Katrina among people who had been living in New Orleans compared with those who had been living in other similar cities, which they concluded represented the effect of migration because New Orleans residents migrated to areas with better socioeconomic conditions and lower baseline mortality after the hurricane. 9 A randomized experiment on the effects of resettling a population on that scale would have been infeasible.

In event-study analyses, researchers rely on exogenous and variable timing of interventions in exposed groups to study changes within groups over time (eg, estimating the effect size for the association between care continuity and outcomes by studying patients whose primary care physicians retired at different points in time). Although event-study analyses do not require control groups, control groups without any exposure are frequently incorporated into this approach. 4 Interrupted time series analyses are similar, but typically focus on changes in outcomes before and after a single event that affects a population of interest (eg, a citywide soda excise tax). 10

Each of these types of study designs and analyses have important limitations that should be considered, including not controlling for unobserved or unmeasured differences between the groups, risk of selection bias due to allocation that cannot be concealed from the researchers, non-parallel trends that could affect comparisons between the groups, and spillover influences from 1 group to the other. For example, in RDD studies, assumptions must be tested to ensure that observed variables are continuous at the point where the treatment and outcome discontinuities occur, such that there are no abrupt changes in the relationship between the observed variables and the treatment or outcome except at the discontinuity cutoff. Similarly, studies that use instrumental variable analysis must ensure that an appropriate instrumental variable is selected and should acknowledge the possible threats to validity from unmeasured confounding factors.

Natural experiments offer an important approach for examining potential causal links between interventions and outcomes. Studies that appropriately use these methods could help provide data to inform questions affecting the health of patients that otherwise may remain unanswered.

Published: June 11, 2021. doi:10.1001/jamahealthforum.2021.0290

Open Access: This is an open access article distributed under the terms of the CC-BY License . © 2021 Khullar D et al. JAMA Health Forum .

Corresponding Author: Anupam B. Jena, MD, PhD, Department of Health Care Policy, Harvard Medical School, 180 Longwood Ave, Boston, MA 02115 ( [email protected] ).

Conflict of Interest Disclosures : Dr Jena reports consulting fees from Pfizer, Bioverativ, Bristol Myers Squibb, Merck Sharp & Dohme, Janssen, Edwards Life Sciences, Novartis, Amgen, Eli Lilly, Vertex Pharmaceuticals, AstraZeneca, Celgene, Tesaro, Sanofi, Aventis, Precision Health Economics, and Analysis Group, all outside of the submitted work.

See More About

Khullar D , Jena AB. “Natural Experiments” in Health Care Research. JAMA Health Forum. 2021;2(6):e210290. doi:10.1001/jamahealthforum.2021.0290

Manage citations:

© 2024

Artificial Intelligence Resource Center

Best of JAMA Network 2022

Browse and subscribe to JAMA Network podcasts!

Others Also Liked