what is background of the study in quantitative research

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• J Korean Med Sci
• v.37(16); 2022 Apr 25

A Practical Guide to Writing Quantitative and Qualitative Research Questions and Hypotheses in Scholarly Articles

Edward barroga.

1 Department of General Education, Graduate School of Nursing Science, St. Luke’s International University, Tokyo, Japan.

Glafera Janet Matanguihan

2 Department of Biological Sciences, Messiah University, Mechanicsburg, PA, USA.

The development of research questions and the subsequent hypotheses are prerequisites to defining the main research purpose and specific objectives of a study. Consequently, these objectives determine the study design and research outcome. The development of research questions is a process based on knowledge of current trends, cutting-edge studies, and technological advances in the research field. Excellent research questions are focused and require a comprehensive literature search and in-depth understanding of the problem being investigated. Initially, research questions may be written as descriptive questions which could be developed into inferential questions. These questions must be specific and concise to provide a clear foundation for developing hypotheses. Hypotheses are more formal predictions about the research outcomes. These specify the possible results that may or may not be expected regarding the relationship between groups. Thus, research questions and hypotheses clarify the main purpose and specific objectives of the study, which in turn dictate the design of the study, its direction, and outcome. Studies developed from good research questions and hypotheses will have trustworthy outcomes with wide-ranging social and health implications.

INTRODUCTION

Scientific research is usually initiated by posing evidenced-based research questions which are then explicitly restated as hypotheses. 1 , 2 The hypotheses provide directions to guide the study, solutions, explanations, and expected results. 3 , 4 Both research questions and hypotheses are essentially formulated based on conventional theories and real-world processes, which allow the inception of novel studies and the ethical testing of ideas. 5 , 6

It is crucial to have knowledge of both quantitative and qualitative research 2 as both types of research involve writing research questions and hypotheses. 7 However, these crucial elements of research are sometimes overlooked; if not overlooked, then framed without the forethought and meticulous attention it needs. Planning and careful consideration are needed when developing quantitative or qualitative research, particularly when conceptualizing research questions and hypotheses. 4

There is a continuing need to support researchers in the creation of innovative research questions and hypotheses, as well as for journal articles that carefully review these elements. 1 When research questions and hypotheses are not carefully thought of, unethical studies and poor outcomes usually ensue. Carefully formulated research questions and hypotheses define well-founded objectives, which in turn determine the appropriate design, course, and outcome of the study. This article then aims to discuss in detail the various aspects of crafting research questions and hypotheses, with the goal of guiding researchers as they develop their own. Examples from the authors and peer-reviewed scientific articles in the healthcare field are provided to illustrate key points.

DEFINITIONS AND RELATIONSHIP OF RESEARCH QUESTIONS AND HYPOTHESES

A research question is what a study aims to answer after data analysis and interpretation. The answer is written in length in the discussion section of the paper. Thus, the research question gives a preview of the different parts and variables of the study meant to address the problem posed in the research question. 1 An excellent research question clarifies the research writing while facilitating understanding of the research topic, objective, scope, and limitations of the study. 5

On the other hand, a research hypothesis is an educated statement of an expected outcome. This statement is based on background research and current knowledge. 8 , 9 The research hypothesis makes a specific prediction about a new phenomenon 10 or a formal statement on the expected relationship between an independent variable and a dependent variable. 3 , 11 It provides a tentative answer to the research question to be tested or explored. 4

Hypotheses employ reasoning to predict a theory-based outcome. 10 These can also be developed from theories by focusing on components of theories that have not yet been observed. 10 The validity of hypotheses is often based on the testability of the prediction made in a reproducible experiment. 8

Conversely, hypotheses can also be rephrased as research questions. Several hypotheses based on existing theories and knowledge may be needed to answer a research question. Developing ethical research questions and hypotheses creates a research design that has logical relationships among variables. These relationships serve as a solid foundation for the conduct of the study. 4 , 11 Haphazardly constructed research questions can result in poorly formulated hypotheses and improper study designs, leading to unreliable results. Thus, the formulations of relevant research questions and verifiable hypotheses are crucial when beginning research. 12

CHARACTERISTICS OF GOOD RESEARCH QUESTIONS AND HYPOTHESES

Excellent research questions are specific and focused. These integrate collective data and observations to confirm or refute the subsequent hypotheses. Well-constructed hypotheses are based on previous reports and verify the research context. These are realistic, in-depth, sufficiently complex, and reproducible. More importantly, these hypotheses can be addressed and tested. 13

There are several characteristics of well-developed hypotheses. Good hypotheses are 1) empirically testable 7 , 10 , 11 , 13 ; 2) backed by preliminary evidence 9 ; 3) testable by ethical research 7 , 9 ; 4) based on original ideas 9 ; 5) have evidenced-based logical reasoning 10 ; and 6) can be predicted. 11 Good hypotheses can infer ethical and positive implications, indicating the presence of a relationship or effect relevant to the research theme. 7 , 11 These are initially developed from a general theory and branch into specific hypotheses by deductive reasoning. In the absence of a theory to base the hypotheses, inductive reasoning based on specific observations or findings form more general hypotheses. 10

TYPES OF RESEARCH QUESTIONS AND HYPOTHESES

Research questions and hypotheses are developed according to the type of research, which can be broadly classified into quantitative and qualitative research. We provide a summary of the types of research questions and hypotheses under quantitative and qualitative research categories in Table 1 .

Research questions in quantitative research

In quantitative research, research questions inquire about the relationships among variables being investigated and are usually framed at the start of the study. These are precise and typically linked to the subject population, dependent and independent variables, and research design. 1 Research questions may also attempt to describe the behavior of a population in relation to one or more variables, or describe the characteristics of variables to be measured ( descriptive research questions ). 1 , 5 , 14 These questions may also aim to discover differences between groups within the context of an outcome variable ( comparative research questions ), 1 , 5 , 14 or elucidate trends and interactions among variables ( relationship research questions ). 1 , 5 We provide examples of descriptive, comparative, and relationship research questions in quantitative research in Table 2 .

Hypotheses in quantitative research

In quantitative research, hypotheses predict the expected relationships among variables. 15 Relationships among variables that can be predicted include 1) between a single dependent variable and a single independent variable ( simple hypothesis ) or 2) between two or more independent and dependent variables ( complex hypothesis ). 4 , 11 Hypotheses may also specify the expected direction to be followed and imply an intellectual commitment to a particular outcome ( directional hypothesis ) 4 . On the other hand, hypotheses may not predict the exact direction and are used in the absence of a theory, or when findings contradict previous studies ( non-directional hypothesis ). 4 In addition, hypotheses can 1) define interdependency between variables ( associative hypothesis ), 4 2) propose an effect on the dependent variable from manipulation of the independent variable ( causal hypothesis ), 4 3) state a negative relationship between two variables ( null hypothesis ), 4 , 11 , 15 4) replace the working hypothesis if rejected ( alternative hypothesis ), 15 explain the relationship of phenomena to possibly generate a theory ( working hypothesis ), 11 5) involve quantifiable variables that can be tested statistically ( statistical hypothesis ), 11 6) or express a relationship whose interlinks can be verified logically ( logical hypothesis ). 11 We provide examples of simple, complex, directional, non-directional, associative, causal, null, alternative, working, statistical, and logical hypotheses in quantitative research, as well as the definition of quantitative hypothesis-testing research in Table 3 .

Research questions in qualitative research

Unlike research questions in quantitative research, research questions in qualitative research are usually continuously reviewed and reformulated. The central question and associated subquestions are stated more than the hypotheses. 15 The central question broadly explores a complex set of factors surrounding the central phenomenon, aiming to present the varied perspectives of participants. 15

There are varied goals for which qualitative research questions are developed. These questions can function in several ways, such as to 1) identify and describe existing conditions ( contextual research question s); 2) describe a phenomenon ( descriptive research questions ); 3) assess the effectiveness of existing methods, protocols, theories, or procedures ( evaluation research questions ); 4) examine a phenomenon or analyze the reasons or relationships between subjects or phenomena ( explanatory research questions ); or 5) focus on unknown aspects of a particular topic ( exploratory research questions ). 5 In addition, some qualitative research questions provide new ideas for the development of theories and actions ( generative research questions ) or advance specific ideologies of a position ( ideological research questions ). 1 Other qualitative research questions may build on a body of existing literature and become working guidelines ( ethnographic research questions ). Research questions may also be broadly stated without specific reference to the existing literature or a typology of questions ( phenomenological research questions ), may be directed towards generating a theory of some process ( grounded theory questions ), or may address a description of the case and the emerging themes ( qualitative case study questions ). 15 We provide examples of contextual, descriptive, evaluation, explanatory, exploratory, generative, ideological, ethnographic, phenomenological, grounded theory, and qualitative case study research questions in qualitative research in Table 4 , and the definition of qualitative hypothesis-generating research in Table 5 .

Qualitative studies usually pose at least one central research question and several subquestions starting with How or What . These research questions use exploratory verbs such as explore or describe . These also focus on one central phenomenon of interest, and may mention the participants and research site. 15

Hypotheses in qualitative research

Hypotheses in qualitative research are stated in the form of a clear statement concerning the problem to be investigated. Unlike in quantitative research where hypotheses are usually developed to be tested, qualitative research can lead to both hypothesis-testing and hypothesis-generating outcomes. 2 When studies require both quantitative and qualitative research questions, this suggests an integrative process between both research methods wherein a single mixed-methods research question can be developed. 1

FRAMEWORKS FOR DEVELOPING RESEARCH QUESTIONS AND HYPOTHESES

Research questions followed by hypotheses should be developed before the start of the study. 1 , 12 , 14 It is crucial to develop feasible research questions on a topic that is interesting to both the researcher and the scientific community. This can be achieved by a meticulous review of previous and current studies to establish a novel topic. Specific areas are subsequently focused on to generate ethical research questions. The relevance of the research questions is evaluated in terms of clarity of the resulting data, specificity of the methodology, objectivity of the outcome, depth of the research, and impact of the study. 1 , 5 These aspects constitute the FINER criteria (i.e., Feasible, Interesting, Novel, Ethical, and Relevant). 1 Clarity and effectiveness are achieved if research questions meet the FINER criteria. In addition to the FINER criteria, Ratan et al. described focus, complexity, novelty, feasibility, and measurability for evaluating the effectiveness of research questions. 14

The PICOT and PEO frameworks are also used when developing research questions. 1 The following elements are addressed in these frameworks, PICOT: P-population/patients/problem, I-intervention or indicator being studied, C-comparison group, O-outcome of interest, and T-timeframe of the study; PEO: P-population being studied, E-exposure to preexisting conditions, and O-outcome of interest. 1 Research questions are also considered good if these meet the “FINERMAPS” framework: Feasible, Interesting, Novel, Ethical, Relevant, Manageable, Appropriate, Potential value/publishable, and Systematic. 14

As we indicated earlier, research questions and hypotheses that are not carefully formulated result in unethical studies or poor outcomes. To illustrate this, we provide some examples of ambiguous research question and hypotheses that result in unclear and weak research objectives in quantitative research ( Table 6 ) 16 and qualitative research ( Table 7 ) 17 , and how to transform these ambiguous research question(s) and hypothesis(es) into clear and good statements.

a These statements were composed for comparison and illustrative purposes only.

b These statements are direct quotes from Higashihara and Horiuchi. 16

a This statement is a direct quote from Shimoda et al. 17

The other statements were composed for comparison and illustrative purposes only.

CONSTRUCTING RESEARCH QUESTIONS AND HYPOTHESES

To construct effective research questions and hypotheses, it is very important to 1) clarify the background and 2) identify the research problem at the outset of the research, within a specific timeframe. 9 Then, 3) review or conduct preliminary research to collect all available knowledge about the possible research questions by studying theories and previous studies. 18 Afterwards, 4) construct research questions to investigate the research problem. Identify variables to be accessed from the research questions 4 and make operational definitions of constructs from the research problem and questions. Thereafter, 5) construct specific deductive or inductive predictions in the form of hypotheses. 4 Finally, 6) state the study aims . This general flow for constructing effective research questions and hypotheses prior to conducting research is shown in Fig. 1 .

Research questions are used more frequently in qualitative research than objectives or hypotheses. 3 These questions seek to discover, understand, explore or describe experiences by asking “What” or “How.” The questions are open-ended to elicit a description rather than to relate variables or compare groups. The questions are continually reviewed, reformulated, and changed during the qualitative study. 3 Research questions are also used more frequently in survey projects than hypotheses in experiments in quantitative research to compare variables and their relationships.

Hypotheses are constructed based on the variables identified and as an if-then statement, following the template, ‘If a specific action is taken, then a certain outcome is expected.’ At this stage, some ideas regarding expectations from the research to be conducted must be drawn. 18 Then, the variables to be manipulated (independent) and influenced (dependent) are defined. 4 Thereafter, the hypothesis is stated and refined, and reproducible data tailored to the hypothesis are identified, collected, and analyzed. 4 The hypotheses must be testable and specific, 18 and should describe the variables and their relationships, the specific group being studied, and the predicted research outcome. 18 Hypotheses construction involves a testable proposition to be deduced from theory, and independent and dependent variables to be separated and measured separately. 3 Therefore, good hypotheses must be based on good research questions constructed at the start of a study or trial. 12

In summary, research questions are constructed after establishing the background of the study. Hypotheses are then developed based on the research questions. Thus, it is crucial to have excellent research questions to generate superior hypotheses. In turn, these would determine the research objectives and the design of the study, and ultimately, the outcome of the research. 12 Algorithms for building research questions and hypotheses are shown in Fig. 2 for quantitative research and in Fig. 3 for qualitative research.

EXAMPLES OF RESEARCH QUESTIONS FROM PUBLISHED ARTICLES

• EXAMPLE 1. Descriptive research question (quantitative research)
• - Presents research variables to be assessed (distinct phenotypes and subphenotypes)
• “BACKGROUND: Since COVID-19 was identified, its clinical and biological heterogeneity has been recognized. Identifying COVID-19 phenotypes might help guide basic, clinical, and translational research efforts.
• RESEARCH QUESTION: Does the clinical spectrum of patients with COVID-19 contain distinct phenotypes and subphenotypes? ” 19
• EXAMPLE 2. Relationship research question (quantitative research)
• - Shows interactions between dependent variable (static postural control) and independent variable (peripheral visual field loss)
• “Background: Integration of visual, vestibular, and proprioceptive sensations contributes to postural control. People with peripheral visual field loss have serious postural instability. However, the directional specificity of postural stability and sensory reweighting caused by gradual peripheral visual field loss remain unclear.
• Research question: What are the effects of peripheral visual field loss on static postural control ?” 20
• EXAMPLE 3. Comparative research question (quantitative research)
• - Clarifies the difference among groups with an outcome variable (patients enrolled in COMPERA with moderate PH or severe PH in COPD) and another group without the outcome variable (patients with idiopathic pulmonary arterial hypertension (IPAH))
• “BACKGROUND: Pulmonary hypertension (PH) in COPD is a poorly investigated clinical condition.
• RESEARCH QUESTION: Which factors determine the outcome of PH in COPD?
• STUDY DESIGN AND METHODS: We analyzed the characteristics and outcome of patients enrolled in the Comparative, Prospective Registry of Newly Initiated Therapies for Pulmonary Hypertension (COMPERA) with moderate or severe PH in COPD as defined during the 6th PH World Symposium who received medical therapy for PH and compared them with patients with idiopathic pulmonary arterial hypertension (IPAH) .” 21
• EXAMPLE 4. Exploratory research question (qualitative research)
• - Explores areas that have not been fully investigated (perspectives of families and children who receive care in clinic-based child obesity treatment) to have a deeper understanding of the research problem
• “Problem: Interventions for children with obesity lead to only modest improvements in BMI and long-term outcomes, and data are limited on the perspectives of families of children with obesity in clinic-based treatment. This scoping review seeks to answer the question: What is known about the perspectives of families and children who receive care in clinic-based child obesity treatment? This review aims to explore the scope of perspectives reported by families of children with obesity who have received individualized outpatient clinic-based obesity treatment.” 22
• EXAMPLE 5. Relationship research question (quantitative research)
• - Defines interactions between dependent variable (use of ankle strategies) and independent variable (changes in muscle tone)
• “Background: To maintain an upright standing posture against external disturbances, the human body mainly employs two types of postural control strategies: “ankle strategy” and “hip strategy.” While it has been reported that the magnitude of the disturbance alters the use of postural control strategies, it has not been elucidated how the level of muscle tone, one of the crucial parameters of bodily function, determines the use of each strategy. We have previously confirmed using forward dynamics simulations of human musculoskeletal models that an increased muscle tone promotes the use of ankle strategies. The objective of the present study was to experimentally evaluate a hypothesis: an increased muscle tone promotes the use of ankle strategies. Research question: Do changes in the muscle tone affect the use of ankle strategies ?” 23

EXAMPLES OF HYPOTHESES IN PUBLISHED ARTICLES

• EXAMPLE 1. Working hypothesis (quantitative research)
• - A hypothesis that is initially accepted for further research to produce a feasible theory
• “As fever may have benefit in shortening the duration of viral illness, it is plausible to hypothesize that the antipyretic efficacy of ibuprofen may be hindering the benefits of a fever response when taken during the early stages of COVID-19 illness .” 24
• “In conclusion, it is plausible to hypothesize that the antipyretic efficacy of ibuprofen may be hindering the benefits of a fever response . The difference in perceived safety of these agents in COVID-19 illness could be related to the more potent efficacy to reduce fever with ibuprofen compared to acetaminophen. Compelling data on the benefit of fever warrant further research and review to determine when to treat or withhold ibuprofen for early stage fever for COVID-19 and other related viral illnesses .” 24
• EXAMPLE 2. Exploratory hypothesis (qualitative research)
• - Explores particular areas deeper to clarify subjective experience and develop a formal hypothesis potentially testable in a future quantitative approach
• “We hypothesized that when thinking about a past experience of help-seeking, a self distancing prompt would cause increased help-seeking intentions and more favorable help-seeking outcome expectations .” 25
• “Conclusion
• Although a priori hypotheses were not supported, further research is warranted as results indicate the potential for using self-distancing approaches to increasing help-seeking among some people with depressive symptomatology.” 25
• EXAMPLE 3. Hypothesis-generating research to establish a framework for hypothesis testing (qualitative research)
• “We hypothesize that compassionate care is beneficial for patients (better outcomes), healthcare systems and payers (lower costs), and healthcare providers (lower burnout). ” 26
• Compassionomics is the branch of knowledge and scientific study of the effects of compassionate healthcare. Our main hypotheses are that compassionate healthcare is beneficial for (1) patients, by improving clinical outcomes, (2) healthcare systems and payers, by supporting financial sustainability, and (3) HCPs, by lowering burnout and promoting resilience and well-being. The purpose of this paper is to establish a scientific framework for testing the hypotheses above . If these hypotheses are confirmed through rigorous research, compassionomics will belong in the science of evidence-based medicine, with major implications for all healthcare domains.” 26
• EXAMPLE 4. Statistical hypothesis (quantitative research)
• - An assumption is made about the relationship among several population characteristics ( gender differences in sociodemographic and clinical characteristics of adults with ADHD ). Validity is tested by statistical experiment or analysis ( chi-square test, Students t-test, and logistic regression analysis)
• “Our research investigated gender differences in sociodemographic and clinical characteristics of adults with ADHD in a Japanese clinical sample. Due to unique Japanese cultural ideals and expectations of women's behavior that are in opposition to ADHD symptoms, we hypothesized that women with ADHD experience more difficulties and present more dysfunctions than men . We tested the following hypotheses: first, women with ADHD have more comorbidities than men with ADHD; second, women with ADHD experience more social hardships than men, such as having less full-time employment and being more likely to be divorced.” 27
• “Statistical Analysis
• ( text omitted ) Between-gender comparisons were made using the chi-squared test for categorical variables and Students t-test for continuous variables…( text omitted ). A logistic regression analysis was performed for employment status, marital status, and comorbidity to evaluate the independent effects of gender on these dependent variables.” 27

EXAMPLES OF HYPOTHESIS AS WRITTEN IN PUBLISHED ARTICLES IN RELATION TO OTHER PARTS

• EXAMPLE 1. Background, hypotheses, and aims are provided
• “Pregnant women need skilled care during pregnancy and childbirth, but that skilled care is often delayed in some countries …( text omitted ). The focused antenatal care (FANC) model of WHO recommends that nurses provide information or counseling to all pregnant women …( text omitted ). Job aids are visual support materials that provide the right kind of information using graphics and words in a simple and yet effective manner. When nurses are not highly trained or have many work details to attend to, these job aids can serve as a content reminder for the nurses and can be used for educating their patients (Jennings, Yebadokpo, Affo, & Agbogbe, 2010) ( text omitted ). Importantly, additional evidence is needed to confirm how job aids can further improve the quality of ANC counseling by health workers in maternal care …( text omitted )” 28
• “ This has led us to hypothesize that the quality of ANC counseling would be better if supported by job aids. Consequently, a better quality of ANC counseling is expected to produce higher levels of awareness concerning the danger signs of pregnancy and a more favorable impression of the caring behavior of nurses .” 28
• “This study aimed to examine the differences in the responses of pregnant women to a job aid-supported intervention during ANC visit in terms of 1) their understanding of the danger signs of pregnancy and 2) their impression of the caring behaviors of nurses to pregnant women in rural Tanzania.” 28
• EXAMPLE 2. Background, hypotheses, and aims are provided
• “We conducted a two-arm randomized controlled trial (RCT) to evaluate and compare changes in salivary cortisol and oxytocin levels of first-time pregnant women between experimental and control groups. The women in the experimental group touched and held an infant for 30 min (experimental intervention protocol), whereas those in the control group watched a DVD movie of an infant (control intervention protocol). The primary outcome was salivary cortisol level and the secondary outcome was salivary oxytocin level.” 29
• “ We hypothesize that at 30 min after touching and holding an infant, the salivary cortisol level will significantly decrease and the salivary oxytocin level will increase in the experimental group compared with the control group .” 29
• EXAMPLE 3. Background, aim, and hypothesis are provided
• “In countries where the maternal mortality ratio remains high, antenatal education to increase Birth Preparedness and Complication Readiness (BPCR) is considered one of the top priorities [1]. BPCR includes birth plans during the antenatal period, such as the birthplace, birth attendant, transportation, health facility for complications, expenses, and birth materials, as well as family coordination to achieve such birth plans. In Tanzania, although increasing, only about half of all pregnant women attend an antenatal clinic more than four times [4]. Moreover, the information provided during antenatal care (ANC) is insufficient. In the resource-poor settings, antenatal group education is a potential approach because of the limited time for individual counseling at antenatal clinics.” 30
• “This study aimed to evaluate an antenatal group education program among pregnant women and their families with respect to birth-preparedness and maternal and infant outcomes in rural villages of Tanzania.” 30
• “ The study hypothesis was if Tanzanian pregnant women and their families received a family-oriented antenatal group education, they would (1) have a higher level of BPCR, (2) attend antenatal clinic four or more times, (3) give birth in a health facility, (4) have less complications of women at birth, and (5) have less complications and deaths of infants than those who did not receive the education .” 30

Research questions and hypotheses are crucial components to any type of research, whether quantitative or qualitative. These questions should be developed at the very beginning of the study. Excellent research questions lead to superior hypotheses, which, like a compass, set the direction of research, and can often determine the successful conduct of the study. Many research studies have floundered because the development of research questions and subsequent hypotheses was not given the thought and meticulous attention needed. The development of research questions and hypotheses is an iterative process based on extensive knowledge of the literature and insightful grasp of the knowledge gap. Focused, concise, and specific research questions provide a strong foundation for constructing hypotheses which serve as formal predictions about the research outcomes. Research questions and hypotheses are crucial elements of research that should not be overlooked. They should be carefully thought of and constructed when planning research. This avoids unethical studies and poor outcomes by defining well-founded objectives that determine the design, course, and outcome of the study.

Disclosure: The authors have no potential conflicts of interest to disclose.

Author Contributions:

• Conceptualization: Barroga E, Matanguihan GJ.
• Methodology: Barroga E, Matanguihan GJ.
• Writing - original draft: Barroga E, Matanguihan GJ.
• Writing - review & editing: Barroga E, Matanguihan GJ.

• Resources Home 🏠
• Try SciSpace Copilot
• Search research papers
• Add Copilot Extension
• Try AI Detector
• Try Paraphraser
• Try Citation Generator
• April Papers
• June Papers
• July Papers

How to Write an Effective Background of the Study: A Comprehensive Guide

Table of Contents

The background of the study in a research paper offers a clear context, highlighting why the research is essential and the problem it aims to address.

As a researcher, this foundational section is essential for you to chart the course of your study, Moreover, it allows readers to understand the importance and path of your research.

Whether in academic communities or to the general public, a well-articulated background aids in communicating the essence of the research effectively.

While it may seem straightforward, crafting an effective background requires a blend of clarity, precision, and relevance. Therefore, this article aims to be your guide, offering insights into:

• Understanding the concept of the background of the study.
• Learning how to craft a compelling background effectively.
• Identifying and sidestepping common pitfalls in writing the background.
• Exploring practical examples that bring the theory to life.
• Enhancing both your writing and reading of academic papers.

Keeping these compelling insights in mind, let's delve deeper into the details of the empirical background of the study, exploring its definition, distinctions, and the art of writing it effectively.

What is the background of the study?

The background of the study is placed at the beginning of a research paper. It provides the context, circumstances, and history that led to the research problem or topic being explored.

It offers readers a snapshot of the existing knowledge on the topic and the reasons that spurred your current research.

When crafting the background of your study, consider the following questions.

• What's the context of your research?
• Which previous research will you refer to?
• Are there any knowledge gaps in the existing relevant literature?
• How will you justify the need for your current research?
• Have you concisely presented the research question or problem?

In a typical research paper structure, after presenting the background, the introduction section follows. The introduction delves deeper into the specific objectives of the research and often outlines the structure or main points that the paper will cover.

Together, they create a cohesive starting point, ensuring readers are well-equipped to understand the subsequent sections of the research paper.

While the background of the study and the introduction section of the research manuscript may seem similar and sometimes even overlap, each serves a unique purpose in the research narrative.

Difference between background and introduction

A well-written background of the study and introduction are preliminary sections of a research paper and serve distinct purposes.

Here’s a detailed tabular comparison between the two of them.

What is the relevance of the background of the study?

It is necessary for you to provide your readers with the background of your research. Without this, readers may grapple with questions such as: Why was this specific research topic chosen? What led to this decision? Why is this study relevant? Is it worth their time?

Such uncertainties can deter them from fully engaging with your study, leading to the rejection of your research paper. Additionally, this can diminish its impact in the academic community, and reduce its potential for real-world application or policy influence .

To address these concerns and offer clarity, the background section plays a pivotal role in research papers.

The background of the study in research is important as it:

• Provides context: It offers readers a clear picture of the existing knowledge, helping them understand where the current research fits in.
• Highlights relevance: By detailing the reasons for the research, it underscores the study's significance and its potential impact.
• Guides the narrative: The background shapes the narrative flow of the paper, ensuring a logical progression from what's known to what the research aims to uncover.
• Enhances engagement: A well-crafted background piques the reader's interest, encouraging them to delve deeper into the research paper.
• Aids in comprehension: By setting the scenario, it aids readers in better grasping the research objectives, methodologies, and findings.

How to write the background of the study in a research paper?

The journey of presenting a compelling argument begins with the background study. This section holds the power to either captivate or lose the reader's interest.

An effectively written background not only provides context but also sets the tone for the entire research paper. It's the bridge that connects a broad topic to a specific research question, guiding readers through the logic behind the study.

But how does one craft a background of the study that resonates, informs, and engages?

Here, we’ll discuss how to write an impactful background study, ensuring your research stands out and captures the attention it deserves.

Identify the research problem

The first step is to start pinpointing the specific issue or gap you're addressing. This should be a significant and relevant problem in your field.

A well-defined problem is specific, relevant, and significant to your field. It should resonate with both experts and readers.

Here’s more on how to write an effective research problem .

Provide context

Here, you need to provide a broader perspective, illustrating how your research aligns with or contributes to the overarching context or the wider field of study. A comprehensive context is grounded in facts, offers multiple perspectives, and is relatable.

In addition to stating facts, you should weave a story that connects key concepts from the past, present, and potential future research. For instance, consider the following approach.

• Offer a brief history of the topic, highlighting major milestones or turning points that have shaped the current landscape.
• Discuss contemporary developments or current trends that provide relevant information to your research problem. This could include technological advancements, policy changes, or shifts in societal attitudes.
• Highlight the views of different stakeholders. For a topic like sustainable agriculture, this could mean discussing the perspectives of farmers, environmentalists, policymakers, and consumers.
• If relevant, compare and contrast global trends with local conditions and circumstances. This can offer readers a more holistic understanding of the topic.

Literature review

For this step, you’ll deep dive into the existing literature on the same topic. It's where you explore what scholars, researchers, and experts have already discovered or discussed about your topic.

Conducting a thorough literature review isn't just a recap of past works. To elevate its efficacy, it's essential to analyze the methods, outcomes, and intricacies of prior research work, demonstrating a thorough engagement with the existing body of knowledge.

• Instead of merely listing past research study, delve into their methodologies, findings, and limitations. Highlight groundbreaking studies and those that had contrasting results.
• Try to identify patterns. Look for recurring themes or trends in the literature. Are there common conclusions or contentious points?
• The next step would be to connect the dots. Show how different pieces of research relate to each other. This can help in understanding the evolution of thought on the topic.

By showcasing what's already known, you can better highlight the background of the study in research.

Highlight the research gap

This step involves identifying the unexplored areas or unanswered questions in the existing literature. Your research seeks to address these gaps, providing new insights or answers.

A clear research gap shows you've thoroughly engaged with existing literature and found an area that needs further exploration.

How can you efficiently highlight the research gap?

• Find the overlooked areas. Point out topics or angles that haven't been adequately addressed.
• Highlight questions that have emerged due to recent developments or changing circumstances.
• Identify areas where insights from other fields might be beneficial but haven't been explored yet.

State your objectives

Here, it’s all about laying out your game plan — What do you hope to achieve with your research? You need to mention a clear objective that’s specific, actionable, and directly tied to the research gap.

How to state your objectives?

• List the primary questions guiding your research.
• If applicable, state any hypotheses or predictions you aim to test.
• Specify what you hope to achieve, whether it's new insights, solutions, or methodologies.

Discuss the significance

This step describes your 'why'. Why is your research important? What broader implications does it have?

The significance of “why” should be both theoretical (adding to the existing literature) and practical (having real-world implications).

How do we effectively discuss the significance?

• Discuss how your research adds to the existing body of knowledge.
• Highlight how your findings could be applied in real-world scenarios, from policy changes to on-ground practices.
• Point out how your research could pave the way for further studies or open up new areas of exploration.

Summarize your points

A concise summary acts as a bridge, smoothly transitioning readers from the background to the main body of the paper. This step is a brief recap, ensuring that readers have grasped the foundational concepts.

How to summarize your study?

• Revisit the key points discussed, from the research problem to its significance.
• Prepare the reader for the subsequent sections, ensuring they understand the research's direction.

Include examples for better understanding

Research and come up with real-world or hypothetical examples to clarify complex concepts or to illustrate the practical applications of your research. Relevant examples make abstract ideas tangible, aiding comprehension.

How to include an effective example of the background of the study?

• Use past events or scenarios to explain concepts.
• Craft potential scenarios to demonstrate the implications of your findings.
• Use comparisons to simplify complex ideas, making them more relatable.

Crafting a compelling background of the study in research is about striking the right balance between providing essential context, showcasing your comprehensive understanding of the existing literature, and highlighting the unique value of your research .

While writing the background of the study, keep your readers at the forefront of your mind. Every piece of information, every example, and every objective should be geared toward helping them understand and appreciate your research.

How to avoid mistakes in the background of the study in research?

To write a well-crafted background of the study, you should be aware of the following potential research pitfalls .

• Stay away from ambiguity. Always assume that your reader might not be familiar with intricate details about your topic.
• Avoid discussing unrelated themes. Stick to what's directly relevant to your research problem.
• Ensure your background is well-organized. Information should flow logically, making it easy for readers to follow.
• While it's vital to provide context, avoid overwhelming the reader with excessive details that might not be directly relevant to your research problem.
• Ensure you've covered the most significant and relevant studies i` n your field. Overlooking key pieces of literature can make your background seem incomplete.
• Aim for a balanced presentation of facts, and avoid showing overt bias or presenting only one side of an argument.
• While academic paper often involves specialized terms, ensure they're adequately explained or use simpler alternatives when possible.
• Every claim or piece of information taken from existing literature should be appropriately cited. Failing to do so can lead to issues of plagiarism.
• Avoid making the background too lengthy. While thoroughness is appreciated, it should not come at the expense of losing the reader's interest. Maybe prefer to keep it to one-two paragraphs long.
• Especially in rapidly evolving fields, it's crucial to ensure that your literature review section is up-to-date and includes the latest research.

Example of an effective background of the study

Let's consider a topic: "The Impact of Online Learning on Student Performance." The ideal background of the study section for this topic would be as follows.

In the last decade, the rise of the internet has revolutionized many sectors, including education. Online learning platforms, once a supplementary educational tool, have now become a primary mode of instruction for many institutions worldwide. With the recent global events, such as the COVID-19 pandemic, there has been a rapid shift from traditional classroom learning to online modes, making it imperative to understand its effects on student performance.

Previous studies have explored various facets of online learning, from its accessibility to its flexibility. However, there is a growing need to assess its direct impact on student outcomes. While some educators advocate for its benefits, citing the convenience and vast resources available, others express concerns about potential drawbacks, such as reduced student engagement and the challenges of self-discipline.

This research aims to delve deeper into this debate, evaluating the true impact of online learning on student performance.

Why is this example considered as an effective background section of a research paper?

This background section example effectively sets the context by highlighting the rise of online learning and its increased relevance due to recent global events. It references prior research on the topic, indicating a foundation built on existing knowledge.

By presenting both the potential advantages and concerns of online learning, it establishes a balanced view, leading to the clear purpose of the study: to evaluate the true impact of online learning on student performance.

As we've explored, writing an effective background of the study in research requires clarity, precision, and a keen understanding of both the broader landscape and the specific details of your topic.

From identifying the research problem, providing context, reviewing existing literature to highlighting research gaps and stating objectives, each step is pivotal in shaping the narrative of your research. And while there are best practices to follow, it's equally crucial to be aware of the pitfalls to avoid.

Remember, writing or refining the background of your study is essential to engage your readers, familiarize them with the research context, and set the ground for the insights your research project will unveil.

Drawing from all the important details, insights and guidance shared, you're now in a strong position to craft a background of the study that not only informs but also engages and resonates with your readers.

Now that you've a clear understanding of what the background of the study aims to achieve, the natural progression is to delve into the next crucial component — write an effective introduction section of a research paper. Read here .

Frequently Asked Questions

The background of the study should include a clear context for the research, references to relevant previous studies, identification of knowledge gaps, justification for the current research, a concise overview of the research problem or question, and an indication of the study's significance or potential impact.

The background of the study is written to provide readers with a clear understanding of the context, significance, and rationale behind the research. It offers a snapshot of existing knowledge on the topic, highlights the relevance of the study, and sets the stage for the research questions and objectives. It ensures that readers can grasp the importance of the research and its place within the broader field of study.

The background of the study is a section in a research paper that provides context, circumstances, and history leading to the research problem or topic being explored. It presents existing knowledge on the topic and outlines the reasons that spurred the current research, helping readers understand the research's foundation and its significance in the broader academic landscape.

The number of paragraphs in the background of the study can vary based on the complexity of the topic and the depth of the context required. Typically, it might range from 3 to 5 paragraphs, but in more detailed or complex research papers, it could be longer. The key is to ensure that all relevant information is presented clearly and concisely, without unnecessary repetition.

You might also like

Boosting Citations: A Comparative Analysis of Graphical Abstract vs. Video Abstract

The Impact of Visual Abstracts on Boosting Citations

Introducing SciSpace’s Citation Booster To Increase Research Visibility

• Manuscript Preparation

What is the Background of a Study and How Should it be Written?

• 3 minute read

Table of Contents

The background of a study is one of the most important components of a research paper. The quality of the background determines whether the reader will be interested in the rest of the study. Thus, to ensure that the audience is invested in reading the entire research paper, it is important to write an appealing and effective background. So, what constitutes the background of a study, and how must it be written?

What is the background of a study?

The background of a study is the first section of the paper and establishes the context underlying the research. It contains the rationale, the key problem statement, and a brief overview of research questions that are addressed in the rest of the paper. The background forms the crux of the study because it introduces an unaware audience to the research and its importance in a clear and logical manner. At times, the background may even explore whether the study builds on or refutes findings from previous studies. Any relevant information that the readers need to know before delving into the paper should be made available to them in the background.

How is a background different from the introduction?

The introduction of your research paper is presented before the background. Let’s find out what factors differentiate the background from the introduction.

• The introduction only contains preliminary data about the research topic and does not state the purpose of the study. On the contrary, the background clarifies the importance of the study in detail.
• The introduction provides an overview of the research topic from a broader perspective, while the background provides a detailed understanding of the topic.
• The introduction should end with the mention of the research questions, aims, and objectives of the study. In contrast, the background follows no such format and only provides essential context to the study.

How should one write the background of a research paper?

The length and detail presented in the background varies for different research papers, depending on the complexity and novelty of the research topic. At times, a simple background suffices, even if the study is complex. Before writing and adding details in the background, take a note of these additional points:

• Start with a strong beginning: Begin the background by defining the research topic and then identify the target audience.
• Cover key components: Explain all theories, concepts, terms, and ideas that may feel unfamiliar to the target audience thoroughly.
• Take note of important prerequisites: Go through the relevant literature in detail. Take notes while reading and cite the sources.
• Maintain a balance: Make sure that the background is focused on important details, but also appeals to a broader audience.
• Include historical data: Current issues largely originate from historical events or findings. If the research borrows information from a historical context, add relevant data in the background.
• Explain novelty: If the research study or methodology is unique or novel, provide an explanation that helps to understand the research better.
• Increase engagement: To make the background engaging, build a story around the central theme of the research

Avoid these mistakes while writing the background:

• Ambiguity: Don’t be ambiguous. While writing, assume that the reader does not understand any intricate detail about your research.
• Unrelated themes: Steer clear from topics that are not related to the key aspects of your research topic.
• Poor organization: Do not place information without a structure. Make sure that the background reads in a chronological manner and organize the sub-sections so that it flows well.

Writing the background for a research paper should not be a daunting task. But directions to go about it can always help. At Elsevier Author Services we provide essential insights on how to write a high quality, appealing, and logically structured paper for publication, beginning with a robust background. For further queries, contact our experts now!

How to Use Tables and Figures effectively in Research Papers

• Research Process

The Top 5 Qualities of Every Good Researcher

You may also like.

Page-Turner Articles are More Than Just Good Arguments: Be Mindful of Tone and Structure!

A Must-see for Researchers! How to Ensure Inclusivity in Your Scientific Writing

Make Hook, Line, and Sinker: The Art of Crafting Engaging Introductions

Can Describing Study Limitations Improve the Quality of Your Paper?

A Guide to Crafting Shorter, Impactful Sentences in Academic Writing

6 Steps to Write an Excellent Discussion in Your Manuscript

How to Write Clear and Crisp Civil Engineering Papers? Here are 5 Key Tips to Consider

The Clear Path to An Impactful Paper: ②

Input your search keywords and press Enter.

What Is Background in a Research Paper?

So you have carefully written your research paper  and probably ran it through your colleagues ten to fifteen times. While there are many elements to a good research article, one of the most important elements for your readers is the background of your study.

What is Background of the Study in Research

The background of your study will provide context to the information discussed throughout the research paper . Background information may include both important and relevant studies. This is particularly important if a study either supports or refutes your thesis.

Why is Background of the Study Necessary in Research?

The background of the study discusses your problem statement, rationale, and research questions. It links  introduction to your research topic  and ensures a logical flow of ideas.  Thus, it helps readers understand your reasons for conducting the study.

Providing Background Information

The reader should be able to understand your topic and its importance. The length and detail of your background also depend on the degree to which you need to demonstrate your understanding of the topic. Paying close attention to the following questions will help you in writing background information:

• Are there any theories, concepts, terms, and ideas that may be unfamiliar to the target audience and will require you to provide any additional explanation?
• Any historical data that need to be shared in order to provide context on why the current issue emerged?
• Are there any concepts that may have been borrowed from other disciplines that may be unfamiliar to the reader and need an explanation?

Related: Ready with the background and searching for more information on journal ranking? Check this infographic on the SCImago Journal Rank today!

Is the research study unique for which additional explanation is needed? For instance, you may have used a completely new method

How to Write a Background of the Study

The structure of a background study in a research paper generally follows a logical sequence to provide context, justification, and an understanding of the research problem. It includes an introduction, general background, literature review , rationale , objectives, scope and limitations , significance of the study and the research hypothesis . Following the structure can provide a comprehensive and well-organized background for your research.

Here are the steps to effectively write a background of the study.

1. Identify Your Audience:

Determine the level of expertise of your target audience. Tailor the depth and complexity of your background information accordingly.

2. Understand the Research Problem:

Define the research problem or question your study aims to address. Identify the significance of the problem within the broader context of the field.

3. Review Existing Literature:

Conduct a thorough literature review to understand what is already known in the area. Summarize key findings, theories, and concepts relevant to your research.

4. Include Historical Data:

Integrate historical data if relevant to the research, as current issues often trace back to historical events.

5. Identify Controversies and Gaps:

Note any controversies or debates within the existing literature. Identify gaps , limitations, or unanswered questions that your research can address.

6. Select Key Components:

Choose the most critical elements to include in the background based on their relevance to your research problem. Prioritize information that helps build a strong foundation for your study.

7. Craft a Logical Flow:

Organize the background information in a logical sequence. Start with general context, move to specific theories and concepts, and then focus on the specific problem.

8. Highlight the Novelty of Your Research:

Clearly explain the unique aspects or contributions of your study. Emphasize why your research is different from or builds upon existing work.

Here are some extra tips to increase the quality of your research background:

Example of a Research Background

Here is an example of a research background to help you understand better.

The above hypothetical example provides a research background, addresses the gap and highlights the potential outcome of the study; thereby aiding a better understanding of the proposed research.

What Makes the Introduction Different from the Background?

Your introduction is different from your background in a number of ways.

• The introduction contains preliminary data about your topic that  the reader will most likely read , whereas the background clarifies the importance of the paper.
• The background of your study discusses in depth about the topic, whereas the introduction only gives an overview.
• The introduction should end with your research questions, aims, and objectives, whereas your background should not (except in some cases where your background is integrated into your introduction). For instance, the C.A.R.S. ( Creating a Research Space ) model, created by John Swales is based on his analysis of journal articles. This model attempts to explain and describe the organizational pattern of writing the introduction in social sciences.

Points to Note

Your background should begin with defining a topic and audience. It is important that you identify which topic you need to review and what your audience already knows about the topic. You should proceed by searching and researching the relevant literature. In this case, it is advisable to keep track of the search terms you used and the articles that you downloaded. It is helpful to use one of the research paper management systems such as Papers, Mendeley, Evernote, or Sente. Next, it is helpful to take notes while reading. Be careful when copying quotes verbatim and make sure to put them in quotation marks and cite the sources. In addition, you should keep your background focused but balanced enough so that it is relevant to a broader audience. Aside from these, your background should be critical, consistent, and logically structured.

Writing the background of your study should not be an overly daunting task. Many guides that can help you organize your thoughts as you write the background. The background of the study is the key to introduce your audience to your research topic and should be done with strong knowledge and thoughtful writing.

The background of a research paper typically ranges from one to two paragraphs, summarizing the relevant literature and context of the study. It should be concise, providing enough information to contextualize the research problem and justify the need for the study. Journal instructions about any word count limits should be kept in mind while deciding on the length of the final content.

The background of a research paper provides the context and relevant literature to understand the research problem, while the introduction also introduces the specific research topic, states the research objectives, and outlines the scope of the study. The background focuses on the broader context, whereas the introduction focuses on the specific research project and its objectives.

When writing the background for a study, start by providing a brief overview of the research topic and its significance in the field. Then, highlight the gaps in existing knowledge or unresolved issues that the study aims to address. Finally, summarize the key findings from relevant literature to establish the context and rationale for conducting the research, emphasizing the need and importance of the study within the broader academic landscape.

The background in a research paper is crucial as it sets the stage for the study by providing essential context and rationale. It helps readers understand the significance of the research problem and its relevance in the broader field. By presenting relevant literature and highlighting gaps, the background justifies the need for the study, building a strong foundation for the research and enhancing its credibility.

The presentation very informative

It is really educative. I love the workshop. It really motivated me into writing my first paper for publication.

an interesting clue here, thanks.

thanks for the answers.

Good and interesting explanation. Thanks

Thank you for good presentation.

Hi Adam, we are glad to know that you found our article beneficial

The background of the study is the key to introduce your audience to YOUR research topic.

Awesome. Exactly what i was looking forwards to 😉

Hi Maryam, we are glad to know that you found our resource useful.

my understanding of ‘Background of study’ has been elevated.

Hi Peter, we are glad to know that our article has helped you get a better understanding of the background in a research paper.

thanks to give advanced information

Hi Shimelis, we are glad to know that you found the information in our article beneficial.

When i was studying it is very much hard for me to conduct a research study and know the background because my teacher in practical research is having a research so i make it now so that i will done my research

Very informative……….Thank you.

The confusion i had before, regarding an introduction and background to a research work is now a thing of the past. Thank you so much.

Thanks for your help…

Thanks for your kind information about the background of a research paper.

Thanks for the answer

Very informative. I liked even more when the difference between background and introduction was given. I am looking forward to learning more from this site. I am in Botswana

Hello, I am Benoît from Central African Republic. Right now I am writing down my research paper in order to get my master degree in British Literature. Thank you very much for posting all this information about the background of the study. I really appreciate. Thanks!

The write up is quite good, detailed and informative. Thanks a lot. The article has certainly enhanced my understanding of the topic.

Rate this article Cancel Reply

Your email address will not be published.

Enago Academy's Most Popular Articles

• AI in Academia
• Infographic
• Manuscripts & Grants
• Reporting Research
• Trending Now

Can AI Tools Prepare a Research Manuscript From Scratch? — A comprehensive guide

As technology continues to advance, the question of whether artificial intelligence (AI) tools can prepare…

Abstract Vs. Introduction — Do you know the difference?

Ross wants to publish his research. Feeling positive about his research outcomes, he begins to…

• Old Webinars
• Webinar Mobile App

Demystifying Research Methodology With Field Experts

Choosing research methodology Research design and methodology Evidence-based research approach How RAxter can assist researchers

• Manuscript Preparation
• Publishing Research

How to Choose Best Research Methodology for Your Study

Successful research conduction requires proper planning and execution. While there are multiple reasons and aspects…

Top 5 Key Differences Between Methods and Methodology

While burning the midnight oil during literature review, most researchers do not realize that the…

How to Draft the Acknowledgment Section of a Manuscript

Discussion Vs. Conclusion: Know the Difference Before Drafting Manuscripts

Sign-up to read more

Subscribe for free to get unrestricted access to all our resources on research writing and academic publishing including:

• 2000+ blog articles
• 50+ Webinars
• 10+ Expert podcasts
• 50+ Infographics
• 10+ Checklists
• Research Guides

We hate spam too. We promise to protect your privacy and never spam you.

I am looking for Editing/ Proofreading services for my manuscript Tentative date of next journal submission:

What would be most effective in reducing research misconduct?

Writing Research Background

Research background is a brief outline of the most important studies that have been conducted so far presented in a chronological order. Research background part in introduction chapter can be also headed ‘Background of the Study.” Research background should also include a brief discussion of major theories and models related to the research problem.

Specifically, when writing research background you can discuss major theories and models related to your research problem in a chronological order to outline historical developments in the research area.  When writing research background, you also need to demonstrate how your research relates to what has been done so far in the research area.

Research background is written after the literature review. Therefore, literature review has to be the first and the longest stage in the research process, even before the formulation of research aims and objectives, right after the selection of the research area. Once the research area is selected, the literature review is commenced in order to identify gaps in the research area.

Research aims and objectives need to be closely associated with the elimination of this gap in the literature. The main difference between background of the study and literature review is that the former only provides general information about what has been done so far in the research area, whereas the latter elaborates and critically reviews previous works.

John Dudovskiy

• Affiliate Program

• UNITED STATES
• 台灣 (TAIWAN)
• TÜRKIYE (TURKEY)
• Academic Editing Services
• - Research Paper
• - Journal Manuscript
• - Dissertation
• - College & University Assignments
• Admissions Editing Services
• - Application Essay
• - Personal Statement
• - Recommendation Letter
• - Cover Letter
• - CV/Resume
• Business Editing Services
• - Business Documents
• - Report & Brochure
• - Website & Blog
• Writer Editing Services
• - Script & Screenplay
• Our Editors
• Client Reviews
• Editing & Proofreading Prices
• Wordvice Points
• Partner Discount
• Plagiarism Checker
• APA Citation Generator
• MLA Citation Generator
• Chicago Citation Generator
• Vancouver Citation Generator
• - APA Style
• - MLA Style
• - Chicago Style
• - Vancouver Style
• Writing & Editing Guide
• Academic Resources
• Admissions Resources

How to Write the Rationale of the Study in Research (Examples)

What is the Rationale of the Study?

The rationale of the study is the justification for taking on a given study. It explains the reason the study was conducted or should be conducted. This means the study rationale should explain to the reader or examiner why the study is/was necessary. It is also sometimes called the “purpose” or “justification” of a study. While this is not difficult to grasp in itself, you might wonder how the rationale of the study is different from your research question or from the statement of the problem of your study, and how it fits into the rest of your thesis or research paper. 

The rationale of the study links the background of the study to your specific research question and justifies the need for the latter on the basis of the former. In brief, you first provide and discuss existing data on the topic, and then you tell the reader, based on the background evidence you just presented, where you identified gaps or issues and why you think it is important to address those. The problem statement, lastly, is the formulation of the specific research question you choose to investigate, following logically from your rationale, and the approach you are planning to use to do that.

Table of Contents:

How to write a rationale for a research paper , how do you justify the need for a research study.

• Study Rationale Example: Where Does It Go In Your Paper?

The basis for writing a research rationale is preliminary data or a clear description of an observation. If you are doing basic/theoretical research, then a literature review will help you identify gaps in current knowledge. In applied/practical research, you base your rationale on an existing issue with a certain process (e.g., vaccine proof registration) or practice (e.g., patient treatment) that is well documented and needs to be addressed. By presenting the reader with earlier evidence or observations, you can (and have to) convince them that you are not just repeating what other people have already done or said and that your ideas are not coming out of thin air. 

Once you have explained where you are coming from, you should justify the need for doing additional research–this is essentially the rationale of your study. Finally, when you have convinced the reader of the purpose of your work, you can end your introduction section with the statement of the problem of your research that contains clear aims and objectives and also briefly describes (and justifies) your methodological approach. 

When is the Rationale for Research Written?

The author can present the study rationale both before and after the research is conducted. 

• Before conducting research : The study rationale is a central component of the research proposal . It represents the plan of your work, constructed before the study is actually executed.
• Once research has been conducted : After the study is completed, the rationale is presented in a research article or  PhD dissertation  to explain why you focused on this specific research question. When writing the study rationale for this purpose, the author should link the rationale of the research to the aims and outcomes of the study.

What to Include in the Study Rationale

Although every study rationale is different and discusses different specific elements of a study’s method or approach, there are some elements that should be included to write a good rationale. Make sure to touch on the following:

• A summary of conclusions from your review of the relevant literature
• What is currently unknown (gaps in knowledge)
• Inconclusive or contested results  from previous studies on the same or similar topic
• The necessity to improve or build on previous research, such as to improve methodology or utilize newer techniques and/or technologies

There are different types of limitations that you can use to justify the need for your study. In applied/practical research, the justification for investigating something is always that an existing process/practice has a problem or is not satisfactory. Let’s say, for example, that people in a certain country/city/community commonly complain about hospital care on weekends (not enough staff, not enough attention, no decisions being made), but you looked into it and realized that nobody ever investigated whether these perceived problems are actually based on objective shortages/non-availabilities of care or whether the lower numbers of patients who are treated during weekends are commensurate with the provided services.

In this case, “lack of data” is your justification for digging deeper into the problem. Or, if it is obvious that there is a shortage of staff and provided services on weekends, you could decide to investigate which of the usual procedures are skipped during weekends as a result and what the negative consequences are. 

In basic/theoretical research, lack of knowledge is of course a common and accepted justification for additional research—but make sure that it is not your only motivation. “Nobody has ever done this” is only a convincing reason for a study if you explain to the reader why you think we should know more about this specific phenomenon. If there is earlier research but you think it has limitations, then those can usually be classified into “methodological”, “contextual”, and “conceptual” limitations. To identify such limitations, you can ask specific questions and let those questions guide you when you explain to the reader why your study was necessary:

Methodological limitations

• Did earlier studies try but failed to measure/identify a specific phenomenon?
• Was earlier research based on incorrect conceptualizations of variables?
• Were earlier studies based on questionable operationalizations of key concepts?
• Did earlier studies use questionable or inappropriate research designs?

Contextual limitations

• Have recent changes in the studied problem made previous studies irrelevant?
• Are you studying a new/particular context that previous findings do not apply to?

Conceptual limitations

• Do previous findings only make sense within a specific framework or ideology?

Study Rationale Examples

Let’s look at an example from one of our earlier articles on the statement of the problem to clarify how your rationale fits into your introduction section. This is a very short introduction for a practical research study on the challenges of online learning. Your introduction might be much longer (especially the context/background section), and this example does not contain any sources (which you will have to provide for all claims you make and all earlier studies you cite)—but please pay attention to how the background presentation , rationale, and problem statement blend into each other in a logical way so that the reader can follow and has no reason to question your motivation or the foundation of your research.

Background presentation

Since the beginning of the Covid pandemic, most educational institutions around the world have transitioned to a fully online study model, at least during peak times of infections and social distancing measures. This transition has not been easy and even two years into the pandemic, problems with online teaching and studying persist (reference needed) . 

While the increasing gap between those with access to technology and equipment and those without access has been determined to be one of the main challenges (reference needed) , others claim that online learning offers more opportunities for many students by breaking down barriers of location and distance (reference needed) .  

Rationale of the study

Since teachers and students cannot wait for circumstances to go back to normal, the measures that schools and universities have implemented during the last two years, their advantages and disadvantages, and the impact of those measures on students’ progress, satisfaction, and well-being need to be understood so that improvements can be made and demographics that have been left behind can receive the support they need as soon as possible.

Statement of the problem

To identify what changes in the learning environment were considered the most challenging and how those changes relate to a variety of student outcome measures, we conducted surveys and interviews among teachers and students at ten institutions of higher education in four different major cities, two in the US (New York and Chicago), one in South Korea (Seoul), and one in the UK (London). Responses were analyzed with a focus on different student demographics and how they might have been affected differently by the current situation.

How long is a study rationale?

In a research article bound for journal publication, your rationale should not be longer than a few sentences (no longer than one brief paragraph). A  dissertation or thesis  usually allows for a longer description; depending on the length and nature of your document, this could be up to a couple of paragraphs in length. A completely novel or unconventional approach might warrant a longer and more detailed justification than an approach that slightly deviates from well-established methods and approaches.

Consider Using Professional Academic Editing Services

Now that you know how to write the rationale of the study for a research proposal or paper, you should make use of Wordvice AI’s free AI Grammar Checker , or receive professional academic proofreading services from Wordvice, including research paper editing services and manuscript editing services to polish your submitted research documents.

You can also find many more articles, for example on writing the other parts of your research paper , on choosing a title , or on making sure you understand and adhere to the author instructions before you submit to a journal, on the Wordvice academic resources pages.

PHILO-notes

Free Online Learning Materials

How to Write the Background of the Study in Research (Part 1)

Background of the Study in Research: Definition and the Core Elements it Contains

Before we embark on a detailed discussion on how to write the background of the study of your proposed research or thesis, it is important to first discuss its meaning and the core elements that it should contain. This is obviously because understanding the nature of the background of the study in research and knowing exactly what to include in it allow us to have both greater control and clear direction of the writing process.

So, what really is the background of the study and what are the core elements that it should contain?

The background of the study, which usually forms the first section of the introduction to a research paper or thesis, provides the overview of the study. In other words, it is that section of the research paper or thesis that establishes the context of the study. Its main function is to explain why the proposed research is important and essential to understanding the main aspects of the study.

The background of the study, therefore, is the section of the research paper or thesis that identifies the problem or gap of the study that needs to addressed and justifies the need for conducting the study. It also articulates the main goal of the study and the thesis statement, that is, the main claim or argument of the paper.

Given this brief understanding of the background of the study, we can anticipate what readers or thesis committee members expect from it. As we can see, the background of the study should contain the following major points:

1) brief discussion on what is known about the topic under investigation; 2) An articulation of the research gap or problem that needs to be addressed; 3) What the researcher would like to do or aim to achieve in the study ( research goal); 4) The thesis statement, that is, the main argument or contention of the paper (which also serves as the reason why the researcher would want to pursue the study); 5) The major significance or contribution of the study to a particular discipline; and 6) Depending on the nature of the study, an articulation of the hypothesis of the study.

Thus, when writing the background of the study, you should plan and structure it based on the major points just mentioned. With this, you will have a clear picture of the flow of the tasks that need to be completed in writing this section of your research or thesis proposal.

Now, how do you go about writing the background of the study in your proposed research or thesis?

The next lessons will address this question.

How to Write the Opening Paragraphs of the Background of the Study?

To begin with, let us assume that you already have conducted a preliminary research on your chosen topic, that is, you already have read a lot of literature and gathered relevant information for writing the background of your study. Let us also assume that you already have identified the gap of your proposed research and have already developed the research questions and thesis statement. If you have not yet identified the gap in your proposed research, you might as well go back to our lesson on how to identify a research gap.

So, we will just put together everything that you have researched into a background of the study (assuming, again, that you already have the necessary information). But in this lesson, let’s just focus on writing the opening paragraphs.

It is important to note at this point that there are different styles of writing the background of the study. Hence, what I will be sharing with you here is not just “the” only way of writing the background of the study. As a matter of fact, there is no “one-size-fits-all” style of writing this part of the research or thesis. At the end of the day, you are free to develop your own. However, whatever style it would be, it always starts with a plan which structures the writing process into stages or steps. The steps that I will share with below are just some of the most effective ways of writing the background of the study in research.

So, let’s begin.

It is always a good idea to begin the background of your study by giving an overview of your research topic. This may include providing a definition of the key concepts of your research or highlighting the main developments of the research topic.

Let us suppose that the topic of your study is the “lived experiences of students with mathematical anxiety”.

Here, you may start the background of your study with a discussion on the meaning, nature, and dynamics of the term “mathematical anxiety”. The reason for this is too obvious: “mathematical anxiety” is a highly technical term that is specific to mathematics. Hence, this term is not readily understandable to non-specialists in this field.

So, you may write the opening paragraph of your background of the study with this:

“Mathematical anxiety refers to the individual’s unpleasant emotional mood responses when confronted with a mathematical situation.”

Since you do not invent the definition of the term “mathematical anxiety”, then you need to provide a citation to the source of the material from which you are quoting. For example, you may now say:

“Mathematical anxiety refers to the individual’s unpleasant emotional mood responses when confronted with a mathematical situation (Eliot, 2020).”

And then you may proceed with the discussion on the nature and dynamics of the term “mathematical anxiety”. You may say:

“Lou (2019) specifically identifies some of the manifestations of this type of anxiety, which include, but not limited to, depression, helplessness, nervousness and fearfulness in doing mathematical and numerical tasks.”

After explaining to your readers the meaning, nature, and dynamics (as well as some historical development if you wish to) of the term “mathematical anxiety”, you may now proceed to showing the problem or gap of the study. As you may already know, the research gap is the problem that needs to be addressed in the study. This is important because no research activity is possible without the research gap.

Let us suppose that your research problem or gap is: “Mathematical anxiety can negatively affect not just the academic achievement of the students but also their future career plans and total well-being. Also, there are no known studies that deal with the mathematical anxiety of junior high school students in New Zealand.” With this, you may say:

“If left unchecked, as Shapiro (2019) claims, this problem will expand and create a total avoidance pattern on the part of the students, which can be expressed most visibly in the form of cutting classes and habitual absenteeism. As we can see, this will negatively affect the performance of students in mathematics. In fact, the study conducted by Luttenberger and Wimmer (2018) revealed that the outcomes of mathematical anxiety do not only negatively affect the students’ performance in math-related situations but also their future career as professionals. Without a doubt, therefore, mathematical anxiety is a recurring problem for many individuals which will negatively affect the academic success and future career of the student.”

Now that you already have both explained the meaning, nature, and dynamics of the term “mathematical anxiety” and articulated the gap of your proposed research, you may now state the main goal of your study. You may say:

“Hence, it is precisely in this context that the researcher aims to determine the lived experiences of those students with mathematical anxiety. In particular, this proposed thesis aims to determine the lived experiences of the junior high school students in New Zealand and identify the factors that caused them to become disinterested in mathematics.”

Please note that you should not end the first paragraph of your background of the study with the articulation of the research goal. You also need to articulate the “thesis statement”, which usually comes after the research goal. As is well known, the thesis statement is the statement of your argument or contention in the study. It is more of a personal argument or claim of the researcher, which specifically highlights the possible contribution of the study. For example, you may say:

“The researcher argues that there is a need to determine the lived experiences of these students with mathematical anxiety because knowing and understanding the difficulties and challenges that they have encountered will put the researcher in the best position to offer some alternatives to the problem. Indeed, it is only when we have performed some kind of a ‘diagnosis’ that we can offer practicable solutions to the problem. And in the case of the junior high school students in New Zealand who are having mathematical anxiety, determining their lived experiences as well as identifying the factors that caused them to become disinterested in mathematics are the very first steps in addressing the problem.”

If we combine the bits and pieces that we have written above, we can now come up with the opening paragraphs of your background of the study, which reads:

As we can see, we can find in the first paragraph 5 essential elements that must be articulated in the background of the study, namely:

1) A brief discussion on what is known about the topic under investigation; 2) An articulation of the research gap or problem that needs to be addressed; 3) What the researcher would like to do or aim to achieve in the study (research goal); 4) The thesis statement , that is, the main argument or claim of the paper; and 5) The major significance or contribution of the study to a particular discipline. So, that’s how you write the opening paragraphs of your background of the study. The next lesson will talk about writing the body of the background of the study.

How to Write the Body of the Background of the Study?

If we liken the background of the study to a sitting cat, then the opening paragraphs that we have completed in the previous lesson would just represent the head of the cat.

This means we still have to write the body (body of the cat) and the conclusion (tail). But how do we write the body of the background of the study? What should be its content?

Truly, this is one of the most difficult challenges that fledgling scholars faced. Because they are inexperienced researchers and didn’t know what to do next, they just wrote whatever they wished to write. Fortunately, this is relatively easy if they know the technique.

One of the best ways to write the body of the background of the study is to attack it from the vantage point of the research gap. If you recall, when we articulated the research gap in the opening paragraphs, we made a bold claim there, that is, there are junior high school students in New Zealand who are experiencing mathematical anxiety. Now, you have to remember that a “statement” remains an assumption until you can provide concrete proofs to it. This is what we call the “epistemological” aspect of research. As we may already know, epistemology is a specific branch of philosophy that deals with the validity of knowledge. And to validate knowledge is to provide concrete proofs to our statements. Hence, the reason why we need to provide proofs to our claim that there are indeed junior high school students in New Zealand who are experiencing mathematical anxiety is the obvious fact that if there are none, then we cannot proceed with our study. We have no one to interview with in the first. In short, we don’t have respondents.

The body of the background of the study, therefore, should be a presentation and articulation of the proofs to our claim that indeed there are junior high school students in New Zealand who are experiencing mathematical anxiety. Please note, however, that this idea is true only if you follow the style of writing the background of the study that I introduced in this course.

So, how do we do this?

One of the best ways to do this is to look for literature on mathematical anxiety among junior high school students in New Zealand and cite them here. However, if there are not enough literature on this topic in New Zealand, then we need to conduct initial interviews with these students or make actual classroom observations and record instances of mathematical anxiety among these students. But it is always a good idea if we combine literature review with interviews and actual observations.

Assuming you already have the data, then you may now proceed with the writing of the body of your background of the study. For example, you may say:

“According to records and based on the researcher’s firsthand experience with students in some junior high schools in New Zealand, indeed, there are students who lost interest in mathematics. For one, while checking the daily attendance and monitoring of the students, it was observed that some of them are not always attending classes in mathematics but are regularly attending the rest of the required subjects.”

After this sentence, you may insert some literature that will support this position. For example, you may say:

“As a matter of fact, this phenomenon is also observed in the work of Estonanto. In his study titled ‘Impact of Math Anxiety on Academic Performance in Pre-Calculus of Senior High School’, Estonanto (2019) found out that, inter alia, students with mathematical anxiety have the tendency to intentionally prioritize other subjects and commit habitual tardiness and absences.”

Then you may proceed saying:

“With this initial knowledge in mind, the researcher conducted initial interviews with some of these students. The researcher learned that one student did not regularly attend his math subject because he believed that he is not good in math and no matter how he listens to the topic he will not learn.”

Then you may say:

“Another student also mentioned that she was influenced by her friends’ perception that mathematics is hard; hence, she avoids the subject. Indeed, these are concrete proofs that there are some junior high school students in New Zealand who have mathematical anxiety. As already hinted, “disinterest” or the loss of interest in mathematics is one of the manifestations of a mathematical anxiety.”

If we combine what we have just written above, then we can have the first two paragraphs of the body of our background of the study. It reads:

“According to records and based on the researcher’s firsthand experience with students in some junior high schools in New Zealand, indeed there are students who lost interest in mathematics. For one, while checking the daily attendance and monitoring of the students, it was observed that some of them are not always attending classes in mathematics but are regularly attending the rest of the required subjects. As a matter of fact, this phenomenon is also observed in the work of Estonanto. In his study titled ‘Impact of Math Anxiety on Academic Performance in Pre-Calculus of Senior High School’, Estonanto (2019) found out that, inter alia, students with mathematical anxiety have the tendency to intentionally prioritize other subjects and commit habitual tardiness and absences.

With this initial knowledge in mind, the researcher conducted initial interviews with some of these students. The researcher learned that one student did not regularly attend his math subject because he believed that he is not good in math and no matter how he listens to the topic he will not learn. Another student also mentioned that she was influenced by her friends’ perception that mathematics is hard; hence, she avoids the subject. Indeed, these are concrete proofs that there are some junior high school students in New Zealand who have mathematical anxiety. As already hinted, “disinterest” or the loss of interest in mathematics is one of the manifestations of a mathematical anxiety.”

And then you need validate this observation by conducting another round of interview and observation in other schools. So, you may continue writing the body of the background of the study with this:

“To validate the information gathered from the initial interviews and observations, the researcher conducted another round of interview and observation with other junior high school students in New Zealand.”

“On the one hand, the researcher found out that during mathematics time some students felt uneasy; in fact, they showed a feeling of being tensed or anxious while working with numbers and mathematical problems. Some were even afraid to seat in front, while some students at the back were secretly playing with their mobile phones. These students also show remarkable apprehension during board works like trembling hands, nervous laughter, and the like.”

Then provide some literature that will support your position. You may say:

“As Finlayson (2017) corroborates, emotional symptoms of mathematical anxiety involve feeling of helplessness, lack of confidence, and being nervous for being put on the spot. It must be noted that these occasionally extreme emotional reactions are not triggered by provocative procedures. As a matter of fact, there are no personally sensitive questions or intentional manipulations of stress. The teacher simply asked a very simple question, like identifying the parts of a circle. Certainly, this observation also conforms with the study of Ashcraft (2016) when he mentions that students with mathematical anxiety show a negative attitude towards math and hold self-perceptions about their mathematical abilities.”

And then you proceed:

“On the other hand, when the class had their other subjects, the students show a feeling of excitement. They even hurried to seat in front and attentively participating in the class discussion without hesitation and without the feeling of being tensed or anxious. For sure, this is another concrete proof that there are junior high school students in New Zealand who have mathematical anxiety.”

To further prove the point that there indeed junior high school students in New Zealand who have mathematical anxiety, you may solicit observations from other math teachers. For instance, you may say:

“The researcher further verified if the problem is also happening in other sections and whether other mathematics teachers experienced the same observation that the researcher had. This validation or verification is important in establishing credibility of the claim (Buchbinder, 2016) and ensuring reliability and validity of the assertion (Morse et al., 2002). In this regard, the researcher attempted to open up the issue of math anxiety during the Departmentalized Learning Action Cell (LAC), a group discussion of educators per quarter, with the objective of ‘Teaching Strategies to Develop Critical Thinking of the Students’. During the session, one teacher corroborates the researcher’s observation that there are indeed junior high school students in New Zealand who have mathematical anxiety. The teacher pointed out that truly there were students who showed no extra effort in mathematics class in addition to the fact that some students really avoided the subject. In addition, another math teacher expressed her frustrations about these students who have mathematical anxiety. She quipped: “How can a teacher develop the critical thinking skills or ability of the students if in the first place these students show avoidance and disinterest in the subject?’.”

Again, if we combine what we have just written above, then we can now have the remaining parts of the body of the background of the study. It reads:

So, that’s how we write the body of the background of the study in research . Of course, you may add any relevant points which you think might amplify your content. What is important at this point is that you now have a clear idea of how to write the body of the background of the study.

How to Write the Concluding Part of the Background of the Study?

Since we have already completed the body of our background of the study in the previous lesson, we may now write the concluding paragraph (the tail of the cat). This is important because one of the rules of thumb in writing is that we always put a close to what we have started.

It is important to note that the conclusion of the background of the study is just a rehashing of the research gap and main goal of the study stated in the introductory paragraph, but framed differently. The purpose of this is just to emphasize, after presenting the justifications, what the study aims to attain and why it wants to do it. The conclusion, therefore, will look just like this:

“Given the above discussion, it is evident that there are indeed junior high school students in New Zealand who are experiencing mathematical anxiety. And as we can see, mathematical anxiety can negatively affect not just the academic achievement of the students but also their future career plans and total well-being. Again, it is for this reason that the researcher attempts to determine the lived experiences of those junior high school students in New Zealand who are experiencing a mathematical anxiety.”

If we combine all that we have written from the very beginning, the entire background of the study would now read:

If we analyze the background of the study that we have just completed, we can observe that in addition to the important elements that it should contain, it has also addressed other important elements that readers or thesis committee members expect from it.

On the one hand, it provides the researcher with a clear direction in the conduct of the study. As we can see, the background of the study that we have just completed enables us to move in the right direction with a strong focus as it has set clear goals and the reasons why we want to do it. Indeed, we now exactly know what to do next and how to write the rest of the research paper or thesis.

On the other hand, most researchers start their research with scattered ideas and usually get stuck with how to proceed further. But with a well-written background of the study, just as the one above, we have decluttered and organized our thoughts. We have also become aware of what have and have not been done in our area of study, as well as what we can significantly contribute in the already existing body of knowledge in this area of study.

Please note, however, as I already mentioned previously, that the model that I have just presented is only one of the many models available in textbooks and other sources. You are, of course, free to choose your own style of writing the background of the study. You may also consult your thesis supervisor for some guidance on how to attack the writing of your background of the study.

Lastly, and as you may already know, universities around the world have their own thesis formats. Hence, you should follow your university’s rules on the format and style in writing your research or thesis. What is important is that with the lessons that you learned in this course, you can now easily write the introductory part of your thesis, such as the background of the study.

Quantitative and Qualitative Research

• I NEED TO . . .

What is Quantitative Research?

• What is Qualitative Research?
• Quantitative vs Qualitative
• Step 1: Accessing CINAHL
• Step 2: Create a Keyword Search
• Step 3: Create a Subject Heading Search
• Step 4: Repeat Steps 1-3 for Second Concept
• Step 5: Repeat Steps 1-3 for Quantitative Terms
• Step 6: Combining All Searches
• Step 7: Adding Limiters
• Step 8: Save Your Search!
• What Kind of Article is This?
• More Research Help This link opens in a new window

Quantitative methodology is the dominant research framework in the social sciences. It refers to a set of strategies, techniques and assumptions used to study psychological, social and economic processes through the exploration of numeric patterns . Quantitative research gathers a range of numeric data. Some of the numeric data is intrinsically quantitative (e.g. personal income), while in other cases the numeric structure is  imposed (e.g. ‘On a scale from 1 to 10, how depressed did you feel last week?’). The collection of quantitative information allows researchers to conduct simple to extremely sophisticated statistical analyses that aggregate the data (e.g. averages, percentages), show relationships among the data (e.g. ‘Students with lower grade point averages tend to score lower on a depression scale’) or compare across aggregated data (e.g. the USA has a higher gross domestic product than Spain). Quantitative research includes methodologies such as questionnaires, structured observations or experiments and stands in contrast to qualitative research. Qualitative research involves the collection and analysis of narratives and/or open-ended observations through methodologies such as interviews, focus groups or ethnographies.

Coghlan, D., Brydon-Miller, M. (2014).  The SAGE encyclopedia of action research  (Vols. 1-2). London, : SAGE Publications Ltd doi: 10.4135/9781446294406

What is the purpose of quantitative research?

The purpose of quantitative research is to generate knowledge and create understanding about the social world. Quantitative research is used by social scientists, including communication researchers, to observe phenomena or occurrences affecting individuals. Social scientists are concerned with the study of people. Quantitative research is a way to learn about a particular group of people, known as a sample population. Using scientific inquiry, quantitative research relies on data that are observed or measured to examine questions about the sample population.

Allen, M. (2017).  The SAGE encyclopedia of communication research methods  (Vols. 1-4). Thousand Oaks, CA: SAGE Publications, Inc doi: 10.4135/9781483381411

How do I know if the study is a quantitative design?  What type of quantitative study is it?

Quantitative Research Designs: Descriptive non-experimental, Quasi-experimental or Experimental?

Studies do not always explicitly state what kind of research design is being used.  You will need to know how to decipher which design type is used.  The following video will help you determine the quantitative design type.

• << Previous: I NEED TO . . .
• Next: What is Qualitative Research? >>
• Last Updated: Jun 12, 2024 8:56 AM
• URL: https://libguides.uta.edu/quantitative_and_qualitative_research

University of Texas Arlington Libraries 702 Planetarium Place · Arlington, TX 76019 · 817-272-3000

• Internet Privacy
• Accessibility
• Problems with a guide? Contact Us.

• Privacy Policy

Home » Quantitative Research – Methods, Types and Analysis

Quantitative Research – Methods, Types and Analysis

Table of Contents

Quantitative Research

Quantitative research is a type of research that collects and analyzes numerical data to test hypotheses and answer research questions . This research typically involves a large sample size and uses statistical analysis to make inferences about a population based on the data collected. It often involves the use of surveys, experiments, or other structured data collection methods to gather quantitative data.

Quantitative Research Methods

Quantitative Research Methods are as follows:

Descriptive Research Design

Descriptive research design is used to describe the characteristics of a population or phenomenon being studied. This research method is used to answer the questions of what, where, when, and how. Descriptive research designs use a variety of methods such as observation, case studies, and surveys to collect data. The data is then analyzed using statistical tools to identify patterns and relationships.

Correlational Research Design

Correlational research design is used to investigate the relationship between two or more variables. Researchers use correlational research to determine whether a relationship exists between variables and to what extent they are related. This research method involves collecting data from a sample and analyzing it using statistical tools such as correlation coefficients.

Quasi-experimental Research Design

Quasi-experimental research design is used to investigate cause-and-effect relationships between variables. This research method is similar to experimental research design, but it lacks full control over the independent variable. Researchers use quasi-experimental research designs when it is not feasible or ethical to manipulate the independent variable.

Experimental Research Design

Experimental research design is used to investigate cause-and-effect relationships between variables. This research method involves manipulating the independent variable and observing the effects on the dependent variable. Researchers use experimental research designs to test hypotheses and establish cause-and-effect relationships.

Survey Research

Survey research involves collecting data from a sample of individuals using a standardized questionnaire. This research method is used to gather information on attitudes, beliefs, and behaviors of individuals. Researchers use survey research to collect data quickly and efficiently from a large sample size. Survey research can be conducted through various methods such as online, phone, mail, or in-person interviews.

Quantitative Research Analysis Methods

Here are some commonly used quantitative research analysis methods:

Statistical Analysis

Statistical analysis is the most common quantitative research analysis method. It involves using statistical tools and techniques to analyze the numerical data collected during the research process. Statistical analysis can be used to identify patterns, trends, and relationships between variables, and to test hypotheses and theories.

Regression Analysis

Regression analysis is a statistical technique used to analyze the relationship between one dependent variable and one or more independent variables. Researchers use regression analysis to identify and quantify the impact of independent variables on the dependent variable.

Factor Analysis

Factor analysis is a statistical technique used to identify underlying factors that explain the correlations among a set of variables. Researchers use factor analysis to reduce a large number of variables to a smaller set of factors that capture the most important information.

Structural Equation Modeling

Structural equation modeling is a statistical technique used to test complex relationships between variables. It involves specifying a model that includes both observed and unobserved variables, and then using statistical methods to test the fit of the model to the data.

Time Series Analysis

Time series analysis is a statistical technique used to analyze data that is collected over time. It involves identifying patterns and trends in the data, as well as any seasonal or cyclical variations.

Multilevel Modeling

Multilevel modeling is a statistical technique used to analyze data that is nested within multiple levels. For example, researchers might use multilevel modeling to analyze data that is collected from individuals who are nested within groups, such as students nested within schools.

Applications of Quantitative Research

Quantitative research has many applications across a wide range of fields. Here are some common examples:

• Market Research : Quantitative research is used extensively in market research to understand consumer behavior, preferences, and trends. Researchers use surveys, experiments, and other quantitative methods to collect data that can inform marketing strategies, product development, and pricing decisions.
• Health Research: Quantitative research is used in health research to study the effectiveness of medical treatments, identify risk factors for diseases, and track health outcomes over time. Researchers use statistical methods to analyze data from clinical trials, surveys, and other sources to inform medical practice and policy.
• Social Science Research: Quantitative research is used in social science research to study human behavior, attitudes, and social structures. Researchers use surveys, experiments, and other quantitative methods to collect data that can inform social policies, educational programs, and community interventions.
• Education Research: Quantitative research is used in education research to study the effectiveness of teaching methods, assess student learning outcomes, and identify factors that influence student success. Researchers use experimental and quasi-experimental designs, as well as surveys and other quantitative methods, to collect and analyze data.
• Environmental Research: Quantitative research is used in environmental research to study the impact of human activities on the environment, assess the effectiveness of conservation strategies, and identify ways to reduce environmental risks. Researchers use statistical methods to analyze data from field studies, experiments, and other sources.

Characteristics of Quantitative Research

Here are some key characteristics of quantitative research:

• Numerical data : Quantitative research involves collecting numerical data through standardized methods such as surveys, experiments, and observational studies. This data is analyzed using statistical methods to identify patterns and relationships.
• Large sample size: Quantitative research often involves collecting data from a large sample of individuals or groups in order to increase the reliability and generalizability of the findings.
• Objective approach: Quantitative research aims to be objective and impartial in its approach, focusing on the collection and analysis of data rather than personal beliefs, opinions, or experiences.
• Control over variables: Quantitative research often involves manipulating variables to test hypotheses and establish cause-and-effect relationships. Researchers aim to control for extraneous variables that may impact the results.
• Replicable : Quantitative research aims to be replicable, meaning that other researchers should be able to conduct similar studies and obtain similar results using the same methods.
• Statistical analysis: Quantitative research involves using statistical tools and techniques to analyze the numerical data collected during the research process. Statistical analysis allows researchers to identify patterns, trends, and relationships between variables, and to test hypotheses and theories.
• Generalizability: Quantitative research aims to produce findings that can be generalized to larger populations beyond the specific sample studied. This is achieved through the use of random sampling methods and statistical inference.

Examples of Quantitative Research

Here are some examples of quantitative research in different fields:

• Market Research: A company conducts a survey of 1000 consumers to determine their brand awareness and preferences. The data is analyzed using statistical methods to identify trends and patterns that can inform marketing strategies.
• Health Research : A researcher conducts a randomized controlled trial to test the effectiveness of a new drug for treating a particular medical condition. The study involves collecting data from a large sample of patients and analyzing the results using statistical methods.
• Social Science Research : A sociologist conducts a survey of 500 people to study attitudes toward immigration in a particular country. The data is analyzed using statistical methods to identify factors that influence these attitudes.
• Education Research: A researcher conducts an experiment to compare the effectiveness of two different teaching methods for improving student learning outcomes. The study involves randomly assigning students to different groups and collecting data on their performance on standardized tests.
• Environmental Research : A team of researchers conduct a study to investigate the impact of climate change on the distribution and abundance of a particular species of plant or animal. The study involves collecting data on environmental factors and population sizes over time and analyzing the results using statistical methods.
• Psychology : A researcher conducts a survey of 500 college students to investigate the relationship between social media use and mental health. The data is analyzed using statistical methods to identify correlations and potential causal relationships.
• Political Science: A team of researchers conducts a study to investigate voter behavior during an election. They use survey methods to collect data on voting patterns, demographics, and political attitudes, and analyze the results using statistical methods.

How to Conduct Quantitative Research

Here is a general overview of how to conduct quantitative research:

• Develop a research question: The first step in conducting quantitative research is to develop a clear and specific research question. This question should be based on a gap in existing knowledge, and should be answerable using quantitative methods.
• Develop a research design: Once you have a research question, you will need to develop a research design. This involves deciding on the appropriate methods to collect data, such as surveys, experiments, or observational studies. You will also need to determine the appropriate sample size, data collection instruments, and data analysis techniques.
• Collect data: The next step is to collect data. This may involve administering surveys or questionnaires, conducting experiments, or gathering data from existing sources. It is important to use standardized methods to ensure that the data is reliable and valid.
• Analyze data : Once the data has been collected, it is time to analyze it. This involves using statistical methods to identify patterns, trends, and relationships between variables. Common statistical techniques include correlation analysis, regression analysis, and hypothesis testing.
• Interpret results: After analyzing the data, you will need to interpret the results. This involves identifying the key findings, determining their significance, and drawing conclusions based on the data.
• Communicate findings: Finally, you will need to communicate your findings. This may involve writing a research report, presenting at a conference, or publishing in a peer-reviewed journal. It is important to clearly communicate the research question, methods, results, and conclusions to ensure that others can understand and replicate your research.

When to use Quantitative Research

Here are some situations when quantitative research can be appropriate:

• To test a hypothesis: Quantitative research is often used to test a hypothesis or a theory. It involves collecting numerical data and using statistical analysis to determine if the data supports or refutes the hypothesis.
• To generalize findings: If you want to generalize the findings of your study to a larger population, quantitative research can be useful. This is because it allows you to collect numerical data from a representative sample of the population and use statistical analysis to make inferences about the population as a whole.
• To measure relationships between variables: If you want to measure the relationship between two or more variables, such as the relationship between age and income, or between education level and job satisfaction, quantitative research can be useful. It allows you to collect numerical data on both variables and use statistical analysis to determine the strength and direction of the relationship.
• To identify patterns or trends: Quantitative research can be useful for identifying patterns or trends in data. For example, you can use quantitative research to identify trends in consumer behavior or to identify patterns in stock market data.
• To quantify attitudes or opinions : If you want to measure attitudes or opinions on a particular topic, quantitative research can be useful. It allows you to collect numerical data using surveys or questionnaires and analyze the data using statistical methods to determine the prevalence of certain attitudes or opinions.

Purpose of Quantitative Research

The purpose of quantitative research is to systematically investigate and measure the relationships between variables or phenomena using numerical data and statistical analysis. The main objectives of quantitative research include:

• Description : To provide a detailed and accurate description of a particular phenomenon or population.
• Explanation : To explain the reasons for the occurrence of a particular phenomenon, such as identifying the factors that influence a behavior or attitude.
• Prediction : To predict future trends or behaviors based on past patterns and relationships between variables.
• Control : To identify the best strategies for controlling or influencing a particular outcome or behavior.

Quantitative research is used in many different fields, including social sciences, business, engineering, and health sciences. It can be used to investigate a wide range of phenomena, from human behavior and attitudes to physical and biological processes. The purpose of quantitative research is to provide reliable and valid data that can be used to inform decision-making and improve understanding of the world around us.

Advantages of Quantitative Research

There are several advantages of quantitative research, including:

• Objectivity : Quantitative research is based on objective data and statistical analysis, which reduces the potential for bias or subjectivity in the research process.
• Reproducibility : Because quantitative research involves standardized methods and measurements, it is more likely to be reproducible and reliable.
• Generalizability : Quantitative research allows for generalizations to be made about a population based on a representative sample, which can inform decision-making and policy development.
• Precision : Quantitative research allows for precise measurement and analysis of data, which can provide a more accurate understanding of phenomena and relationships between variables.
• Efficiency : Quantitative research can be conducted relatively quickly and efficiently, especially when compared to qualitative research, which may involve lengthy data collection and analysis.
• Large sample sizes : Quantitative research can accommodate large sample sizes, which can increase the representativeness and generalizability of the results.

Limitations of Quantitative Research

There are several limitations of quantitative research, including:

• Limited understanding of context: Quantitative research typically focuses on numerical data and statistical analysis, which may not provide a comprehensive understanding of the context or underlying factors that influence a phenomenon.
• Simplification of complex phenomena: Quantitative research often involves simplifying complex phenomena into measurable variables, which may not capture the full complexity of the phenomenon being studied.
• Potential for researcher bias: Although quantitative research aims to be objective, there is still the potential for researcher bias in areas such as sampling, data collection, and data analysis.
• Limited ability to explore new ideas: Quantitative research is often based on pre-determined research questions and hypotheses, which may limit the ability to explore new ideas or unexpected findings.
• Limited ability to capture subjective experiences : Quantitative research is typically focused on objective data and may not capture the subjective experiences of individuals or groups being studied.
• Ethical concerns : Quantitative research may raise ethical concerns, such as invasion of privacy or the potential for harm to participants.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Exploratory Research – Types, Methods and...

Case Study – Methods, Examples and Guide

Questionnaire – Definition, Types, and Examples

Qualitative Research – Methods, Analysis Types...

One-to-One Interview – Methods and Guide

Focus Groups – Steps, Examples and Guide

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• Qualitative vs. Quantitative Research | Differences, Examples & Methods

Qualitative vs. Quantitative Research | Differences, Examples & Methods

Published on April 12, 2019 by Raimo Streefkerk . Revised on June 22, 2023.

When collecting and analyzing data, quantitative research deals with numbers and statistics, while qualitative research deals with words and meanings. Both are important for gaining different kinds of knowledge.

Common quantitative methods include experiments, observations recorded as numbers, and surveys with closed-ended questions.

Quantitative research is at risk for research biases including information bias , omitted variable bias , sampling bias , or selection bias . Qualitative research Qualitative research is expressed in words . It is used to understand concepts, thoughts or experiences. This type of research enables you to gather in-depth insights on topics that are not well understood.

Common qualitative methods include interviews with open-ended questions, observations described in words, and literature reviews that explore concepts and theories.

Table of contents

The differences between quantitative and qualitative research, data collection methods, when to use qualitative vs. quantitative research, how to analyze qualitative and quantitative data, other interesting articles, frequently asked questions about qualitative and quantitative research.

Quantitative and qualitative research use different research methods to collect and analyze data, and they allow you to answer different kinds of research questions.

Quantitative and qualitative data can be collected using various methods. It is important to use a data collection method that will help answer your research question(s).

Many data collection methods can be either qualitative or quantitative. For example, in surveys, observational studies or case studies , your data can be represented as numbers (e.g., using rating scales or counting frequencies) or as words (e.g., with open-ended questions or descriptions of what you observe).

However, some methods are more commonly used in one type or the other.

Quantitative data collection methods

• Surveys :  List of closed or multiple choice questions that is distributed to a sample (online, in person, or over the phone).
• Experiments : Situation in which different types of variables are controlled and manipulated to establish cause-and-effect relationships.
• Observations : Observing subjects in a natural environment where variables can’t be controlled.

Qualitative data collection methods

• Interviews : Asking open-ended questions verbally to respondents.
• Focus groups : Discussion among a group of people about a topic to gather opinions that can be used for further research.
• Ethnography : Participating in a community or organization for an extended period of time to closely observe culture and behavior.
• Literature review : Survey of published works by other authors.

A rule of thumb for deciding whether to use qualitative or quantitative data is:

• Use quantitative research if you want to confirm or test something (a theory or hypothesis )
• Use qualitative research if you want to understand something (concepts, thoughts, experiences)

For most research topics you can choose a qualitative, quantitative or mixed methods approach . Which type you choose depends on, among other things, whether you’re taking an inductive vs. deductive research approach ; your research question(s) ; whether you’re doing experimental , correlational , or descriptive research ; and practical considerations such as time, money, availability of data, and access to respondents.

Quantitative research approach

You survey 300 students at your university and ask them questions such as: “on a scale from 1-5, how satisfied are your with your professors?”

You can perform statistical analysis on the data and draw conclusions such as: “on average students rated their professors 4.4”.

Qualitative research approach

You conduct in-depth interviews with 15 students and ask them open-ended questions such as: “How satisfied are you with your studies?”, “What is the most positive aspect of your study program?” and “What can be done to improve the study program?”

Based on the answers you get you can ask follow-up questions to clarify things. You transcribe all interviews using transcription software and try to find commonalities and patterns.

Mixed methods approach

You conduct interviews to find out how satisfied students are with their studies. Through open-ended questions you learn things you never thought about before and gain new insights. Later, you use a survey to test these insights on a larger scale.

It’s also possible to start with a survey to find out the overall trends, followed by interviews to better understand the reasons behind the trends.

Qualitative or quantitative data by itself can’t prove or demonstrate anything, but has to be analyzed to show its meaning in relation to the research questions. The method of analysis differs for each type of data.

Analyzing quantitative data

Quantitative data is based on numbers. Simple math or more advanced statistical analysis is used to discover commonalities or patterns in the data. The results are often reported in graphs and tables.

Applications such as Excel, SPSS, or R can be used to calculate things like:

• Average scores ( means )
• The number of times a particular answer was given
• The correlation or causation between two or more variables
• The reliability and validity of the results

Analyzing qualitative data

Qualitative data is more difficult to analyze than quantitative data. It consists of text, images or videos instead of numbers.

Some common approaches to analyzing qualitative data include:

• Qualitative content analysis : Tracking the occurrence, position and meaning of words or phrases
• Thematic analysis : Closely examining the data to identify the main themes and patterns
• Discourse analysis : Studying how communication works in social contexts

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Chi square goodness of fit test
• Degrees of freedom
• Null hypothesis
• Discourse analysis
• Control groups
• Mixed methods research
• Non-probability sampling
• Quantitative research
• Inclusion and exclusion criteria

Research bias

• Rosenthal effect
• Implicit bias
• Cognitive bias
• Selection bias
• Negativity bias
• Status quo bias

Quantitative research deals with numbers and statistics, while qualitative research deals with words and meanings.

Quantitative methods allow you to systematically measure variables and test hypotheses . Qualitative methods allow you to explore concepts and experiences in more detail.

In mixed methods research , you use both qualitative and quantitative data collection and analysis methods to answer your research question .

The research methods you use depend on the type of data you need to answer your research question .

• If you want to measure something or test a hypothesis , use quantitative methods . If you want to explore ideas, thoughts and meanings, use qualitative methods .
• If you want to analyze a large amount of readily-available data, use secondary data. If you want data specific to your purposes with control over how it is generated, collect primary data.
• If you want to establish cause-and-effect relationships between variables , use experimental methods. If you want to understand the characteristics of a research subject, use descriptive methods.

Data collection is the systematic process by which observations or measurements are gathered in research. It is used in many different contexts by academics, governments, businesses, and other organizations.

There are various approaches to qualitative data analysis , but they all share five steps in common:

• Prepare and organize your data.
• Review and explore your data.
• Develop a data coding system.
• Assign codes to the data.
• Identify recurring themes.

The specifics of each step depend on the focus of the analysis. Some common approaches include textual analysis , thematic analysis , and discourse analysis .

A research project is an academic, scientific, or professional undertaking to answer a research question . Research projects can take many forms, such as qualitative or quantitative , descriptive , longitudinal , experimental , or correlational . What kind of research approach you choose will depend on your topic.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Streefkerk, R. (2023, June 22). Qualitative vs. Quantitative Research | Differences, Examples & Methods. Scribbr. Retrieved June 26, 2024, from https://www.scribbr.com/methodology/qualitative-quantitative-research/

Is this article helpful?

Raimo Streefkerk

Other students also liked, what is quantitative research | definition, uses & methods, what is qualitative research | methods & examples, mixed methods research | definition, guide & examples, what is your plagiarism score.

Educational resources and simple solutions for your research journey

What is a Theoretical Framework? How to Write It (with Examples) 

Theoretical framework 1,2 is the structure that supports and describes a theory. A theory is a set of interrelated concepts and definitions that present a systematic view of phenomena by describing the relationship among the variables for explaining these phenomena. A theory is developed after a long research process and explains the existence of a research problem in a study. A theoretical framework guides the research process like a roadmap for the research study and helps researchers clearly interpret their findings by providing a structure for organizing data and developing conclusions.   

A theoretical framework in research is an important part of a manuscript and should be presented in the first section. It shows an understanding of the theories and concepts relevant to the research and helps limit the scope of the research.  

Table of Contents

What is a theoretical framework ?  

A theoretical framework in research can be defined as a set of concepts, theories, ideas, and assumptions that help you understand a specific phenomenon or problem. It can be considered a blueprint that is borrowed by researchers to develop their own research inquiry. A theoretical framework in research helps researchers design and conduct their research and analyze and interpret their findings. It explains the relationship between variables, identifies gaps in existing knowledge, and guides the development of research questions, hypotheses, and methodologies to address that gap.  

Now that you know the answer to ‘ What is a theoretical framework? ’, check the following table that lists the different types of theoretical frameworks in research: 3

Developing a theoretical framework in research can help in the following situations: 4

• When conducting research on complex phenomena because a theoretical framework helps organize the research questions, hypotheses, and findings  
• When the research problem requires a deeper understanding of the underlying concepts  
• When conducting research that seeks to address a specific gap in knowledge  
• When conducting research that involves the analysis of existing theories  

Summarizing existing literature for theoretical frameworks is easy. Get our Research Ideation pack  

Importance of a theoretical framework  

The purpose of theoretical framework s is to support you in the following ways during the research process: 2  

• Provide a structure for the complete research process  
• Assist researchers in incorporating formal theories into their study as a guide  
• Provide a broad guideline to maintain the research focus  
• Guide the selection of research methods, data collection, and data analysis  
• Help understand the relationships between different concepts and develop hypotheses and research questions  
• Address gaps in existing literature  
• Analyze the data collected and draw meaningful conclusions and make the findings more generalizable  

Theoretical vs. Conceptual framework  

While a theoretical framework covers the theoretical aspect of your study, that is, the various theories that can guide your research, a conceptual framework defines the variables for your study and presents how they relate to each other. The conceptual framework is developed before collecting the data. However, both frameworks help in understanding the research problem and guide the development, collection, and analysis of the research.  

The following table lists some differences between conceptual and theoretical frameworks . 5

How to write a theoretical framework  

The following general steps can help those wondering how to write a theoretical framework: 2

• Identify and define the key concepts clearly and organize them into a suitable structure.  
• Use appropriate terminology and define all key terms to ensure consistency.  
• Identify the relationships between concepts and provide a logical and coherent structure.  
• Develop hypotheses that can be tested through data collection and analysis.  
• Keep it concise and focused with clear and specific aims.  

Write a theoretical framework 2x faster. Get our Manuscript Writing pack  

Examples of a theoretical framework  

Here are two examples of a theoretical framework. 6,7

Example 1 .   

An insurance company is facing a challenge cross-selling its products. The sales department indicates that most customers have just one policy, although the company offers over 10 unique policies. The company would want its customers to purchase more than one policy since most customers are purchasing policies from other companies.  

Objective : To sell more insurance products to existing customers.  

Problem : Many customers are purchasing additional policies from other companies.  

Research question : How can customer product awareness be improved to increase cross-selling of insurance products?  

Sub-questions: What is the relationship between product awareness and sales? Which factors determine product awareness?  

Since “product awareness” is the main focus in this study, the theoretical framework should analyze this concept and study previous literature on this subject and propose theories that discuss the relationship between product awareness and its improvement in sales of other products.  

Example 2 .

A company is facing a continued decline in its sales and profitability. The main reason for the decline in the profitability is poor services, which have resulted in a high level of dissatisfaction among customers and consequently a decline in customer loyalty. The management is planning to concentrate on clients’ satisfaction and customer loyalty.  

Objective: To provide better service to customers and increase customer loyalty and satisfaction.  

Problem: Continued decrease in sales and profitability.  

Research question: How can customer satisfaction help in increasing sales and profitability?  

Sub-questions: What is the relationship between customer loyalty and sales? Which factors influence the level of satisfaction gained by customers?  

Since customer satisfaction, loyalty, profitability, and sales are the important topics in this example, the theoretical framework should focus on these concepts.  

Benefits of a theoretical framework  

There are several benefits of a theoretical framework in research: 2  

• Provides a structured approach allowing researchers to organize their thoughts in a coherent way.  
• Helps to identify gaps in knowledge highlighting areas where further research is needed.  
• Increases research efficiency by providing a clear direction for research and focusing efforts on relevant data.  
• Improves the quality of research by providing a rigorous and systematic approach to research, which can increase the likelihood of producing valid and reliable results.  
• Provides a basis for comparison by providing a common language and conceptual framework for researchers to compare their findings with other research in the field, facilitating the exchange of ideas and the development of new knowledge.  

Frequently Asked Questions 

Q1. How do I develop a theoretical framework ? 7

A1. The following steps can be used for developing a theoretical framework :  

• Identify the research problem and research questions by clearly defining the problem that the research aims to address and identifying the specific questions that the research aims to answer.
• Review the existing literature to identify the key concepts that have been studied previously. These concepts should be clearly defined and organized into a structure.
• Develop propositions that describe the relationships between the concepts. These propositions should be based on the existing literature and should be testable.
• Develop hypotheses that can be tested through data collection and analysis.
• Test the theoretical framework through data collection and analysis to determine whether the framework is valid and reliable.

Q2. How do I know if I have developed a good theoretical framework or not? 8

A2. The following checklist could help you answer this question:  

• Is my theoretical framework clearly seen as emerging from my literature review?  
• Is it the result of my analysis of the main theories previously studied in my same research field?  
• Does it represent or is it relevant to the most current state of theoretical knowledge on my topic?  
• Does the theoretical framework in research present a logical, coherent, and analytical structure that will support my data analysis?  
• Do the different parts of the theory help analyze the relationships among the variables in my research?  
• Does the theoretical framework target how I will answer my research questions or test the hypotheses?  
• Have I documented every source I have used in developing this theoretical framework ?  
• Is my theoretical framework a model, a table, a figure, or a description?  
• Have I explained why this is the appropriate theoretical framework for my data analysis?  

Q3. Can I use multiple theoretical frameworks in a single study?  

A3. Using multiple theoretical frameworks in a single study is acceptable as long as each theory is clearly defined and related to the study. Each theory should also be discussed individually. This approach may, however, be tedious and effort intensive. Therefore, multiple theoretical frameworks should be used only if absolutely necessary for the study.  

Q4. Is it necessary to include a theoretical framework in every research study?  

A4. The theoretical framework connects researchers to existing knowledge. So, including a theoretical framework would help researchers get a clear idea about the research process and help structure their study effectively by clearly defining an objective, a research problem, and a research question.  

Q5. Can a theoretical framework be developed for qualitative research?  

A5. Yes, a theoretical framework can be developed for qualitative research. However, qualitative research methods may or may not involve a theory developed beforehand. In these studies, a theoretical framework can guide the study and help develop a theory during the data analysis phase. This resulting framework uses inductive reasoning. The outcome of this inductive approach can be referred to as an emergent theoretical framework . This method helps researchers develop a theory inductively, which explains a phenomenon without a guiding framework at the outset.  

Q6. What is the main difference between a literature review and a theoretical framework ?  

A6. A literature review explores already existing studies about a specific topic in order to highlight a gap, which becomes the focus of the current research study. A theoretical framework can be considered the next step in the process, in which the researcher plans a specific conceptual and analytical approach to address the identified gap in the research.  

Theoretical frameworks are thus important components of the research process and researchers should therefore devote ample amount of time to develop a solid theoretical framework so that it can effectively guide their research in a suitable direction. We hope this article has provided a good insight into the concept of theoretical frameworks in research and their benefits.  

References  

• Organizing academic research papers: Theoretical framework. Sacred Heart University library. Accessed August 4, 2023. https://library.sacredheart.edu/c.php?g=29803&p=185919#:~:text=The%20theoretical%20framework%20is%20the,research%20problem%20under%20study%20exists .  
• Salomao A. Understanding what is theoretical framework. Mind the Graph website. Accessed August 5, 2023. https://mindthegraph.com/blog/what-is-theoretical-framework/  
• Theoretical framework—Types, examples, and writing guide. Research Method website. Accessed August 6, 2023. https://researchmethod.net/theoretical-framework/  
• Grant C., Osanloo A. Understanding, selecting, and integrating a theoretical framework in dissertation research: Creating the blueprint for your “house.” Administrative Issues Journal : Connecting Education, Practice, and Research; 4(2):12-26. 2014. Accessed August 7, 2023. https://files.eric.ed.gov/fulltext/EJ1058505.pdf  
• Difference between conceptual framework and theoretical framework. MIM Learnovate website. Accessed August 7, 2023. https://mimlearnovate.com/difference-between-conceptual-framework-and-theoretical-framework/  
• Example of a theoretical framework—Thesis & dissertation. BacherlorPrint website. Accessed August 6, 2023. https://www.bachelorprint.com/dissertation/example-of-a-theoretical-framework/  
• Sample theoretical framework in dissertation and thesis—Overview and example. Students assignment help website. Accessed August 6, 2023. https://www.studentsassignmenthelp.co.uk/blogs/sample-dissertation-theoretical-framework/#Example_of_the_theoretical_framework  
• Kivunja C. Distinguishing between theory, theoretical framework, and conceptual framework: A systematic review of lessons from the field. Accessed August 8, 2023. https://files.eric.ed.gov/fulltext/EJ1198682.pdf  

Researcher.Life is a subscription-based platform that unifies the best AI tools and services designed to speed up, simplify, and streamline every step of a researcher’s journey. The Researcher.Life All Access Pack is a one-of-a-kind subscription that unlocks full access to an AI writing assistant, literature recommender, journal finder, scientific illustration tool, and exclusive discounts on professional publication services from Editage.  

Based on 21+ years of experience in academia, Researcher.Life All Access empowers researchers to put their best research forward and move closer to success. Explore our top AI Tools pack, AI Tools + Publication Services pack, or Build Your Own Plan. Find everything a researcher needs to succeed, all in one place –  Get All Access now starting at just $17 a month !    

Related Posts

Thesis Defense: How to Ace this Crucial Step

What is Independent Publishing in Academia?

• Open access
• Published: 25 June 2024

Achieving research impact in medical research through collaboration across organizational boundaries: Insights from a mixed methods study in the Netherlands

• Jacqueline C. F. van Oijen   ORCID: orcid.org/0000-0002-5100-0671 1 ,
• Annemieke van Dongen-Leunis 1 ,
• Jeroen Postma 1 ,
• Thed van Leeuwen 2 &
• Roland Bal 1  

Health Research Policy and Systems volume  22 , Article number:  72 ( 2024 ) Cite this article

Metrics details

In the Netherlands, university medical centres (UMCs) bear primary responsibility for conducting medical research and delivering highly specialized care. The TopCare program was a policy experiment lasting 4 years in which three non-academic hospitals received funding from the Dutch Ministry of Health to also conduct medical research and deliver highly specialized care in specific domains. This study investigates research collaboration outcomes for all Dutch UMCs and non-academic hospitals in general and, more specifically, for the domains in the non-academic hospitals participating in the TopCare program. Additionally, it explores the organizational boundary work employed by these hospitals to foster productive research collaborations.

A mixed method research design was employed combining quantitative bibliometric analysis of publications and citations across all Dutch UMCs and non-academic hospitals and the TopCare domains with geographical distances, document analysis and ethnographic interviews with actors in the TopCare program.

Quantitative analysis shows that, over the period of study, international collaboration increased among all hospitals while national collaboration and single institution research declined slightly. Collaborative efforts correlated with higher impact scores, and international collaboration scored higher than national collaboration. A total of 60% of all non-academic hospitals’ publications were produced in collaboration with UMCs, whereas almost 30% of the UMCs’ publications were the result of such collaboration. Non-academic hospitals showed a higher rate of collaboration with the UMC that was nearest geographically, whereas TopCare hospitals prioritized expertise over geographical proximity within their specialized domains. Boundary work mechanisms adopted by TopCare hospitals included aligning research activities with organizational mindset (identity), bolstering research infrastructure (competence) and finding and mobilizing strategic partnerships with academic partners (power). These efforts aimed to establish credibility and attractiveness as collaboration partners.

Conclusions

Research collaboration between non-academic hospitals and UMCs, particularly where this also involves international collaboration, pays off in terms of publications and impact. The TopCare hospitals used the program’s resources to perform boundary work aimed at becoming an attractive and credible collaboration partner for academia. Local factors such as research history, strategic domain focus, in-house expertise, patient flows, infrastructure and network relationships influenced collaboration dynamics within TopCare hospitals and between them and UMCs.

Peer Review reports

Introduction

Research collaboration has taken flight worldwide in recent decades [ 1 ], as reflected by the growing number of authors listed on research papers [ 2 , 3 ]. Collaborative research has become the norm for many, if not most, scientific disciplines [ 4 , 5 , 6 , 7 , 8 ]. Several studies have found a positive relationship between collaboration and output [ 9 , 10 , 11 , 12 , 13 ]. Publications resulting from research collaborations tend to be cited more frequently [ 14 , 15 , 16 , 17 , 18 ] and to be of higher research quality [ 5 , 14 , 19 , 20 ]. In particular, international collaboration can lead to more citations [ 17 , 21 , 22 , 23 , 24 ], although there are major differences internationally and between fields [ 25 ]. Moreover, international collaboration is often set as an eligibility requirement for European research grants, which have become necessary as national-level resources dwindle. Funding consortia also encourage and require boundary crossings, such as research collaborations between academia and societal partners. Collaboration within public research organizations and universities further plays a crucial role in the international dissemination of knowledge [ 26 ].

In the medical domain, initiatives have been rolled out in numerous countries to encourage long-term collaboration and the exchange of knowledge and research findings. Each initiative takes a strategic approach to assembling the processes needed to support these exchanges across the boundaries of stakeholder groups. In the Netherlands, medical research has traditionally been concentrated in public academia, especially the university medical centres (UMCs). Increasingly, however, research activities are being undertaken in non-academic teaching hospitals (hereafter, non-academic hospitals), driven by their changing patterns of patient influx. In 2013, a Dutch study based on citation analysis showed that collaboration between UMCs and non-academic hospitals leads to high-quality research [ 27 ]. There was further encouragement for medical research in Dutch non-academic hospitals in 2014, when a 4-year policy experiment, the TopCare program, was launched, with three such hospitals receiving additional funding from the Ministry of Health to also provide highly specialized care and undertake medical research. Funding for this combination of care and research is available for UMCs under the budgetary “academic component” of the Dutch healthcare system. Such additional funds are not available for non-academic hospitals, nor can they allocate their regular budgets to research. In the past, these hospitals managed to conduct research and provide specialized care through their own financial and time investments, or by securing occasional external research funding. The TopCare policy experiment was thus meant to find new ways of organizing and funding highly specialized care and medical research in non-academic hospitals.

Despite the increasing emphasis on research collaboration, we still know little about its impact and how it can be achieved. This study integrates two sides of research collaboration in Dutch hospitals and combines elements of quantitative and qualitative research for a broad (output and impact) and deep (boundary work to achieve collaboration) understanding of the phenomenon. We define research collaboration as collaboration between two or more organizations (at least one being a UMC or non-academic hospital) that has resulted in a co-authored (joint) scientific publication [ 28 ]. The research questions are: How high is the level of collaboration in the Dutch medical research field, what is the impact of collaboration, and how are productive research collaborations achieved?

To answer these questions, we performed mixed methods research in UMCs and non-academic hospitals. To examine the impact of various collaboration models – namely, single institution, national and international – across all eight Dutch UMCs and 28 non-academic hospitals between 2009 and 2018/2019, we conducted a bibliometric analysis of publications and citations. We additionally carried out a similar analysis for the TopCare non-academic hospitals between 2010 and 2016 to examine the effects of collaboration in the two domains funded through the program at each hospital. The latter timeframe was chosen to match the duration of the program, 2014–2018. We further conducted an in-depth qualitative analysis of the organizational boundary work done by two non-academic hospitals participating in the TopCare program to initiate and enhance productive research collaborations around specialized research and care within and between hospitals on a national level. Historically, such endeavours have been predominantly reserved for UMCs. The program was therefore a unique opportunity to examine such boundary work.

Background and theory

The landscape of medical research in the netherlands, collaboration in medical research.

The Netherlands has a three-tiered hospital system: general hospitals (including non-academic hospitals), specialized hospitals focusing on a specific medical field or patient population, and UMCs. Nowadays, there are 7 UMCs, 17 specialized hospitals and 58 general hospitals, of which 26 are non-academic [ 29 ].

UMCs receive special funding (the budgetary “academic component”) for research and oversee medical training programs in their region. Non-academic hospitals do not receive structural government funding for medical research and have less chance of obtaining other funding because they are not formally acknowledged as knowledge-producing organizations. Research has less priority in most of these hospitals than in UMCs. On the introduction of government policies regarding competition in healthcare and the development of quality guidelines emphasizing high-volume treatments, some non-academic hospitals began focusing on specific disease areas, in a bid to distinguish themselves from other hospitals and to perform research in and hence develop more knowledge about these priority areas. This led to a greater concentration of highly specialized care [ 30 ]. Non-academic hospitals have also become important partners in medical research for UMCs due to their large patient volumes.

The TopCare program

To further stimulate research in non-academic hospitals, the Ministry of Health awarded three such hospitals €28.8 million in funding over a 4-year period (2014–2018) to support medical research and specialized care for which they do not normally receive funding [ 31 ]. It should be noted that, in non-academic hospitals, the concept of highly specialized research and care applies not to the entire hospital but rather to specific departments or disease areas. This is why the TopCare non-academic hospitals have been evaluated on the basis of specific domains. The funding recipients were two non-academic hospitals and one specialized hospital. In this article, we focus on UMCs and general non-academic hospitals and therefore excluded the specialized hospital from our analysis. Hospital #1 is the largest non-academic hospital in the Netherlands (1100 beds), even larger than some UMCs. Its fields of excellence (known as “domains”) are lung and heart care. Hospital #2 is a large non-academic hospital (950 beds) that focuses on emergency care and neurology. According to the two hospitals, these four highly specialized care and research-intensive domains are comparable to high-complexity care and research in UMCs [ 31 ].

The TopCare program ran through ZonMw, the Netherlands Organization for Health Research and Development, the main funding body for health research in the Netherlands. ZonMw established a committee to assess the research proposals and complex care initiatives of the participating hospitals and to set several criteria for funding eligibility. One requirement was that participating hospitals had to collaborate with universities or UMCs on research projects and were not allowed to conduct basic research in the context of the program, as this was seen as the special province of UMCs.

Boundary work

In the qualitative part of this study, we analyse the boundary work done by actors to influence organizational boundaries as well as the practices undertaken to initiate or enhance collaboration between TopCare non-academic hospitals and academia (universities and UMCs). We refer to boundary work when actors create, shape or disrupt organizational boundaries [ 32 , 33 , 34 , 35 ]. In particular, boundary work involves opening a boundary for collaboration and creating linkages with external partners [ 36 ]. In this article, we use three organizational boundary concepts – “identity”, “competence” and “power” – out of four presented by Santos and Eisenhardt. These concepts are concerned with fostering collaboration, whereas the fourth is concerned with “efficiency” and is less relevant here. Identity involves creating a reputation for research to become an attractive partner while preserving identity. Competence involves creating opportunities for research, for example, in manpower and infrastructure. Finally, power involves creating a negotiating position vis-à-vis relevant others [ 35 ].

The data for this study consist of different types of analysis: (1) quantitative bibliometric data on the publications and citations of all eight Dutch UMCs and 28 non-academic hospitals, and (2) quantitative bibliometric data on the publications and citations in the four domains of two TopCare non-academic hospitals, qualitative (policy) document analysis and in-depth ethnographic interviews with various actors in the Dutch TopCare program. The quantitative data collected from Dutch UMCs and non-academic hospitals were utilized to contextualize data gathered within the TopCare program. We discuss and explain the data collection and methodology in detail in the two sections below.

Quantitative approach: bibliometric analysis of all 8 Dutch UMCs and 28 non-academic hospitals

Data collection

We performed a bibliometric analysis of the publications of 28 non-academic hospitals and 8 UMCs Footnote 1 in the Netherlands between 2009 and 2018. Data for the study were derived from the Center for Science and Technology Studies’ (CWTS) in-house version of the Web of Science (WoS) database. The year 2009 was chosen because the address affiliations in publications are more accurately defined from this year onward. To examine trends over time, we divided the period 2009–2018/2019 into two blocks of 4 years and an additional year for citation impact measurement (2009–2012/2013 and 2014–2017/2018; see explanation in Appendix 1).

Methodology

The bibliometric analysis includes several bibliometric indicators that describe both the output and impact of the relevant research (Table  5 in Appendix 1). One of the indicators, the mean normalized citation score (MNCS), reveals the average impact of a hospital’s publications compared with the average score of all other publications in that area of research. If the MNCS is higher than 1, then on average, the output of that hospital’s domain is cited more often than an “average” publication in that research area.

To map the ways hospitals cooperate, we follow two lines of analysis. The first is centred around a typology of scientific activities and differentiates between (i) a single institution (SI;  all publications with only one address) and (ii) international collaboration (IC; collaboration with at least one international partner). All other publications are grouped as (iii) national collaboration (NC; collaboration with Dutch organizations only).

The second line is centred around geographical distance and size of collaboration. The geographical distances between each non-academic hospital and each of the eight UMCs were measured in Google Maps. The size of collaboration was measured by counting the joint publications of each non-academic hospital and the eight UMCs. Subsequently, we assessed whether the non-academic hospitals also had the most joint publications with the nearest UMC.

Quantitative and qualitative approach to the two TopCare hospitals and their four domains, the “TopCare program” case study

Quantitative approach

The quantitative approach to the TopCare program relies on a bibliometric analysis of publications within each hospital’s two domains: lung and heart care in TopCare non-academic hospital #1, and trauma and neurology in TopCare non-academic hospital #2. Our bibliometric analysis focused on publications within the four selected TopCare domains between 2010 and 2016, following the same methodology described in the previous section under ‘Data collection’. Each domain provided an overview of its publications. The number of publications produced by the two domains at each TopCare hospital is combined in the results. Although this timeframe differs from the broader analysis of all UMCs and non-academic hospitals, comparing these two periods offers insights into the “representative position” of the two domains of each non-academic hospital participating in the TopCare program, in terms of publications and citations.

Qualitative approach

We took a qualitative approach to analysing the collaborative activities in the two TopCare non-academic hospitals, where each domain has its own leadership arrangements, regional demographic priorities and history of research collaboration [cf. 37 ]. This part of the study consisted of interviews and document analysis.

Ethnographic interviews

Over the course of the 4-year program, J.P. and/or R.B. conducted and recorded 90 semi-structured interviews that were then transcribed. For this study, we used repeated in-depth ethnographic interviews with the main actors in the Dutch TopCare program, which took place between 2014 and 2018. We conducted a total of 27 interviews; 20 of the interviews were with a single person, 5 with two persons, and 2 with three persons. The interviews were held with 20 different respondents; 12 respondents were interviewed multiple times. Table 1 presents the different respondents in non-academic hospitals #1 and #2.

Document analysis

Desk research was performed for documents related to the TopCare program (Table  6 – details of document analysis in Appendix 1).

The bibliometric analysis of the four domains in the two TopCare non-academic hospitals follows the same methodology as described in Abramo et al. [ 1 ].

We tested the assumption that joint publications are most frequent between a non-academic hospital and its nearest UMC. If the geographical distance between TopCare non-academic hospitals and their collaborative academic partners is described as “nearby”, then they both work within the same region.

The ethnographic interviews were audio-recorded and transcribed in full with the respondents’ permission. These transcripts were subject to close reading and coding by two authors, J.P. and J.O., to identify key themes derived from the theory [ 35 ] (Table  7 in the Appendix). These were then discussed and debated with the wider research team with the goal of developing a critical interpretation of the boundary work done to initiate or enhance research collaboration [cf. 37 ]. The processed interview data were submitted to the respondents for member check. All respondents gave permission to use the data for this study, including the specific quotes. In the Netherlands, this research requires no ethical approval.

Triangulating the results of the document analysis and the interviews enables us to identify different overarching themes within each boundary concept (identity, competence and power). These themes were utilized as a framework for structuring individual paragraphs, which we explain in greater detail in Table  4 in the Results.

Bibliometric analysis of all Dutch UMCs and non-academic hospitals

This section reports the results of the quantitative bibliometric analysis of the output, trends and impact of collaboration between all UMCs and non-academic hospitals from 2009 to 2018/2019. It provides a broad picture of the output – in terms of research publications – of both existing and ongoing collaborations between all UMCs and non-academic hospitals within the specified timeframe. It furthermore describes the analysis results concerning the relationship between collaboration and the geographical distance between two collaborating hospitals.

Output: distribution of the types of collaboration for UMCs and non-academic hospitals from 2009 to 2018/2019

The first step in understanding the degree of collaboration between hospitals is to measure the research output by number of publications. The total number of publications between 2009 and 2018 is shown in Table  8 ( Appendix 1) and Fig.  1 .

Types of collaboration for UMCs and non-academic hospitals from 2009 to 2018/2019. # Total number of publications. Percentage of total (100%) accounted for by single institution, national collaboration and international collaboration

The majority of these publications (89%) are affiliated with UMCs. UMCs, in particular, tend to have a relatively higher proportion of single-institution publications and are more engaged in international collaboration. This pattern may be indicative of UMCs’ enhanced access to research grants and EU subsidies, as well as their active involvement in international consortia.

Collaboration between UMCs and non-academic hospitals appears to be more prevalent and impactful for non-academic hospitals than for UMCs: 70% of all publications originating from a non-academic hospital were the result of joint efforts between a UMC and a non-academic hospital, whereas only 8% of all UMC publications were produced in collaboration with a non-academic hospital (Table  8 in Appendix 1).

Trend analysis of collaboration in relative number of publications

Table  9 Appendix 1) and Fig.  2 show the relative number of publications of all 8 UMCs and all 28 non-academic hospitals in the two periods: 2009–2012/2013 and 2014–2017/2018. For both UMCs and non-academic hospitals, international collaboration accounted for a relatively larger share of publications in recent years.

Type of research collaboration for UMCs and non-academic hospitals over time. Percentage of total (100%) accounted for by single institution, national collaboration and international collaboration in each period

Analysis of relationship between distance and collaboration

As the non-academic hospitals often collaborate with UMCs, it is interesting to analyse these collaborations geographically (distance). The assumption is that geographical proximity matters, with the most-frequent joint publications being between a non-academic hospital and the nearest UMC.

Figure  3 shows that 61% (17 out of 28) of the non-academic hospitals collaborate most frequently with the nearest UMCs. Geographical proximity is thus an important but not the only determining factor in collaboration.

Collaboration with nearest UMC from 2009 to 2018

Impact of collaboration on bibliometric output of UMCs and non-academic hospitals

The mean normalized citation scores (MNCS) shown in Table  2 cover all 8 UMCs and 28 non-academic hospitals.

The MNCS in Table  2 and the mean normalized journal scores (MNJS) in Table  10 (Appendix 1) show similar patterns. The impact score for both UMCs and non-academic hospitals is greatest for international collaboration. Non-academic hospitals’ single-institution publications score lower than the global average, which was defined as 1.

In sum, quantitative analysis exposes two trends. The first is growth in international collaboration for all UMCs and non-academic hospitals over time, also revealing that collaboration leads to higher MNCS impact scores. Second, geographical proximity between UMCs and non-academic hospitals is an important but not the only determining factor in collaboration. This is the context in which the TopCare program operated in 2014–2018.

“TopCare program” case study

This section presents the results of our analysis of the collaboration networks of the two TopCare non-academic hospitals, consisting of: (1) quantitative bibliometric analysis of the output and impact of these networks between 2010 and 2016, along with the geographical distance to their academic partners, and (2) qualitative ethnographic interviews to identify the boundary work conducted by these hospitals.

Bibliometric analysis of the two TopCare non-academic hospitals’ international and national collaboration networks across four domains

The results of the bibliometric analysis indicate the representative positions of the two domains within each TopCare non-academic hospital. Between 2010 and 2016, these hospitals generated a higher number of single-institution publications compared with the average of all non-academic hospitals. Percentage-wise, their output resembled that of the UMCs, underscoring their leading positions in their respective domains. The percentage of publications based on national collaboration in the domains of TopCare hospital #2 is comparable to that of non-academic hospitals overall, while there is more international collaboration in the domains of TopCare hospital #1 than at non-academic hospitals overall (Fig.  4 , Appendix 1 and Fig.  1 ). The impact of the research is above the global average, and the publications have a higher average impact when there is collaboration with international partners; this is true across all four domains (Table  11 in Appendix 1).

In terms of geographical distance, only the neurology domain of TopCare hospital #2 collaborates with an academic partner within the same region. All other domains collaborate with partners outside the region, a striking difference from the geographical results shown in Fig.  3 .

Ethnographic analysis

This section reviews the results of our ethnographic analysis of the two TopCare hospitals from 2014 to 2018. To analyse the boundary work these hospitals performed to initiate and/or enhance productive research collaborations, we use the framework suggested by Santos and Eisenhardt (2005) for examining organizational boundary work through the concepts of identity, competence and power. Table 3 provides a description of each boundary and how these concepts are defined in our case study on the basis of the overarching themes in the document analysis and the interviews.

Identity: enhancing hospitals’ value proposition

In the TopCare program, the non-academic hospitals used their unique history and expertise to create a joint research focus in a domain and to enhance their positions and influence their collaboration with UMCs and universities.

A manager in hospital #1’s lung domain explained the work being done from a historical perspective, emphasizing not only the innovative history of the hospital but also its central position in patient care:

The first-ever lung lavage, lung transplant and angioplasty were performed in this hospital. Nationally, this hospital has always, and we’re talking about 50–60 years ago now, been at the forefront, and has always invested in this line of research and care. So that is truly institutionally built, there is just that history and you can’t just copy that. And we have the numbers: for interstitial lung diseases, we have 2000 patients in our practice and receive 600 new patients per year. (interview with manager at hospital #1 in 2018).

To explain why patient care and research into rare interstitial lung diseases is centred in hospital #1 as a strategic domain focus, a leading international pulmonary physician – a “boundary spanner” (see below) – pointed to the importance of building team expertise and creating facilities:

I lead that care program for interstitial lung diseases and preside over the related research. I’ve often been asked: you’re a professor, so why don’t you go to a UMC, couldn’t you do much more there? But the care was developed here [in this hospital]. The expertise needed to recognize interstitial lung diseases depends not only on me but also on the radiologist and pathologist; together we have a team that can do this. We have created facilities that no other hospital has for these diseases. If I leave to do the same work in a UMC, I’d have to start over and I’d be going back 30 years. (interview with pulmonary physician at hospital #1 in 2014).

The doctors working in this hospital’s lung and heart domains finance the working hours they put into research themselves. “This fits in with the spirit of a top clinical hospital and the entrepreneurial character of our hospital.” (interview with project leader at hospital #1 in 2018).

Hospital #2, the result of a merger in 2016, struggled to find its strategic focus. A surgical oncologist at this hospital clarified one of the disadvantages of the merger: “People are [still] busy dealing with the money and positions, and the gaze is turned inward, the primary processes. So clinical research is very low on the agenda.” She continued by saying that a small project team acting on behalf of the hospital’s board of directors (BoD) was seeking the best-fit profile for the program, which had raised some opposition in departments excluded from the chosen strategic focus. As a consequence, the hospital had begun to showcase its highly specialized care in the field of neurosurgical treatments. It had a long history and was the first to use a Gamma Knife device for treating brain tumours. The experts in this domain could thus act as authorities, and they became a national centre of expertise. Their strategic partner was a nearby UMC, and they treated relevant patients from other hospitals in their region.

To generate impact, research priorities in a domain are aligned with the focus of the hospital. A member of the BoD of hospital #2 stressed the urgency of “specializing or focusing on a particular area of care” and emphasized that the TopCare budget was being utilized to create a joint focus within a domain. The resulting collective identity mobilized internal affairs and was recognized as valuable by third parties. An important reason for joining the TopCare program for both hospitals was to be able to position themselves strategically as attractive and credible research partners:

The focus is on the domains of neurology and trauma because we think as a non-academic hospital we have something extra to offer: the very close relationship between patient care and research, because we have a larger number of patients of this type here than the universities. (interview with care manager at hospital #2 in 2013).

In short, the boundary of identity requires a closer alignment between these hospitals’ research activities and their strategic objectives and organizational mindset, and demands that they also showcase their staff’s expertise. The TopCare program offered opportunities to transform and consolidate their identity by enhancing their value proposition, that is, their unique history, strategic domain focus, expertise and number of patients.

Competence: Enhancing research infrastructures

All domains in the TopCare program chose to utilize the TopCare funding to invest in their research infrastructure, and to build research networks to share and learn. A research infrastructure consists of all the organizational, human, material and technological facilities needed for specialist care and research [ 31 ].

The TopCare data show that funding is essential for generating research impact. A manager at hospital #1 described its current financial circumstances:

A lot of research and much of the care is currently not funded, it is actually paid for mostly by the hospital... We have had massive budgetary adjustments the past two or three years. ...It is increasingly difficult to finance these kinds of activities within your own operation. (interview with manager at hospital #1 in 2018).

The TopCare funding was used to enhance the material infrastructure in hospital #1’s heart domain:

A number of things in healthcare are really terribly expensive, and there is simply no financing at all for them. …Cardiac devices, for example. We are constantly trying things out, but there’s no compensation for it. (interview with project leader at hospital #1 in 2018).

Hospital #1 had a long-standing and firm relationship with a UMC in the lung domain, giving it a solid material infrastructure. For example, there were spaces where researchers, especially PhD students, could meet, collaborate and share knowledge [ 31 ]. Another essential part of the material infrastructure for the lung domain was the biobank, as highlighted by a leading international pulmonary physician:

Our board of directors made funds available through the innovation fund to start up a biobank, but developing it and keeping it afloat has now been made possible thanks to the TopCare funding. It’s a gift from heaven! It will allow for further expansion and we can now seek international cooperation. (interview with pulmonary physician at hospital #1 in 2014).

Notably, the program allowed both non-academic hospitals to digitize their infrastructure, for example, with clinical registration and data management systems. According to an orthopaedic surgeon at hospital #2, “Logistics have been created, which can very easily be applied to other areas. By purchasing a data system, everyone can record data in a similar way.”

Besides investing in data infrastructure, the human dimension was another crucial factor in the research infrastructure. Instead of working on research “at night”, it became embedded in physicians’ working hours. All domains indicated the importance of having researchers, statisticians and data management expertise available to ensure and enhance the quality of research, and both hospitals invested in research staffing.

After losing many research-minded traumatologists to academia, hospital #2 decided to invest in dedicated researchers to form an intermediate layer of full-time senior researchers linked to clinicians within the two domains.

I personally think this is the most important layer in a hospital, with both a doctor and a senior researcher supervising students and PhD candidates. Clinicians ask practical questions and researchers ask a lot of theoretical questions. Both perspectives are needed to change practices. I have also learned that it takes a few years before the two can understand each other’s language. (interview with neurosurgeon at hospital #2 in 2018).

Competence: Finding alignments within hospitals and research networks

The program offered the hospitals opportunities to structure internal forms of collaboration and build a knowledge base within a domain. For example, hospital #1 organized educational sessions with all PhD students in the heart domain.

Having more researchers working in our hospital has given the whole research culture a boost, as well as the fact that they are producing more publications and dissertations. (interview with cardiologist at hospital #1 in 2018).

Hospital #2 also encouraged cross-domain learning by organizing meetings between the neurology and trauma domains.

You know, you may not be able to do much together content-wise, but you can learn a lot from each other in terms of the obstacles you face (interview with project manager at hospital #2 in 2016).

At the beginning there was resistance to participating in the program.

It was doom and gloom; without more support, groups refused to join. That kind of discussion. So the financial details have been important in terms of willingness to participate. (interview with surgical oncologist at hospital #2 in 2018).

Another obstacle was local approval for multicentre studies, which led to considerable delay (interview with psychologist at hospital #2 in 2018). Overall, the TopCare program created a flywheel effect for other domains that proved essential for internal collaborations (interview with surgical oncologist at hospital #2 in 2018).

In hospital #1, collaboration between the heart and lung domains grew closer.

Divisions between the different disciplines are much less pronounced in our hospital than in UMCs. So it’s much easier to work together. We’d already collaborated closely on lung diseases, and this has improved during the program. (interview with cardiologist at hospital #1 in 2016)

At the network level, the TopCare data show that most researchers participated in national networks. For example, the neurology domain in hospital #2 had established a network of 16 non-academic hospitals. Limited funding prevented researchers at non-academic hospitals from attending many international seminars, and they had more trouble building their international networks. One exception concerned the researchers in the lung domain of hospital #1, who expanded their international network by organizing an international seminar during the TopCare program and by contributing to other national and international seminars.

Each TopCare domain provided highly specialized care and wanted to become a centre of expertise. However, a hospital can only provide highly specialized care if research is conducted to determine the best treatment strategies. The data show how the two are interwoven.

For example, a PhD student has sought to collaborate with a UMC on a specific aorta subject in which we have greater expertise and more volume in terms of patients than UMCs. Based on this link with this UMC, a different policy was drawn up and also implemented immediately in all kinds of other UMCs. (interview with cardiologist at hospital #1 in 2018).

Often, a leading scientist who is the driving force behind a domain in a hospital is a “boundary spanner”, a person in a unique position to bridge organizational boundaries and foster research collaboration by “enabling exchange between production and use of knowledge” [ 40 , p. 1176], [ 41 ]. For example, the leading pulmonary physician in hospital #1 is a boundary spanner who has done a huge amount of work to enhance collaboration. With interstitial lung disease care being concentrated here, this professor can offer fellowships and stimulate virtual knowledge-sharing by video conferencing for “second-opinion” consultations. The TopCare funding was used to finance this. The network is successful at a non-academic level.

These consultations are with colleagues in other hospitals and they avoid patients having to be referred. (interview with project leader at hospital #1 in 2018). Our network now [in 2018] consists of more than 14 hospitals, which we call every week to discuss patients with an interstitial lung disease. …UMCs participate indirectly in this network. For example, the north has a specific centre for this disease in a non-academic hospital and a nearby UMC refers patients to this centre, who are then discussed in our network. (interview with pulmonary physician at hospital #1 in 2018).

This physician also noted that the network was still growing; other colleagues from non-academic hospitals wanted to join it.

Yesterday, colleagues from XX and XX were here. And they all said, “I’ve never learned so much about interstitial lung diseases.” We’re imparting enormous amounts of expertise. (interview with pulmonary physician at hospital #1 in 2018).

In sum, focusing on the boundary of competence, the TopCare hospitals created and mobilized resources to invest in their research infrastructure. In every domain, this infrastructure was used to strengthen the relationship between research, care and education, and to build and enhance internal and external research networks to share and learn.

Power: Enhancing the relationship with or finding and mobilizing strategic academic partners

For TopCare non-academic hospitals, the boundary of power is concerned with creating the right sphere of influence, meaning BoDs and administrators attempt to find and mobilize new strategic partners and build mutual relationships with various stakeholders at different levels.

A project leader at hospital #2 emphasized that the additional resources of the TopCare program created an opportunity for the non-academic hospitals “to show our collaborative partners that we’re a valuable partner.” For once, the tables were turned:

We’ve always had a good relationship with one UMC; they always used the data from our surgeries. But it’s nice that we can finally ask them whether they want to join us. That makes it a little more equal, and we can be a clinical partner. (interview with neurosurgeon at hospital #2 in 2018).

One of the requirements in each domain when applying to ZonMw for funding was alignment with academia in a research and innovation network. Collaboration often appeared more difficult at the administrative level when the academic partners worked in the same field of expertise, and tended to be more successful when the partners focused on different fields, where their interests did not conflict. According to a board member at hospital #2 who played a crucial role in a partnership agreement, a conscious decision was taken beforehand to seek partners beyond the medical domain as well.

There may be conflict with other groups within the walls of a UMC and I don’t see that as promising. You have to work together and we aren’t in a real position to do so. (interview with board member at hospital #2 in 2018).

Just before the end of the program, it was announced that this hospital had concluded a partnership agreement with a university to broaden their joint research program alongside neurology and trauma. An important prerequisite was that both organizations invest 1 million euros in the partnership. The board member revealed that the relationship with this university had in fact existed for some time:

So we went and talked to the university and they became interested. Then the top level was reorganized and replaced and we had to start from scratch again. That took a lot of time. Our goals were to awaken the enthusiasm of the board and at least three deans, otherwise it would be a very isolated matter. And we succeeded. Last week we had a matchmaking meeting at the university and there were about 50 pitches showing how we could be of value to each other. (interview with board member at hospital #2 in 2018)

Looking back, he defined the conditions for a successful collaboration with academia:

In terms of substance, the two sides have to be going in the same direction and complement each other, for example, in expertise, techniques, and/or facilities. And what is really important is that people know each other and are willing to meet each other…and there must be appreciation. (interview with board member at hospital #2 in 2018).

The trauma domain in hospital #2 wanted to become a trauma research centre in its region, and after investing in its research infrastructure, it found a new strategic academic partner:

We have also found new partners, for example, the Social Health Care Department of a UMC [name]. And that really has become a strong partnership; the intent was there for years, but we had no money. (interview with epidemiologist at hospital #2 in 2018).

The neurology domain at this hospital worked to form a network with a university of technology and a university social science department.

Officially, our hospital can’t serve as a co-applicant for funding and that is frustrating. However, I am pleased to show that we are contributing to innovation. (interview with neurosurgeon at hospital #2 in 2018).

A board member at this hospital reflected on the qualities needed for research and concluded: “The neuro group has more of those intrinsic qualities than the trauma group. …I think the trauma group is actually at a crossroads and will think twice about whether they can attract capacity to develop the research side or fall back to a very basic level.”

In hospital #1, administrators rejected a proposal to collaborate with the nearest UMC submitted by medical specialists in the heart domain. Past conflicts and unsuccessful ventures still influenced the present, even though the individuals involved had already left.

A further factor was raised by a manager at hospital #1, who reflected on the importance of obtaining a professorship in the heart domain:

If we can, even on the basis of any kind of appointment, obtain a professorship from the heart centre, then yes, that helps! …I think it just helps throughout the whole operation, politically speaking, as extra confirmation, extra legitimization for that status. (interview with manager at hospital #1 in 2016).

Eventually, hospital #1 managed to find alignment with a UMC in another region during the program and a medical specialist from the hospital became a professor by special appointment.

This UMC showed the greatest determination, actually, while we could have chosen to collaborate with the nearest UMC [but we didn’t]. And there was actually also a real click between both the administrators and the specialists. (interview with manager at hospital #1 in 2018).

Additionally, the TopCare data show that, while there may be close alignment with the nearest UMC, collaboration is not limited to this and proximity can sometimes even be detrimental (for example, in some cases hospitals compete for patients). As research and care in the TopCare hospitals’ domains became more specialized, they required the specific expertise of UMCs in other regions.

One critical dependency in the collaboration between a university or UMC and a non-academic hospital is the distribution of dissertation premiums, valued at about €100,000 per successful PhD track. Currently, after completion of a dissertation, the premium goes entirely to the university or UMC, even when much of the candidate’s research and supervision takes place in a non-academic hospital [ 31 ]. This structural difference makes collaboration less financially valuable to non-academic hospitals. For example, the leading pulmonary physician in hospital #1 is a professor who is affiliated with both a UMC and non-academic hospital, a boundary spanner who works across organizational boundaries, is successful in research, and bears responsibility for a significant proportion of the research output in the lung domain and in the collaboration with other organizations. Moreover, he does most of the PhD supervision, and his students do their work in hospital #1. Despite all this work, the dissertation premium goes to the UMC. Although efforts have been made to change this, certain institutional structures are so strongly embedded that it is difficult to open the organizational boundary.

Power: Aligning with the BoDs and administrators of the TopCare non-academic hospitals

During our research, we observed how the BoDs and administrators of the two TopCare hospitals discussed the progress of the program and worked together to learn from each other.

We can learn a lot from hospital #1 regarding the organization of our research, we think. That has been very inspiring. …On the other hand, the focus has been very centred on getting the domain and project requests funded at all. (interview with care manager at hospital #2 in 2013).

The BoDs opted for an approach aimed at building mutual trust and understanding. As a result, their alliance became more intensive during the program. By the time the program’s final report was released, both BoDs were leveraging their power to influence ZonMw’s next step: the follow-up to TopCare. They had a targeted plan for their lobbying. For example, after mutual coordination, the BoD of each hospital sent a letter to the Ministry of Health sketching their vision for the future.

In summary, for the TopCare hospitals, the boundary of power centred on finding alignment with strategic academic partners and the other BoDs and administrators in the TopCare program. Moreover, ties with strategic partners were important for extending the organization’s sphere of influence [ 33 ] in building and enhancing productive research collaborations. These hospitals recognized that they could not dismantle the existing structure of research funding, and they therefore committed themselves to trying to extend the TopCare program. Table 4 summarizes the opportunities and challenges within the three boundary concepts.

In our study, we used a mixed methods research design to explore research collaborations by focusing on the research output and impact of UMCs and non-academic hospitals in the Netherlands and by zeroing in on the boundary work of two Dutch non-academic hospitals for achieving collaboration.

Our bibliometric analysis shows that collaboration matters, especially for non-academic hospitals. Access to research grants, EU funding and international collaborations is harder for non-academic hospitals, and they need to collaborate with UMCs to generate research impact, assessed by means of MNCS impact scores. Conversely, non-academic hospitals are important for UMCs because they have a larger volume of patients. When UMCs and non-academic hospitals collaborate, their impact scores are higher. Impact scores are, moreover, higher for international collaborative publications across all types of hospital and all periods. More in-depth research is needed into why collaboration increases impact.

Bibliometric analysis of the domains of the two TopCare non-academic hospitals underscores their leading role in these domains. Upon receiving TopCare funding, the hospitals had to engage in various forms of boundary work to meet the requirement mandated by ZonMw of establishing a research collaboration with academia. They used the additional program resources to invest [ 33 ] in opening a boundary for research collaboration with academic partners.

Identity work involves creating an image of the organizational unit that legitimizes its research and care status in line with the dominant mindset of the organization. In practice, the relevant unit needs to establish a distinctive history and domain focus that aligns with the organizational strategy of the hospital, in-house expertise and patient flow. This requires coordination work with the BoD. However, not all domains have been successful in creating such an identity. It proved much more difficult for the trauma domain, for example, because their research is not as highly specialized as and more fragmented than the other domains.

Competence work focuses on organizational (a well-functioning science support unit), technological (registration systems) and material (floor space or biobank) infrastructure, depending on individual requirements. Additionally, tremendous efforts go into the human dimension of infrastructure, as TopCare hospitals consider research staff and making time available for doctors to be important conditions for building structurally supportive research programs. In a previous study, we highlighted that collaboration between all non-academic hospitals within the Association of Top Clinical Teaching Hospitals (STZ) is essential for strengthening their research infrastructure [ 42 ], and can also be seen as a matter of efficiency [ 35 ]. Moreover, in each TopCare hospital, competence work served to bring domains together to facilitate shared learning. Knowledge-sharing across departments or communities is an example of opening boundaries to facilitate integration, convergence or enrichment of points of view [ 36 , 43 , 44 ].

Professors with double affiliations can act as boundary spanners. They play a significant role as experts in a domain by creating its distinctive character, and they surmount borders and break down barriers through their network relationships with other hospitals. Additionally, these persons are responsible for a significant share of the research output in their domain and conduct research with worldwide impact in collaboration with other organizations. Their boundary work must be recognized as essential because they bring usable knowledge to the table, create opportunities for improved relationships across disciplines, enhance communication between stakeholders and facilitate more productive research collaborations [cf. 45 ].

The TopCare hospitals do much less work in the power dimension because the domains in which they operate are adjacent to those of academia. Our study shows that more successful, productive research collaborations are created when the hospital’s academic partner works in a complementary but not identical field. Only in one case, the heart domain, did collaboration succeed in an identical field, but that was because the academic partner was located outside of the hospital’s region and was therefore not a competitor. According to Joo et al., a potential partner’s suitability is determined not only by complementarity, their unique contribution to research collaboration in terms of expertise, skills, knowledge, contexts or resources but also by compatibility and capacity. Partner compatibility involves alignment in vision, commitment, trust, culture, values, norms and working styles, which facilitate rapport-building and cross-institutional collaboration [ 46 ]. TopCare data indicate that research collaborations should be managed to ensure all partners can operate as equals [ 47 ]. Partner capacity refers to the ability to provide timely resources (for example, expertise, skills or knowledge) for projects, as well as leadership commitment, community engagement and institutional support for long-term, mission-driven goals, such as the joint research program in neurology and trauma at hospital #2 and a university.

These three qualitative criteria – partner compatibility, complementarity and capacity – are aspects of power dynamics that influence strategic decisions about recruiting research partners. Generally, power dynamics shape a hospital’s strategic choices regarding whether to collaborate, with whom to partner and the extent of the research collaboration [ 48 ]. Future research should examine these power dynamics in a more integrated manner to unlock the full potential of collaboration [ 46 ].

It was possible to unravel how non-academic hospitals participating in the TopCare program engaged in research collaborations with academia. As the program did not interfere with the existing care, research and financing structures within the UMCs, it allowed TopCare non-academic hospitals to also combine top clinical care and research. The boundary concepts allow us to observe a dual dynamic in the collaboration: the opening of boundaries while simultaneously maintaining certain limits. Opening boundaries refers to facilitating collaboration through activities related to identity and competence, while maintaining them involves the power balance. The temporary program did not disrupt the existing power balance associated with the budgetary “academic component” and the dissertation premiums that accrue to academia. Overall, then, the power dimension may well be the primary factor that made it impossible for the TopCare non-academic hospitals to attain their ultimate goal: secure a consistent form of funding for their research and top clinical care. Instead, the national authorities introduced a new, temporary funding program for non-academic hospitals, and preserved the status quo favouring academia.

A key finding is that, if a hospital is successful in establishing coherence between the different forms of boundary work, it can create productive research collaborations and generate research impact. The TopCare hospitals performed boundary work to strengthen their research infrastructure (competence) and their research status (identity) and create a favourable negotiating position opposite academia (power). For example, choosing the lung domain as the hospital’s strategic focus (identity) and establishing a database as a fundamental source of information for research by a boundary spanner (competence) generated sufficient power to make the hospital a key player in this field and a much-respected collaboration partner, nationally and internationally. However, some restrictions remained in place, such as the national lung research network consisting only of non-academic hospitals, with UMCs participating only indirectly.

Another key finding is that possessing a substantial budget is not in itself enough to ensure successful research collaboration. It is clear from this study that extensive boundary work is also needed to facilitate research collaboration. Given the absence of structural funding, the TopCare non-academic hospitals were under pressure to deliver results during the program, making research collaboration even more crucial for them than for the UMCs in this context. Additionally, because highly specialized care and research at the TopCare non-academic hospitals required unique expertise, they had a growing need for collaboration at the national level. Contrary to assumptions and the findings of our analysis of UMCs and non-academic hospitals overall, their collaborative partners were not predominantly located at the nearest UMC.

Does our study align with the literature and support the results of similar initiatives, such as the establishment of Collaborations for Leadership in Applied Health Research and Care (CLAHRC), a regional multi-agency research network of universities and local national health service (NHS) organizations focused on improving patient outcomes in England by conducting and utilizing applied health research [ 49 ]? And what does it contribute to previous research?

While differences exist between the National Health Service (NHS) and the healthcare system in the Netherlands, there are also noteworthy parallels that render a comparison possible. These include encouraging networks to boost research productivity, fostering collaboration within a competitive system and funding research that is relevant to public health priorities. Moreover, building upon the findings of CLAHRC regarding boundary work within a competitive system and developing and funding research that is relevant to patient needs and public health priorities, there are further parallels, such as creating strong local research infrastructures and local networks [ 49 ], and using influential and skilled boundary spanners [ 49 , 50 ]. In addition, we found that research history, strategic domain focus, in-house expertise, patient flows, and network relationships pre-conditioned the TopCare hospitals’ collaboration with academia. Our results further show that, for non-academic hospitals seeking to create productive research collaborations, it is essential to work in complementary fields and to establish a coherence between identity, competence and power.

Our findings indicate that, after opening a boundary with academia, the focus of the TopCare hospitals was on searching for mutual engagement. These hospitals tried to clarify their added value by creating boundaries to distinguish themselves from UMCs, and attempted to extend the TopCare program without it overlapping with the budgetary “academic component”, so that it posed no threat to the UMCs. Boundary-crossing involves a two-way interaction of mutual engagement and commitment to change in practices [ 51 ]. It is likely that the program did not last long enough to instigate changes in practices, as it can take time to develop mutual understanding and foster trusting relationships [ 52 ].

Based on the CLAHRC results and our research findings, the trend towards regionalization in the Netherlands [ 53 ] and a new leading and coordinating role for UMCs in this research landscape [ 52 , 54 ] can only be successful if boundary work is conducted, allowing research-minded non-academic hospitals to:

Build a “collaborative identity” [ 50 , 55 , 56 ] over time with their academic partners (identity);

Establish added value in their research infrastructures compared with that of their academic partners (competence);

Create solid networks for learning and sharing knowledge [ 55 , 57 ] with their academic partners (competence);

Mobilize boundary spanners to bridge disciplinary and professional boundaries in research, teaching and practice [ 49 , 50 , 55 , 58 ] and publish articles in collaboration with academic partners with high research impact (competence);

Find the inspiration and confidence to increase their co-dependence to, for example, gain benefits from interacting with different partners in the field [ 35 ] (power); and

Create long-term collaborations with academia across sectors over time, as well as within sectors; this requires iterative and continual engagement between clinicians, academics, managers, practitioners and patients (power) [ 49 , 52 ].

It is conceivable that the evaluation of the follow-up study to the TopCare program, which will extend to 2025, could unravel these next steps.

Our results demonstrate that collaboration in research is important and should be encouraged. However, the current methods used to assess researchers underestimate this importance. Reward systems and metrics focus on the performance of individual researchers and may even discourage the development of medical research networks and collaboration [ 52 , 59 ]. There is ongoing debate about and rising criticism of the dominance of scientific impact scores as a measure of the performance of health researchers and research organizations [ 60 ]. Other forms of impact, such as the societal impact of medical research, are becoming more important, and different metrics are being developed. Research collaboration among individuals and organizations should be incentivized and rewarded, and should also be embedded in performance assessment and the core competences of all actors involved [ 61 ]. New ways of rewarding research collaboration within organizations should therefore be explored.

Limitations

This study is limited, both geographically and institutionally, to the Netherlands, and factors other than national and international research collaborations may explain the increase in research output and impact. For example, the research articles in our sample have not been analysed on substantive aspects such as methodology and funding. A bias may therefore have been introduced. Furthermore, the research output and impact of the TopCare non-academic hospitals that we measured was limited to the 4-year program period. A further limitation was the use of these hospitals’ research output as a measure of the influence of the TopCare program, as we were interested not only in the short-term effects (publications) but also in the long-term ones (on the work conducted to build research infrastructures). Moreover, the focus in the qualitative material concerning the TopCare program was on the two TopCare non-academic hospitals and, more specifically, on their national rather than their international collaborations.

Research collaboration between non-academic hospitals and academia in the Netherlands pays off in terms of publications and impact. For the publication of scientific articles, collaboration between UMCs and non-academic hospitals appears to be more prevalent and impactful for non-academic hospitals than for UMCs. When UMCs and non-academic hospitals collaborate, their impact scores tend to be higher. More research is needed into why collaboration leads to more impact.

Non-academic hospitals showed a higher rate of collaboration with the nearest UMC, whereas collaborative partners of TopCare hospitals were not predominantly located at the nearest UMC. TopCare hospitals prioritized expertise over geographical proximity as a predicator of their collaborative efforts, particularly as research and care in their domains became more specialized.

Drawing on the additional resources of the TopCare program, participating non-academic hospitals invested significantly in boundary work to open boundaries for research collaboration with academic partners and, simultaneously, to create boundaries that distinguished them from UMCs. Identity work was performed to ensure that their history and domain focuses were coherent with the dominant mindset of their organization, while competence work was done to enhance their research infrastructure. The human dimension of the infrastructure received considerable attention: more research staff, time made available for doctors and recognition that boundary spanners facilitate research collaborations.

Power work to find and mobilize strategic academic partners was mostly focused on complementary fields, as non-academic hospitals work in domains adjacent to those of academia. The TopCare hospitals tended to avoid power conflicts, resulting in a preservation of the status quo favouring academia.

The local research history, strategic domain focus, in-house expertise, patient flows, infrastructure and network relationships of each TopCare hospital influenced collaboration with academia [cf. 37 , 58 . Increased coherence between the different forms of boundary work led to productive research collaborations and generated research impact. To meet future requirements, such as regionalization, further boundary work is needed to create long-term collaborations and new ways of rewarding research collaboration within organizations.

Availability of data and materials

The datasets used and/or analysed during the study are available from the corresponding author upon reasonable request.

The names of the UMCs and non-academic hospitals and their numbers are not up to date due to mergers in the intervening period. The database contains data on eight UMCs; today there are seven, as two UMCs in Amsterdam merged in 2018. There are 28 non-academic hospitals in the database, whereas today 27 such hospitals are members of the Association of Top Clinical Teaching Hospitals ( https://www.stz.nl ). To ensure data consistency, the database remains unchanged.

Abbreviations

Board of directors

Center for Science and Technology Studies

International collaboration

Mean normalized citation score

Mean normalized journal score

National collaboration

Netherlands Federation of University Medical Centers

Single institution

Association of Top Clinical Teaching Hospitals

University medical centre

Abramo G, D’Angelo CA, Di Costa F. Research collaboration and productivity: is there correlation? High Educ. 2009. https://doi.org/10.1007/s10734-008-9139-z .

Article   Google Scholar  

De Solla Price DJ. Little science, big science. New York: Columbia University Press; 1963.

Book   Google Scholar  

Narin F, Carpenter MP. National publication and citation comparisons. JASIS&T. 1975. https://doi.org/10.1002/asi.4630260203 .

Beaver D, Rosen R. Studies in scientific collaboration: part III – professionalization and the natural history of modern scientific co-authorship. Scientometrics. 1979. https://doi.org/10.1007/BF02016308 .

Katz JS, Martin BR. What is research collaboration? Res Policy. 1997. https://doi.org/10.1016/S0048-7333(96)00917-1 .

Clark BY, Llorens JJ. Investments in scientific research: examining the funding threshold effects on scientific collaboration and variation by academic discipline. PSJ. 2012. https://doi.org/10.1111/j.1541-0072.2012.00470.x .

Bozeman B, Fay D, Slade CP. Research collaboration in universities and academic entrepreneurship: the-state-of-the-art. J Technol Transf. 2013. https://doi.org/10.1007/s10961-012-9281-8 .

Van Raan AF. Measuring science. In: Moed HF, Glänzel W, Schmoch U, editors. Handbook of quantitative science and technology research: the use of patent and publication statistics in studies of S&T systems. Dordrecht: Springer; 2004. p. 19–50. https://doi.org/10.1007/1-4020-2755-9_2 .

Chapter   Google Scholar  

Lotka AJ. The frequency distribution of scientific productivity. J Wash Acad Sci. 1926;16:317–23.

Google Scholar  

De Solla Price DJ, Beaver D. Collaboration in an invisible college. Am Psychol. 1966. https://doi.org/10.1037/h0024051 .

Zuckerman H. Nobel laureates in science: patterns of productivity, collaboration, and authorship. Am Sociol Rev. 1967;32:391–403.

Article   CAS   PubMed   Google Scholar  

Morrison PS, Dobbie G, McDonald FJ. Research collaboration among university scientists. High Educ Res Dev. 2003. https://doi.org/10.1080/0729436032000145149 .

Lee S, Bozeman B. The impact of research collaboration on scientific productivity. Soc Stud Sci. 2005. https://doi.org/10.1177/0306312705052359 .

Beaver DB. Collaboration and teamwork in physics. Czechoslov J Phys B. 1986. https://doi.org/10.1007/BF01599717 .

Acedo FJ, Barroso C, Casanueva C, Galán JL. Co-authorship in management and organizational studies: an empirical and network analysis. J Manag Stud. 2006. https://doi.org/10.1111/j.1467-6486.2006.00625.x .

Wuchty S, Jones BF, Uzzi B. The increasing dominance of teams in production of knowledge. Science. 2007. https://doi.org/10.1126/science.1136099 .

Article   PubMed   Google Scholar  

Sooryamoorthy R. Do types of collaboration change citation? Collaboration and citation patterns of South African science publications. Scientometrics. 2009. https://doi.org/10.1007/s11192-009-2126-z .

Gazni A, Didegah F. Investigating different types of research collaboration and citation impact: a case study of Harvard University’s publications. Scientometrics. 2011. https://doi.org/10.1007/s11192-011-0343-8 .

Landry R, Traore N, Godin B. An econometric analysis of the effect of collaboration on academic research productivity. High Educ. 1996. https://doi.org/10.1007/BF00138868 .

Laband DN, Tollison RD. Intellectual collaboration. J Political Econ. 2000. https://doi.org/10.1086/262132 .

Van Raan A. The influence of international collaboration on the impact of research results: some simple mathematical considerations concerning the role of self-citations. Scientometrics. 1998;42(3):423–8.

Glänzel W. National characteristics in international scientific co-authorship relations. Scientometrics. 2001. https://doi.org/10.1023/a:1010512628145 .

Glänzel W, Schubert A. Analysing scientific networks through co-authorship. In: Moed HF, Glänzel W, Schmoch U, editors. Handbook of quantitative science and technology research: the use of patent and publication statistics in studies of S&T systems. Dordrecht: Kluwer; 2004. p. 257–76. https://doi.org/10.1007/1-4020-2755-9_20 .

Didegah F, Thelwall M. Which factors help authors produce the highest impact research? Collaboration, journal and document properties. J Informetr. 2013. https://doi.org/10.1016/J.JOI.2013.08.006 .

Thelwall M, Maflahi N. Academic collaboration rates and citation associations vary substantially between countries and fields. J Assoc Inf Sci Technol. 2020. https://doi.org/10.1002/asi.24315 .

Archibugi D, Coco A. International partnerships for knowledge in business and academia: a comparison between Europe and the USA. Technovation. 2004. https://doi.org/10.1016/S0166-4972(03)00141-X .

Levi M, Sluiter HE, Van Leeuwen T, Rook M, Peeters G. Medisch-wetenschappelijk onderzoek in Nederland: Hoge kwaliteit door samenwerking UMC’s en opleidingsziekenhuizen. NTvG. 2013;157:A6081.

Abramo G, D’Angelo CA, Di Costa F. University-industry research collaboration: a model to assess university capability. High Educ. 2011. https://doi.org/10.48550/arXiv.1811.01763 .

Centraal Bureau voor de Statistiek. Health care institutions; key figures, finance and personnel. https://www.cbs.nl/nl-nl/cijfers/detail/83652ENG . Accessed 6 Mar 2024.

Postma J, Zuiderent-Jerak T. Beyond volume indicators and centralization: toward a broad perspective on policy for improving quality of emergency care. Ann Emerg Med. 2017. https://doi.org/10.1016/j.annemergmed.2017.02.020 .

Postma JP, Van Dongen-Leunis A, Van Hakkaart-van Roijen L, Bal RA. Evaluatie Topzorg. Een evaluatie van 4 jaar specialistische zorg en wetenschappelijk onderzoek in het St. Antonius Ziekenhuis, het Oogziekenhuis en het ETZ. Rotterdam: Erasmus School of Health Policy & Management; 2018.

Gieryn TF. Boundary-work and the demarcation of science from non-science: strains and interests in professional ideologies of scientists. Am Sociol Rev. 1983. https://doi.org/10.2307/2095325 .

Gieryn TF. Cultural boundaries of science: credibility on the line. Chicago: University of Chicago Press; 1999.

Abbott A. The system of professions. Chicago: University of Chicago Press; 1988.

Santos FM, Eisenhardt KM. Organizational boundaries and theories of organization. Organ Sci. 2005. https://doi.org/10.1287/orsc.1050.0152 .

Chreim S, Langley A, Comeau-Vallée M, Huq JL, Reay T. Leadership as boundary work in healthcare teams. Leadership. 2013. https://doi.org/10.1177/174271501246 .

Waring J, Crompton A, Overton C, Roe B. Decentering health research networks: framing collaboration in the context of narrative incompatibility and regional geo-politics. Public Policy Adm. 2022. https://doi.org/10.1177/0952076720911686 .

Siaw CA, Sarpong D. Dynamic exchange capabilities for value co-creation in ecosystems. J Bus Res. 2021. https://doi.org/10.1016/j.jbusres.2021.05.060 .

Velter M, Bitzer V, Bocken N, Kemp R. Boundary work for collaborative sustainable business model innovation: the journey of a Dutch SME. J Bus Models. 2021. https://doi.org/10.5278/jbm.v9i4.6267 .

Bednarek AT, Wyborn C, Cvitanovic C, Meyer R, Colvin RM, Addison PF, et al. Boundary spanning at the science–policy interface: the practitioners’ perspectives. Sustain Sci. 2018. https://doi.org/10.1007/s11625-018-0550-9 .

Article   PubMed   PubMed Central   Google Scholar  

Neal JW, Neal ZP, Brutzman B. Defining brokers, intermediaries, and boundary spanners: a systematic review. Evid Policy. 2022. https://doi.org/10.1332/174426420X16083745764324 .

Van Oijen JCF, Wallenburg I, Bal R, Grit KJ. Institutional work to maintain, repair, and improve the regulatory regime: how actors respond to external challenges in the public supervision of ongoing clinical trials in the Netherlands. PLoS ONE. 2020. https://doi.org/10.1371/journal.pone.0236545 .

Carlile PR. Transferring, translating, and transforming: an integrative framework for managing knowledge across boundaries. Organ Sci. 2004. https://doi.org/10.1287/ORSC.1040.0094 .

Orlikowski WJ. Knowing in practice: enacting a collective capability in distributed organizing. Organ Sci. 2002. https://doi.org/10.1287/orsc.13.3.249.2776 .

Goodrich KA, Sjostrom KD, Vaughan C, Nichols L, Bednarek A, Lemos MC. Who are boundary spanners and how can we support them in making knowledge more actionable in sustainability fields? Curr Opin Environ Sustain. 2020. https://doi.org/10.1016/j.cosust.2020.01.001 .

Joo J, Selingo J, Alamuddin R. Unlocking the power of collaboration. How to develop a successful collaborative network in and around higher education. Ithaka S+R; 2019.

McDonald J, Jayasuriya R, Harris MF. The influence of power dynamics and trust on multidisciplinary collaboration: a qualitative case study of type 2 diabetes mellitus. BMC Health Serv Res. 2012. https://doi.org/10.1186/1472-6963-12-63 .

Harrington S, Fox S, Molinder HT. Power, partnership, and negotiations: the limits of collaboration. WPA-LOGAN. 1998;21:52–64.

Soper B, Hinrichs S, Drabble S, Yaqub O, Marjanovic S, Hanney S, et al. Delivering the aims of the Collaborations for Leadership in Applied Health Research and Care: understanding their strategies and contributions. Health Serv Deliv Res. 2015. https://doi.org/10.3310/hsdr03250 .

Lockett A, El Enany N, Currie G, Oborn E, Barrett M, Racko G, Bishop S, Waring J. A formative evaluation of Collaboration for Leadership in Applied Health Research and Care (CLAHRC): institutional entrepreneurship for service innovation. Health Serv Deliv Res. 2014. https://doi.org/10.3310/hsdr02310 .

Engeström Y. The horizontal dimension of expansive learning: weaving a texture of cognitive trails in the terrain of health care in Helsinki. In: Achtenhagen F, John EG, editors. Milestones of vocational and occupational education and training. Bielefelt: W Bertelmanns Verlag; 2003. p. 152–79.

Gezondheidsraad. Onderzoek waarvan je beter wordt: Een heroriëntatie op umc-onderzoek. Den Haag: Gezondheidsraad; 2016.

van der Woerd O, Schuurmans J, Wallenburg I, van der Scheer W, Bal R. Heading for health policy reform: transforming regions of care from geographical place into governance object. Policy Politics. 2024. https://doi.org/10.1332/03055736Y2024D000000030 .

Iping R, Kroon M, Steegers C, van Leeuwen T. A research intelligence approach to assess the research impact of the Dutch university medical centers. Health Res Policy Syst. 2022. https://doi.org/10.1186/s12961-022-00926-y .

Rycroft-Malone J, Burton C, Wilkinson JE, Harvey G, McCormack B, Baker R, et al. Collective action for knowledge moblisation: a realist evaluation of the collaborations for leadership in applied Health Research and care. Health Serv Deliv Res. 2015. https://doi.org/10.3310/hsdr03440 .

Kislov R, Harvey G, Walshe K. Collaborations for leadership in applied health research and care: lessons from the theory of communities of practice. Implement Sci. 2011. https://doi.org/10.1186/1748-5908-6-64 .

Harvey G, Fitzgerald L, Fielden S, McBride A, Waterman H, Bamford D, et al. The NIHR collaboration for leadership in applied health research and care (CLAHRC) for Greater Manchester: combining empirical, theoretical and experiential evidence to design and evaluate a large-scale implementation strategy. Implement Sci. 2011. https://doi.org/10.1186/1748-5908-6-96 .

Currie G, Lockett A, Enany NE. From what we know to what we do: lessons learned from the translational CLAHRC initiative in England. J Health Serv Res Policy. 2013. https://doi.org/10.1177/1355819613500484 .

Hurley TJ. Collaborative leadership: engaging collective intelligence to achieve results across organisational boundaries. White Paper. Oxford Leadership. 2011.

DORA. The declaration. https://sfdora.org/read . Accessed 6 Mar 2024.

O’Leary R, Gerard C. Collaboration across boundaries: insights and tips from federal senior executives. Washington: IBM Center for The Business of Government; 2012.

Traag VA, Waltman L, Van Eck NJ. From Louvain to Leiden: guaranteeing well-connected communities. Sci Rep. 2019;9(1):5223. https://doi.org/10.1038/s41598-019-41695-z .

Article   CAS   Google Scholar  

Download references

Acknowledgements

The authors thank the two reviewers and the members of the Health Care Governance department of Erasmus School of Health Policy & Management, Erasmus University Rotterdam for their helpful comments on earlier drafts. We are particularly indebted to Kor Grit for his helpful comments and critical appraisal of this paper.

The TopCare program was funded by the Netherlands Organization for Health Research and Development (ZonMw) ( www.zonmw.nl/en ) under Grant [Number 80-84200-98-14001]. ZonMw had no role in the design or conduct of the study; the collection, management, analysis and interpretation of the data; or the preparation, review and approval of the manuscript.

Author information

Authors and affiliations.

Erasmus School of Health Policy & Management, Erasmus University Rotterdam, P.O. Box 1738, 3000 DR, Rotterdam, The Netherlands

Jacqueline C. F. van Oijen, Annemieke van Dongen-Leunis, Jeroen Postma & Roland Bal

Centre for Science and Technology Studies, Leiden University, Leiden, The Netherlands

Thed van Leeuwen

You can also search for this author in PubMed   Google Scholar

Contributions

Conceptualization: J.v.O., A.v.D.L. and T.v.L. (bibliometric analysis of UMCs and non-academic hospitals); A.v.D.L. and T.v.L. (bibliometric analysis of TopCare domains); and J.v.O., J.P. and R.B. (ethnographic interviews in the TopCare program). Formal analysis: J.v.O., A.v.D.L. and T.v.L. (bibliometric analysis of UMCs and non-academic hospitals); A.v.D.L and T.v.L. (bibliometric analysis of TopCare domains); J.v.O., J.B. and R.B. (ethnographic interviews in the TopCare program). Funding acquisition: R.B. (TopCare program). Investigation: A.v.D.L and T.v.L. (database analysis of UMCs and non-academic hospitals and TopCare domains) and J.v.O., J.B. and R.B. (ethnographic interviews in the TopCare program). Methodology: J.v.O., A.v.D.L and T.v.L. (bibliometric analysis of UMCs and non-academic hospitals); A.v.D.L and T.v.L. (bibliometric analysis of TopCare domains); and J.v.O., J.B. and R.B. (ethnographic interviews in the TopCare program). Project administration: T.v.L. and A.v.D.L (bibliometric analysis of UMCs and non-academic hospitals and TopCare domains) and J.P. (TopCare program). Supervision: T.v.L. (bibliometric analysis of UMCs and non-academic hospitals and TopCare domains) and R.B. (bibliometric analysis of UMCs and non-academic hospitals and TopCare domains, and ethnographic interviews in the TopCare program). Visualization: A.v.D.L and T.v.L. (bibliometric analysis of UMCs and non-academic hospitals and TopCare domains). Original draft: J.v.O., A.v.D.L and R.B. Draft & revision: J.v.O., A.v.D.L, J.P., T.v.L. and R.B. All authors read and approved the final manuscript (and agreed to be both personally accountable for their own contributions and to ensure that questions related to the accuracy or integrity of any part of the work, even ones in which the author was not personally involved, would be appropriately investigated and resolved and that the resolution would be documented in the literature).

Corresponding author

Correspondence to Jacqueline C. F. van Oijen .

Ethics declarations

Ethics approval and consent to participate.

Not applicable; at the time we were conducting the research, ethical approval was not required. Nowadays our facility has an Ethics Committee that assesses research proposals involving human subjects (including interview studies), but this was not the case then. This study is not subject to the Dutch Medical Research Involving Human Subjects Act (WMO); it concerns collaboration on medical research in TopCare non-academic hospitals. For research not subject to the WMO, local policy and applicable procedures apply; as the TopCare program began in 2014, there were, as yet, no institutional rules in this area.

Consent for publication

Member check is part of our policy of informed consent of respondents and consent for publication. Specifically, we gave respondents the opportunity to peruse and add to quotes from their semi-structured interviews and to confirm our interpretation. The focus was on confirming and amending the quote and verifying the interpretation. The research team discussed the feedback received from the respondents and weighed it against the context of data analysis. Any disagreement on a respondent’s feedback was discussed directly with the respondent until consensus was reached. The STZ and NFU have given permission to use the data collected by CWTS on behalf of the NFU and STZ for the bibliometric analysis of this study. They have taken note of the results of this study and agreed to its publication.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

See Fig.  4 and Tables 5 , 6 , 7 , 8 , 9 , 10 and 11 .

UMCs produce 18 times (= 27,592/1503) more SI, four times (= 42,557/10880) more NC and 14 times (82,540/5896) more IC publications than non-academic hospitals.

Of all publications, 89% (= 152,688/170967) are attributed to UMCs and 11% (18,279/170967) to non-academic hospitals.

Joint publications in national collaboration: 82% (= 8943/10880) non-academic hospitals and 21% (= 8943/42557) UMCs.

Joint international publications: 66% (= 3874/5896) non-academic hospitals and 5% (= 3874/82540) UMCs.

Joint publications: 70% (= 12,816/18279) non-academic hospitals and 8% (= 12,816/152688) UMCs.

Relationship between joint publications and total publications in each type of collaboration: 17% (= 8943/53436) national collaboration and 4% (= 3874/88435) international collaboration.

Types of collaboration involving TopCare hospitals #1 and #2 between 2010 and 2016. #, total number of publications

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

van Oijen, J.C.F., van Dongen-Leunis, A., Postma, J. et al. Achieving research impact in medical research through collaboration across organizational boundaries: Insights from a mixed methods study in the Netherlands. Health Res Policy Sys 22 , 72 (2024). https://doi.org/10.1186/s12961-024-01157-z

Download citation

Received : 31 December 2022

Accepted : 31 May 2024

Published : 25 June 2024

DOI : https://doi.org/10.1186/s12961-024-01157-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Collaboration
• Research impact
• Bibliometric analysis
• Organizational boundary work

Health Research Policy and Systems

ISSN: 1478-4505

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

Biological Sciences

• Mellon College of Science

M.S. in Quantitative Biology and Bioinformatics

The study of Biology is undergoing a revolution driven by new technologies that enable scientists to generate extensive amounts of data.  For example, the costs of sequencing nucleic acids have dropped dramatically, resulting in unprecedented amounts of genomic, transcriptomic, and proteomic data.  Advances in imaging extend from the nano to the macro scale to probe function and generate enormous amounts of data that describe behaviours of cells from subcellular to organ-levels.  The new datasets cut across all subdisciplines in biology and enable scientists to ask questions in new ways to reveal the fundamental rules of life.

The M.S. in Quantitative Biology and Bioinformatics (MS-QBB) will prepare students for new careers bioinformatics and related fields. Our mission is to provide students who have background in life sciences skills to prepare for careers in bioinformatics. This program allows student to choose a 2-semester or a 3-semester program of study. If you are interested in applying, learn more about the application process on our admissions page or e-mail us .

Program Mission

To provide students who have a background in biology and other sciences with a practical and focused educational experience to prepare them for careers in bioinformatics and quantitative biological science.

2-semester M.S. in QBB

Our 2-semester option allows students to quickly gain the most relevant skills in bioinformatics. Students will begin study in late August and graduate in late May.

3-semester M.S. in QBB - Advanced Study

The 3-semester option allows students to spend a third semester gaining additional experience and some more advanced coursework. Students will begin study in late August, have the option to earn course credit with optional summer internships (interested students may apply to these in the first year), then students will complete their third semester in the following Fall and graduate in late December.

Students are encouraged to seek external internships after their first year and pursue this degree full-time, completing the program in 3 semesters.

Related programs

Students who are interested in this program may also want to consider the M.S. in Computational Biology and M.S. in Automated Science programs . Those programs expect a higher level of quantitative background & skills to enter and are designed to engage students with a more in-depth focus computational machine learning competencies and the application of machine learning to biological research.

How to Apply

The CMU Rales Fellow Program is dedicated to developing a diverse community of STEM leaders from underrepresented and underresourced backgrounds by eliminating cost as a barrier to education. Learn more about this program for master's and Ph.D. students. Learn more

Join our growing network of prospective students!

Fill in the form below to connect with a program advisor

• Graduate Students
• Administration

Psychometric properties of the Metacognitive Awareness Inventory (MAI): standardization to an international spanish with 12 countries

• Published: 24 June 2024

Cite this article

• Antonio P. Gutierrez de Blume   ORCID: orcid.org/0000-0001-6809-1728 1 ,
• Diana Marcela Montoya Londoño   ORCID: orcid.org/0000-0001-8007-0102 2 ,
• Virginia Jiménez Rodríguez 3 ,
• Olivia Morán Núñez 4 ,
• Ariel Cuadro 5 ,
• Lilián Daset 5 ,
• Mauricio Molina Delgado 6 ,
• Claudia García de la Cadena 7 ,
• María Beatríz Beltrán Navarro 8 ,
• Aníbal Puente Ferreras 3 ,
• Sebastián Urquijo 9 &
• Walter Lizandro Arias 10  

Metacognition is defined as a higher-order thinking skill that enables individuals to monitor, control, and regulate their thinking and behavior. In education, this skill is important, as learners need to self-regulate their learning behaviors for successful lifelong learning. Thus, it is essential for educators and learners alike to know their metacognitive skills. Researchers can assist in this endeavor by developing sound and valid quantitative measures for psychological phenomena such as metacognition. No measure is more commonly used for this purpose than the Metacognitive Awareness Inventory (MAI). In the present study, the International Group on Metacognition validated the MAI employing a standard, international Spanish with a robust sample of 12 Spanish-speaking countries and 1,622 undergraduate university students. Results revealed a solid final baseline confirmatory factor analysis model for all 12 countries that supports the original two-factor structure reported in English-speaking samples from the United States. Additionally, multigroup measurement invariance analyses revealed that although five parameters varied slightly across some countries, chi-square difference tests indicated that the comparison model with these constraints freely estimated was not significantly better than the fully constrained null model, supporting measurement invariance across countries. Thus, our version of the MAI using standard, international Spanish is a valid and reliable tool for measuring metacognitive awareness in Spanish-speaking countries.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Similar content being viewed by others

Normative data and standardization of an international protocol for the evaluation of metacognition in Spanish-speaking university students: A cross-cultural analysis

The bifactor model of the junior metacognitive awareness inventory (jr. mai), a comprehensive reanalysis of the metacognitive self-regulation scale from the mslq, data availability.

Data associated with this research are available to anyone upon reasonable request.

Code availability

Not applicable.

References 

Akın, A., Abacı, R., & Çetin, B. (2007). The validity and reliability study of the turkish version of the metacognitive awareness inventory. Educational Sciences: Theory and Practice, 7 (2), 671–678. https://toad.halileksi.net/wp-content/uploads/2022/07/metacognitive-awareness-inventory-toad.pdf .

Google Scholar  

American Educational Research Association (AERA), American Psychological Association (APA), & National Council on Measurement in Education (NCME). (2014). Standards for educational and psychological testing (2014 ed.). American Educational Research Association. https://www.aera.net/Publications/Books/Standards-for-Educational-Psychological-Testing-2014-Edition

Azevedo, R., & Aleven, V. (2013). International handbook of metacognition and learning technologies . Springer. https://doi.org/10.1007/978-1-4419-5546-3

Baker, L., & Cerro, L. (2000). Assessing metacognition in children and adults. In G. Schraw & J. C. Impara (Eds.), Issues in the measurement of metacognition (pp. 99–145). Buros Institute of Mental Measurements.

Bandura, A. (2001). Social cognitive theory: An agentic perspective. Annual Review of Psychology, 52 , 1–26. https://doi.org/10.1146/annurev.psych.52.1.1

Article   Google Scholar  

Bentler, P. M. (2005). EQS 6: Structural equations program manual. Multivariate Software.

Bosworth, A., Dowall, S., García-Dorival, I., Rickett, N., Bruce, C., Matthews, D., Fang, Y., Aljabr, W., Kenny, J., Nelson, C., Laws, T., Williamson, E., Stewart, J., Carroll, M., Hewson, R., & Hiscox, J. (2017). A comparison of host gene expression signatures associated with infection in vitro by the Makona and Ecran (Mayinga) variants of Ebola virus. Scientific Reports, 7 (1), 43144. https://doi.org/10.1038/srep43144

Brown, A. (1987). Metacognition, executive control, self-regulation, and other more mysterious mechanisms. In F. Weinert & R. Kluwe (Eds.), Metacognition, motivación and understanding (pp. 65–116). Lawrence erlbaum.

Buratti, S., & Allwood, C. (2015). Regulating metacognitive processes-support for a meta-metacognitive ability. In A. Peña-Ayala (Ed.), Metacognition: Fundaments, applications and trends. A prolife of the current state -of-the-art (pp. 17–35). Springer.

Chapter   Google Scholar  

Byrne, B. M. (2006). Structural equation modeling with EQS: Basic concepts, applications, and programming (2nd ed.). Lawrence Erlbaum Associates Publishers.

Cheng, E., & Chan, J. (2021). Developing metacognitive teaching strategies through lesson study . Springer.

Book   Google Scholar  

Craig, K., Hale, D., Grainger, C., & Stewart, M. (2020). Evaluating metacognitive self-reports: systematic reviews of the value of self-report in metacognitive research. Metacognition and Learning, 15 , 155–213. https://doi.org/10.1007/s11409-020-09222-y

Cross, D., & Paris, S. (1988). Developmental and instructional analyses of children’s metacognition and reading comprehension. Journal of Educational Psychology, 80 (2), 131–142. https://doi.org/10.1037/0022-0663.80.2.131

Dinsmore, D. L., Alexander, P. A., & Loughlin, S. M. (2008). Focusing the conceptual lens on metacognition, self-regulation, and self-regulated learning. Educational Psychology Review, 20 , 391–409. https://doi.org/10.1007/s10648-008-9083-6

Double, K., & Birney, D. (2019). Reactivity to measures of metacognition. Frontiers in Psychology, 10 (2755), 1–10. https://doi.org/10.3389/fpsyg.2019.02755

Dunlosky, J., & Rawson, K. A. (Eds.). (2019). The Cambridge Handbook of Cognition and Education . London: Cambridge University Press. https://doi.org/10.1017/9781108235631

Efklides, A. (2006). Metacognition and affect: What can metacognitive experiences tell us about the learning process? Educational Research Review, 1 (1), 3–14. https://doi.org/10.1016/j.edurev.2005.11.001

Efklides, A. (2008). Metacognition: Defining its facets and levels of func- tioning in relation to self-regulation and co-regulation. European Psychologist, 13 (4), 277–287. https://doi.org/10.1027/1016-9040.13.4.277

Efklides, A., & Misailidi, P. (2010). Introduction: The present and future in metacognition. In A. Efklides & P. Misailidi (Eds.), Trends and prospects in metacognition research (pp. 1–20). Springer.

Elosua, P. (2017). Avances, proyectos y retos internacionales ligados al uso de tests en Psicología. Estudos de Psicologia Campinas, 34 (2), 201–210. https://doi.org/10.1590/1982-02752017000200002

Flavell, J. H. (1976). Metacognitive aspects of problem solving. In L. B. Resnick (Ed.), The nature of intelligence (pp. 231–235). Lawrence Erlbaum.

Flores, J., Ostrosky, F., & Lozano, A. (2008). Batería de funciones frontales y ejecutivas: presentación. Revista de Neuropsicología, Neuropsiquiatría y Neurociencias , 8 (1), 141–158. Recuperado de https://aalfredoardila.files.wordpress.com/2013/07/ardila-a-ed-2008-funciones-ejecutivas-neuropsicologia-neuropsiquiatria-y-neurociencias-vol-8-n1.pdf . Accessed 19 Mar 2024

Flores-Lázaro, J., Castillo-Preciado, R., & Jiménez-Miramonte, N. (2014a). Desarrollo de funciones ejecutivas, de la niñez a la juventud. Anales de Psicología, 30 (2), 463–473. https://doi.org/10.6018/analesps.30.2.155471

Flores-Lázaro, J., Ostrosky-Schejet, F., & Lozano-Gutiérrez, A. (2014b). BANFE - 2. Batería neuropsicológica de funciones ejecutivas y lóbulos frontales . Manual Moderno.

Follmer, D. J., & Sperling, R. (2016). The mediating role of metacognition in the relationship between executive function and self-regulated learning. British Journal of Educational Psychology, 86 (4), 1–17. https://doi.org/10.1111/bjep.12123

Garner, R. (1990). When children and adults do not use learning strategies: Toward a theory of settings. Review of Educational Research, 60 (4), 517–529. https://doi.org/10.3102/00346543060004517

Garrido, J. (2010). Lengua y globalización: inglés global y español pluricéntrico. Historia y Comunicación Social, 15 , 67–102. https://revistas.ucm.es/index.php/HICS/article/view/HICS1010110063A

González-Cabañes, E., García, T., Álvarez-García, D., Fernández, E., & Rodríguez, C. (2022). Validation of the shortened version of the metacognitive awareness inventory in spanish university students. Psicothema, 34 (3), 454–462. https://doi.org/10.7334/psicothema2022.75

Gutierrez de Blume, A. P., & Montoya Londoño, D. (2021). Validación y examen de la estructura factorial del Metacognitive Awareness Inventory (MAI) en español con una muestra colombiana de estudiantes universitarios. Revista psicogente, 24 (46), 1–28. https://doi.org/10.17081/psico.24.46.4881

Gutierrez de Blume, A., Schraw, G., Kuch, F., & Richmond, A. (2020). General accuracy and general error factors in metacognitive monitoring and the role of time on task in predicting metacognitive judgments. Rev CES Psico, 14 (2), 179–208. https://doi.org/10.21615/cesp.5494

Gutierrez de Blume, A. P., Montoya Londoño, D. M., Daset, L., Cuadro, A., Molina Delgado, M., Morán Núñez, O., García de la Cadena, C., Beltrán Navarro, M. B., Arias Trejo, N., Ramirez Balmaceda, A., Jiménez Rodríguez, V., Puente Ferreras, A., Urquijo, S., Arias, W. L., Rivera, L. I., Schulmeyer, M., & Rivera-Sanchez, J. (2023). Normative data and standardization of an international protocol for the evaluation of metacognition in Spanish-speaking university students: A cross-cultural analysis. Metacognition and Learning, 18 (2), 495–526. https://doi.org/10.1007/s11409-023-09338-x

Gutierrez, A. P., & Price, A. F. (2017). Calibration between undergraduate students’ prediction of and actual performance: The role of gender and performance attributions. The Journal of Experimental Education, 85 , 486–500. https://doi.org/10.1080/00220973.2016.1180278

Gutierrez, A. P., & Schraw, G. (2015). Effects of strategy training and incentives on students’ performance, confidence, and calibration. The Journal of Experimental Education, 83 , 386–404. https://doi.org/10.1080/00220973.2014.907230

Hambleton, R. K. (1996). Adaptación de tests para su uso en diferentes idiomas y culturas: fuentes de error, posibles soluciones y directrices prácticas. En J. Muñiz (Coor.), Psicometría. Universitas.

Harrison, G., & Vallin, L. (2018). Evaluating the metacognitive awareness inventory using empirical factor-structure evidence. Metacognition Learning, 13 , 15–38. https://doi.org/10.1007/s11409-017-9176-z

Hernández, A., Elosua, P., Fernández-Hermida, J., & Muñiz, J. (2022). Comisión de Test: Veinticinco años velando por la calidad de los test. Papeles del psicólogo, 43 (1), 55–62. https://doi.org/10.23923/pap.psicol.2978

Huertas, A., Vesga, G., & Galindo, M. (2014). Validación del instrumento “inventario de habilidades metacognitivas (MAI)” con estudiantes Colombianos. Paxis, 5 , 55–74.

Jacobs, J., & Paris, S. (1987). Children’s metacognition about reading: Issues in definition, measurement, and instruction. Educational Psychologist, 22 (3–4), 255–278. https://doi.org/10.1207/s15326985ep2203&4_4

Kline, R. B. (2005). Principles and practice of structural equation modeling (2nd ed.). Guilford Press.

Lima Filho, R., & Bruni, A. (2015). Metacognitive awareness inventory: Trans- lation and validation from a confirmatory analysis. Psicologia: Ciência e Profissão, 35 (4), 1275–1293. https://doi.org/10.1590/1982-3703002292013

Livingston, J. A. (2003). Metacognition: An overview . (ED474273) ERIC. https://eric.ed.gov/?id=ED474273

López Morales, H. (2010). La andadura del español por el mundo . Taurus.

Magno, C. (2010). The role of metacognitive skills in developing critical thinking. Metacognition and Learning, 5 (2), 137–156. https://doi.org/10.1007/s11409-010-9054-4

Massodi, M. (2020). Metacognitive awareness in university studies: the comparative study of lithuanian and iranian cases . Mykolas Romeris University. https://repository.mruni.eu/handle/007/16501 . Accessed 19 Mar 2024

Metcalfe, J., & Greene, M. (2007). Metacognition of agency. Journal of experimental psychology General, 136 (2), 184–99. https://doi.org/10.1037/0096-3445.136.2.184

Muñiz, J., Elosua, P., & Hambleton, R. (2013). Directrices para la traducción y adapta- ción de los tests: segunda edición. Psicothema, 25 (2), 151–157. https://doi.org/10.7334/psicothema2013.24

Muñiz, J., Hernández, A., & Ponsoda, V. (2015). Nuevas directrices sobre el uso de los tests: investigación, control de calidad y seguridad. Papeles del Psicólogo, 36 (3), 161–173. https://repositorio.uam.es/handle/10486/671956 .

Muñiz, J., Elosua, P., Padilla, J. L., & Hambleton, R. K. (2016). Test adaptation standards for cross-lingual assessment. In C. S. Wells & M. Faulkner-Bond (Eds.), Educational measurement: From foundations to future (pp. 291–304). The Guilford Press.

Omprakash, A., Kumar, A., Kuppusamy, M., Sathiyasekaran, B., Ravinder, T., & Ramaswamy, P. (2021). Validation of Metacognitive Awareness Inventory from a Private Medical University in India. Journal of education and health promotion, 10 , 324. https://doi.org/10.4103/jehp.jehp_39_21

Ozturk, N. (2017). Assessing metacognition: theory and practices. International Journal of Assessment Tools in Education, 4 (2), 134–148. https://doi.org/10.21449/ijate.298299

Pandey, K., & Mohan, A. (2023). Unveiling the Subtle Strength of "Metacognition" in the domain of Education. YMER, 22 (3), 1131–1146. https://www.researchgate.net/publication/371176452_Unveiling_the_Subtle_Strength_of_Metacognition_in_the_domain_of_Education . Accessed 19 Mar 2024

Perikova, E., & Byzova, M. (2022). Factor structure of the russian version of the metacognitive awareness inventory. Кul’turno-istoricheskaya psikhologiya = Cultural-Historical Psychology, 18 (2), 116–126. https://doi.org/10.17759/chp.2022180213

Pintrich, P. R., Wolters, C. A., & Baxter, G. P. (2000). Assessing metacognition and self- regulated learning. In G. Schraw & J. C. Impara (Eds.), Assessing metacognition and self-regulated learning (pp. 43–97). Buros Institute of Mental Measurements.

Pour, A. V., & Ghanizadeh, A. (2017). Validating the Persian version of metacognitive awareness inventory and scrutinizing the role of its components in IELTS academic Reading achievement. Modern Journal Of Language Teaching Methods, 7 (3), 46–63.

Quinn, J., McEachen, J., Fullan, M., Gardner, M., & Drummy, M. (2021). Sumergirse en el aprendizaje profundo. Herramientas atractivas . Morata.

Schraw, G. (1998). Promoting general metacognitive awareness. Instructional Science, 26 (1), 113–125. https://doi.org/10.1023/A:1003044231033

Schraw, G. (2000). Assessing metacognition: Implications of the Buros symposium. In G. Schraw & J. C. Impara (Eds.), Issues in the measurement of metacognition (pp. 297–321). Buros Institute of Mental Measurements.

Schraw, G. (2002). Promoting general metacognitive awareness. In H. Hartman (Ed.), Metacognition in learning and instruction: Theory, research and practice (pp. 3–16). Kluwer Academic.

Schraw, G. (2009a). A conceptual analysis of five measures of metacognitive monitoring. Metacognition and Learning, 4 (1), 33–45. https://doi.org/10.1007/s11409-008-9031-3

Schraw, G. (2009b). Measuring metacognitive judgment. In D. Hacker, J. Dunlosky, & A. Graesser (Eds.), Handbook of Metacognition in Education (pp. 415–429). Routledge Taylor & Francis Group.

Schraw, G., & Dennison, R. (1994). Assessing metacognitive awareness. Contemporary Educational Psychology, 19 , 460–475. https://doi.org/10.1006/ceps.1994.1033

Serra, M., & Metcalfe, J. (2009). Effective implementation of metacognition. In D. Hacker, J. Dunlosky, & A. Graesser (Eds.), Handbook of Metacognition in Education (pp. 278–298). Erlbaum. https://doi.org/10.4324/9780203876428

Tarricone, P. (2011). The Taxonomy of Metacognition . Psychology press.

Teo, T., & Lee, C. B. (2012). Assessing the factorial validity of the metacognitive awareness inventory (MAI) in an Asian country: A confirmatory factor analysis. International Journal of Educational and Psychological Assessment, 10 (2), 92–103.

Tobias, S., & Everson, H. (2009). The importance of knowing what you know: A knowledge monitoring framework for studying metacognition in education. In D. J. Hacker, J. Dunlosky, & A. Graesser (Eds.), Handbook of Metacognition in Education (pp. 107–228). Routledge.

Torres, A. (2013). Del castellano de «un pequeño rincón» al español internacional. Revista de Estudios Lingüísticos Hispánicos, 3 , 205-224.  https://ojs.uv.es/index.php/normas/article/view/4679/4538

Ulloa Ordaya, M. (2019). Propiedades psicométricas del Inventario de Conciencia Metacognitiva (MAI) en estudiantes de secundaria de una institución educativa, UGEL 07, Lima. [Título profesional en psicología, Universidad Cesar Vallejo, Lima.Perú]. Repositorio Académico de la Universidad Cesar Vallejo. https://repositorio.ucv.edu.pe/handle/20.500.12692/44663

Veenman, M. V. J. (2005). The assessment of metacognitive skills. In B. Moschner & C. Artelt (Eds.), Lernstrategien und Metakognition: Implikationenfür Forschung und Praxis (pp. 75–97). Waxmann.

Veenman, M., Van Hout-Wolters, B., & Afflerbach, P. (2006). Metacognition and learning: Conceptual and methodological considerations. Metacognition and Learning, 1 (1), 3–14. https://doi.org/10.1007/s11409-006-6893-0

Veenman, M., Bavelaar, L., De Wolf, L., & Van Haaren, M. (2014). The on-line assessment of metacognitive skills in a computerized learning environment. Learning and Individual Differences, 29 , 123–130. https://doi.org/10.1016/j.lindif.2013.01.003

Wang, M. C., Haertel, G. D., & Walberg, H. J. (1990). What influences learning? A content analysis of review literature. The Journal of Educational Research, 84 (1), 30–43. https://doi.org/10.1080/00220671.1990.10885988

Werts, C. E., Linn, R. L., & Jöreskog, K. G. (1974). Intraclass Reliability Estimates: Testing Structural Assumptions. Educational and Psychological Measurement, 34 (1), 25–33. https://doi.org/10.1177/001316447403400104

Whitebread, D., Coltman, P., Pasternak, D. P., Sangster, C., Grau, V., Bingham, S., & Demetriou, D. (2009). The development of two observational tools for assessing metacognition and self-regulated learning in young children. Metacognition and Learning, 4 (1), 63–85. https://doi.org/10.1007/s11409-008-9033-1

Winne, P., & Azevedo, R. (2014). Metacognition. In K. Sawyer (Ed.), The Cambridge Handbook of the Learning Sciences (pp. 63–87). Cambridge University Press.

Winne, P. H., & Perry, N. E. (2005). Measuring self-regulated learning. In M. Boekaerts, P. R. Pintrich, & M. Zeidner (Eds.), Handbook of self-regulation (pp. 531–566). Elsevier Academic Press.

Xethakis, L. (2020). The psychometric properties of the Metacognitive Awareness Inventory in the Japanese EFL context. 熊本大学社会文化研究 , 18, 121–146.

Zohar, A. (2020). Scaling-up higher order thinking . Springer.

Zohar, A., & Dori, Y. (2012). Metacognition in Science education. Trends in current research . Springer.

Download references

There is no funding to report.

Author information

Authors and affiliations.

Department of Curriculum, Foundations, and Reading, Georgia Southern University, P.O. Box 8144, Statesboro, GA, 30460-8144, USA

Antonio P. Gutierrez de Blume

Departamento de Estudios Educativos, Facultad de Artes y Humanidades, Programa de Psicología, Universidad de Caldas, Universidad de Manizales, Manizales, Colombia

Diana Marcela Montoya Londoño

Universidad Complutense de Madrid, Madrid, Spain

Virginia Jiménez Rodríguez & Aníbal Puente Ferreras

Universidad de Panamá, Panama City, Panama

Olivia Morán Núñez

Universidad Católica de Uruguay, Montevideo, Uruguay

Ariel Cuadro & Lilián Daset

Universidad de Costa Rica, San José, Costa Rica

Mauricio Molina Delgado

Universidad del Valle de Guatemala, Guatemala City, Guatemala

Claudia García de la Cadena

Universidad de Guadalajara, Guadalajara, Mexico

María Beatríz Beltrán Navarro

Universidad Mar del Plata, Mar del Plata, Argentina

Sebastián Urquijo

Universidad Católica San Pablo, Arequipa, Peru

Walter Lizandro Arias

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Antonio P. Gutierrez de Blume .

Ethics declarations

Conflicts of interest/competing interests.

None of the authors have any conflicts of interest to report.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Original English Version of the MAI

Metacognitive awareness inventory.

Directions : Please move the slider to the point on the continuous line under each statement that best corresponds to how true each statement is about you.

Spanish Version of the MAI Translated via the Translation-Back Translation Method

Inventario de conocimiento metacognitivo (schraw & dennison, 1994 ).

Indicaciones: Por favor dibuje una línea vertical sobre la línea continua en cada uno de los enunciados, que mejor corresponda a qué tan cierto es cada una de las siguientes indicaciones sobre usted.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Gutierrez de Blume, A.P., Montoya Londoño, D.M., Jiménez Rodríguez, V. et al. Psychometric properties of the Metacognitive Awareness Inventory (MAI): standardization to an international spanish with 12 countries. Metacognition Learning (2024). https://doi.org/10.1007/s11409-024-09388-9

Download citation

Received : 09 January 2024

Accepted : 26 May 2024

Published : 24 June 2024

DOI : https://doi.org/10.1007/s11409-024-09388-9

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• International spanish validation
• Metacognition
• Subjective metacognitive awareness
• Self-regulated learning
• Find a journal
• Publish with us
• Track your research

This paper is in the following e-collection/theme issue:

Published on 25.6.2024 in Vol 12 (2024)

User Preferences and Needs for Health Data Collection Using Research Electronic Data Capture: Survey Study

Authors of this article:

Original Paper

• Hiral Soni 1 , PhD   ; 
• Julia Ivanova 1 , PhD   ; 
• Hattie Wilczewski 1 , BS   ; 
• Triton Ong 1 , PhD   ; 
• J Nalubega Ross 1 , PhD   ; 
• Alexandra Bailey 1 , MS   ; 
• Mollie Cummins 1, 2 , PhD   ; 
• Janelle Barrera 1, 3 , MPH   ; 
• Brian Bunnell 1, 3 , PhD   ; 
• Brandon Welch 1, 4 , PhD  

1 Doxy.me Research, Doxy.me Inc, Charleston, SC, United States

2 College of Nursing, University of Utah, Salt Lake City, UT, United States

3 Department of Psychiatry and Behavioral Neurosciences, University of South Florida, Tampa, FL, United States

4 Department of Public Health Sciences, Medical University of South Carolina, Charleston, SC, United States

Corresponding Author:

Hiral Soni, PhD

Doxy.me Research

Doxy.me Inc

18 Broad Street, 3rd Floor

Suite 6 and 7

Charleston, SC, 29401

United States

Phone: 1 8444369963

Email: [email protected]

Background: Self-administered web-based questionnaires are widely used to collect health data from patients and clinical research participants. REDCap (Research Electronic Data Capture; Vanderbilt University) is a global, secure web application for building and managing electronic data capture. Unfortunately, stakeholder needs and preferences of electronic data collection via REDCap have rarely been studied.

Objective: This study aims to survey REDCap researchers and administrators to assess their experience with REDCap, especially their perspectives on the advantages, challenges, and suggestions for the enhancement of REDCap as a data collection tool.

Methods: We conducted a web-based survey with representatives of REDCap member organizations in the United States. The survey captured information on respondent demographics, quality of patient-reported data collected via REDCap, patient experience of data collection with REDCap, and open-ended questions focusing on the advantages, challenges, and suggestions to enhance REDCap’s data collection experience. Descriptive and inferential analysis measures were used to analyze quantitative data. Thematic analysis was used to analyze open-ended responses focusing on the advantages, disadvantages, and enhancements in data collection experience.

Results: A total of 207 respondents completed the survey. Respondents strongly agreed or agreed that the data collected via REDCap are accurate (188/207, 90.8%), reliable (182/207, 87.9%), and complete (166/207, 80.2%). More than half of respondents strongly agreed or agreed that patients find REDCap easy to use (165/207, 79.7%), could successfully complete tasks without help (151/207, 72.9%), and could do so in a timely manner (163/207, 78.7%). Thematic analysis of open-ended responses yielded 8 major themes: survey development, user experience, survey distribution, survey results, training and support, technology, security, and platform features. The user experience category included more than half of the advantage codes (307/594, 51.7% of codes); meanwhile, respondents reported higher challenges in survey development (169/516, 32.8% of codes), also suggesting the highest enhancement suggestions for the category (162/439, 36.9% of codes).

Conclusions: Respondents indicated that REDCap is a valued, low-cost, secure resource for clinical research data collection. REDCap’s data collection experience was generally positive among clinical research and care staff members and patients. However, with the advancements in data collection technologies and the availability of modern, intuitive, and mobile-friendly data collection interfaces, there is a critical opportunity to enhance the REDCap experience to meet the needs of researchers and patients.

Introduction

Accurate and complete health outcome data directly from patients or study participants (hereon referred to as patients ) are critical for health care and research [ 1 - 3 ]. Unfortunately, it can be burdensome to extract patient-reported health data that researchers or providers need [ 4 , 5 ]. Collecting patient-reported outcomes data is becoming increasingly important in clinical research and care [ 6 , 7 ]. Self-administered web-based questionnaires, which patients can complete at a clinic or at home, are becoming a conventional approach to collect data for clinical research. Web-based questionnaires have advantages of being low-cost and easy to deploy at scale. A variety of clinical research electronic data capture (EDC) tools exist to streamline remote data collection and management. These systems comply with privacy regulations, integrate with different tools (such as electronic health records [EHRs]) for efficient data collection, and reduce the effort of sharing data [ 8 ]. However, user experience, cost, and maintenance of such commercial EDC systems are often prohibitive. An understanding of user experiences and preferences regarding EDC tools is critical in assessing stakeholder needs, satisfaction, and challenges in clinical and research settings.

REDCap (Research Electronic Data Capture; Vanderbilt University) is a global, secure web application for building and managing EDC for clinical research [ 9 , 10 ]. Developed by Vanderbilt University, REDCap is freely available for its consortium members (ie, network of nonprofit collaborators and supporters), who have an established agreement with the university. REDCap is compliant with global privacy regulations (such as the Health Insurance Portability and Accountability Act [HIPAA] of 1996) and used by more than 2.2 million researchers in more than 140 countries [ 9 ]. REDCap allows researchers to build and conduct electronic surveys, track and manage study information, schedule visits, and manage databases that are fully customizable and at no cost [ 11 ]. REDCap is designed to support data capture for research studies, providing (1) an intuitive interface for validated data capture, (2) audit trails for tracking data manipulation and export procedures, (3) automated export procedures for seamless data downloads to common statistical packages, and (4) procedures for data integration and interoperability with external sources.

Although REDCap is widely used, user needs and preferences of EDC via REDCap have rarely been studied [ 12 , 13 ]. For example, 1 usability study of a REDCap-based patient-facing intervention reported that patient participants found REDCap useful and easy to use but showed concerns about wordiness and inconsistent visual design [ 13 ]. Researchers have reported frequently on the implementation, use, and interventions using REDCap [ 10 , 14 - 20 ]. Understanding the preferences and needs of REDCap administrators and researchers using REDCap to capture data could help enhance existing features and EDC processes in general. While REDCap is a robust clinical research data management system, this study solely focuses on the experience of REDCap as an EDC tool. To the best of our knowledge, such preferences have not yet been studied.

The aim of this study was to survey REDCap administrators and researchers in the United States to assess their experience with REDCap, including perspectives on advantages, challenges, and suggestions for enhancement.

Study Settings and Respondents

We conducted a web-based survey with representatives of member organizations listed as REDCap Partners on the REDCap website [ 21 ]. The roles of the listed members were unclear at the time of invitation sent via email. The email communication included information related to the study goals, voluntary participation, and a link to the REDCap survey. Respondents were compensated with a US $10 electronic gift card for completing the survey.

Ethical Considerations

This study was reviewed and approved as exempt human subjects research by the Medical University of South Carolina Institutional Review Board (Pro00082875).

Survey Design

We developed a web-based survey with multiple-choice and free-response questions ( Multimedia Appendix 1 ) to capture the perspectives of researchers and administrators from participating REDCap consortium organizations. Our research team includes experts in biomedical informatics, behavioral sciences, mixed methods research, and user experience. The survey included 4 sections, as follows:

• Demographics : multiple-choice questions capturing participant role in their respective organization (Q1) and organization use of REDCap (Q2)
• Quality of patient-reported data collected via REDCap : Likert-scale questions capturing perspectives (ranging from 1=strongly agree to 5=strongly disagree) on the accuracy, reliability and completeness of data reported using REDCap (Q3)
• Patient experience with REDCap : Likert-scale question focusing on perspectives (ranging from 1=strongly agree to 5=strongly disagree) on REDCap usability, including ease of use, success rate, and completion time (Q4).
• Data collection experience : Free-response questions asking about the advantages (Q5), challenges (Q6), and suggestions of enhancements related to data collection, patient experience, and engagement (Q7).

Data Collection and Analysis

We collected and managed study data using REDCap EDC tools hosted at the Medical University of South Carolina [ 22 , 23 ]. We generated plots and univariate statistics to summarize the data (eg, frequencies, means, SDs, and percentages). We conducted 1-way ANOVA tests to determine differences in data quality and patient experience variables by participant role and REDCap use duration. For the ANOVAs, the primary role variable was restructured to include “Educators” in the “Other” category due to the low sample size (n=1). Excel (Microsoft Corp) and SPSS (version 29; IBM Corp) were used for analyses. Free-response questions were qualitatively analyzed to identify emerging themes related to REDCap data collection experience [ 24 ]. We randomly selected 15% of the responses for initial coding and codebook development. The coding unit was done by the entirety of the participant entry. Thematic analysis of all qualitative data was done over 4 iterations using MAXQDA, during which emergent themes were identified. While the research team reviewed and honed the codes and codebook, 1 team member coded and finalized thematic coding. Discrepancies were resolved through consensus. Emergent themes were organized by frequency and topic, allowing for further qualitative analysis using complex coding query to determine concurrent themes. We reported the total frequencies per code, which may not align with the number of participants. For example, 1 participant may report a code multiple times throughout their response [ 25 ]. While thematic analysis allows us to identify principle emergent themes, it also can help identify uncommon trends that may be significant but would require further investigation in follow-up research [ 26 ]. Responses from incomplete surveys with missing quantitative or qualitative responses were excluded from the analysis

Demographics

Between October and November 2020, 3058 representatives from 1676 REDCap member organizations in the United States were invited to complete the survey. In total, 285 (9.3%) invitees started the survey, of which 207 completed the survey. Most (150/207, 72.5%) respondents were REDCap administrators, followed by researchers (25/207, 12.1%). Furthermore, 1 (0.5%) respondent was an educator and 31 (15%) respondents served in other roles, including IT directors and managers, research coordinators and managers, program managers, project managers, director of research, library directors, and data analysts. Respondents reported that their organization had used REDCap for <5 years (92/207, 44.4%), 5 to 10 years (83/207, 40.1%), or >10 years (32/207, 15.5%).

Quality of Patient-Reported Data Collected via REDCap

We asked respondents about their perspectives of the quality of the survey data, including the accuracy, reliability, and completeness of the data collected using REDCap ( Figure 1 ). Most respondents strongly agreed or agreed that the data collected via REDCap are accurate (188/207, 90.8%), reliable (182/207, 87.9%), and complete (166/207, 80.2%). We observed no statistically significant group differences in accuracy ( F 2,204 =1.003; P =.37), completeness ( F 2,204 =0.243; P =.78), or reliability ( F 2,204 =0.245; P =.78) among respondent role groups. Furthermore, we observed no statistically significant group differences in accuracy ( F 2,204 =0.672; P =.51), completeness ( F 2,204 =0.045; P =.96), or reliability ( F 2,204 =1.712; P =.18) among REDCap use groups.

Patient Experience With REDCap

We also asked respondents about their perspectives on patient experiences with completing surveys using REDCap. Figure 2 summarizes their responses. More than half of respondents strongly agreed or agreed that patients find REDCap easy to use (165/207, 79.7%), could successfully complete tasks without help (151/207, 72.9%), and could do so in a timely manner (163/207, 78.7%). We observed no statistically significant group differences in ease ( F 2,204 =2.025; P =.13), successful task completion ( F 2,204 =0.671; P =.51), or timely task completion ( F 2,204 =2.303; P =.10) among respondent role groups. Furthermore, we observed no statistically significant group differences in ease ( F 2,204 =0.711; P =.49), successful task completion ( F 2,204 =1.851; P =.16), or timely task completion ( F 2,204 =2.000; P =.13) among REDCap user groups.

REDCap Advantages, Challenges, and Enhancement Suggestions

We asked respondents about the advantages, challenges, and suggestions for future enhancements using free-response questions. The analysis yielded 8 primary codes: survey development, user experience, survey distribution, survey results, training and support, technology, security, and platform features. Within each of these themes, responses were further categorized at secondary and tertiary levels. Multimedia Appendix 2 shows the qualitative codebook with illustrative examples for each code. Table 1 shows the frequencies response classification of advantages, disadvantages, and enhancements for each code category based on respondents’ responses.

a Due to the coding process (eg, double coding), the total number of secondary and tertiary codes may not add up to the primary code or 100%. The percentages are calculated based on the total number of codes in secondary and tertiary categories.

Survey Development and Customization

Respondents perceived that REDCap surveys were generally easy (20 codes) and quick (2 codes) to set up, build, organize, and maintain (2 codes). One participant commented on these topics, “Easy to build surveys Easy to make questions easy to answer Easy to build branching questions.”

However, respondents also noted that incorrect setup by the study staff and limited default formatting options and flexibility could be challenging in developing and completing surveys (3 codes).

While some respondents pointed out that REDCap provides continuous releases with new features (2 codes) and various design and automation options to ask a variety of questions for efficient data collection (8 codes), respondents frequently pointed out the value of well-designed survey instruments in gathering high-quality information and engaging patients. They reported that complex, poorly designed surveys and ambiguous instructions (39 codes) could result in poor patient experience, potentially impacting the survey response rate and quality of data gathered. Respondents provided suggestions for enhancing survey design capabilities to streamline survey design and layout for the patients (including simplifying survey formatting, survey nesting abilities, and use of embedded fields). Respondents also suggested pilot testing of surveys before sending them out to patients (3 codes) and for study teams to follow best practices and guidelines to be more informed in survey methodologies and development. For example, 1 respondent commented:

Study teams following best practices with survey methodology and design, which can involve keeping surveys short & sweet, choosing appropriate field types for the question at hand, phrasing questions and response options well to reduce mental burden and make it easier for patients to answer questions.

Respondents also reported that the availability of various response types, data validation, and branching logic ensure high-quality data collection (13 codes). One respondent commented on this advantage, “The wide array of validations can help patients enter data correctly.”

Another respondent noted similarly, “Data validation and branching logic make participants conform to data standards and allows researchers to obtain higher quality data.”

While data validation was discussed positively, respondents more frequently noted the challenges with response and logic types (22 codes), often pointing out that the actual response and logic types available from REDCap are not conducive to good survey design. One respondent made a clear reference to this issue saying, “It all depends on who sets up the survey, but until recently it has been a challenge to create grids of disparate data entry fields.”

In addition, some respondents noted that due to the logic types, patients can make critical mistakes affecting the completeness of the data:

...branching logic at a very question to determine if they qualify or not. Sometimes, they accidently select different value in a hurry, and the survey gets completed. It is hard for them to change the response or refill the survey without admin help.

Respondents noted many enhancement potentials within this category, such as voice input (4 codes), superior data entry experience (5 codes), use of a more conversational approach in response types (1 code), more effective multimedia (5 codes), and gamification of survey (2 codes). While REDCap offers multimedia options, respondents often suggested that options become more interactive and effective:

...more visual aids in questions, and the ability to answer with images. For example, by painting the areas afflicted on an image.

One respondent explained how multimedia may be further useful:

...ability to add images to response options. Especially when working with minorities (traffic lights, or smiley faces).

In addition to the design of the surveys, respondents noted that while REDCap surveys are readily customizable (7 codes), there are far more reported challenges (17 codes) and need for enhancements (15 codes). Respondents noted customization was not possible in some cases: “Default formatting options are limited.”

However, many respondents focused on the lack of multi-language support (9 codes) as the critical challenge:

...multi-linguistic support. This is always a challenge for any software system/platform, and REDCap is no different...

They frequently suggested enhancements to include multi-language support (8 codes) and customizations in forms’ appearance (6 codes). For example, 1 respondent mentioned, “Allow for some more customization of the overall look/feel of surveys.”

With respect to challenges with survey interactions, respondents reported that REDCap capabilities at the time did not send new surveys or allow patients to complete future surveys if previous surveys were incomplete (2 codes). One respondent mentioned the following:

...[t]he longitudinal design functionality in REDCap requires a participant to take each form before moving to the next, but our experiment design does not require this, and sometimes people will miss sessions and need to move on to the form for the next one. But if we stack all of the forms in one event, we cannot direct people to an individual form, only to the queue.

One participant commented on REDCap’s “ inability to provide staff log-in status .” (1 code). Respondents requested features for internal messaging or chat between study staff (2 codes), enhancing flow and cross-linking between projects (2 codes), ability to easily add study staff members outside of the organization (1 code), and ability for patients to skip longitudinal surveys (1 code).

User Experience

Respondents perceived REDCap to be easy to use for both patients (ie, to take surveys) and the study staff (ie, to build and distribute surveys; 57 codes). One respondent commented as follows:

REDCap is the easiest way to survey patients, families, and staff who are not part of our study team. We would not be able to conduct these surveys without it!

They also perceived REDCap to be user-friendly (11 codes), simple (8 codes), intuitive (3 codes), timely (2 codes), and reliable (3 codes). Although some respondents reported REDCap allows for quick data collection (7 codes), they perceived that lengthy or poorly designed surveys (eg, too many clicks and not enough instructions) could lead to fatigue and poor participation (15 codes). While the usability perceptions were generally positive, respondents reported that the platform was not as user-friendly or outdated as other commercial data collection platforms (4 codes), unintuitive (2 codes), and clunky for study staff (4 codes). They reported that “ REDCap is not the simplest tool to learn how to use ” for study staff (4 codes) and patients (3 codes). Respondents suggested the need to enhance accessibility features, such as the ability to change font size, screen reader view, and text-to-voice, among others (3 codes). In total, 8 (3.9%) of 207 respondents reported that the REDCap interface was advantageous for study staff considering its consistent interface and automated features, which reduce burden.

Respondents generally reported REDCap’s visual user interface as challenging to use. Although some respondents perceived the interface to be clean or simple looking (9 codes) and optimized for various devices (5 codes), other respondents perceived that REDCap’s interface was not modern looking (7 codes) or appealing (5 codes). One respondent mentioned, “The web interface of our survey pages are very basic, and narrow,” whereas another respondent said, “[REDCap has] Very set layout of each item, can’t make it look more ‘modern’ like other websites are at this time.”

Respondents considered REDCap as not having a configurable design (4 codes) and some noted the user interface’s poor functionality (8 codes). One respondent described both issues when explaining the challenges of the user interface:

REDCap is simply not user friendly in any way. The data structures are often too rigid and frankly outdated in being an effective tool for data collection.

Respondents suggested the redesign of the REDCap user interface to be consistent with modern data collection platforms (27 codes), options to change the visual appearance and formatting of the surveys (3 codes), adding progress tracking aids (such as an automatic progress bar) for patients (2 codes), and a more flexible interface (1 code).

Some respondents appreciated REDCap’s mobile access (4 codes), availability of mobile apps for study staff (REDCap mobile app; 2 codes) and patients (MyCap; 6 codes) supporting offline data collection, and perceived REDCap to be easy to use on mobile devices (3 codes) and mobile friendly (5 codes). While respondents appreciated the mobile interface, they reported that the mobile experience is affected by poor and suboptimal mobile user interface and scaling on smaller screens (5 codes). One participant reported the following:

We design our surveys on a computer, but many of our participants use their phones. We try to check how answers scale when the screen size changes, but some phones rescale to a different aspect ratio leading to challenges.

They also reported that although the REDCap mobile app is available for study staff, it is not ideal and is difficult for study staff to set up the app (4 codes). One respondent mentioned the following:

I think that the REDCap mobile app is a bit too far separated from the web version, in as much as there is no access to external modules and other important features.

Respondents suggested a need for an enhanced mobile app and interface (21 codes), including advanced capabilities for the study staff to view study records and perform analysis (2 codes) and push notifications (2 codes). One respondent mentioned the following:

[They need] better workflows with mobile phones, like notifications instead of just text messages. Something like an App except not the current one which is focus on asymmetric internet access.

Respondents also commented on patient experience with REDCap. Overall, respondents noted that REDCap makes it easier for patients to complete the surveys at their convenience (10 codes), all while increasing engagement levels (9 codes). They saw REDCap as a way to make data collection more efficient and empowered (2 codes), especially as patients did not need to register or remember usernames or passwords to use the platform (3 codes). One participant said, “[Survey] Can be done at the patient’s convenience from any digital device.” A common challenge reported was the patient’s desire and motivation to complete the surveys, being able to use the platform, and fatigue with lengthy surveys (13 codes). Suggestions for improving patient experience included maintaining engagement using visual aids and gamification (3 codes), a patient dashboard to keep them up to date on status of longitudinal studies (1 code) and making the platform more patient friendly (1 code). One respondent commented as follows:

For longer surveys, having a way of maintaining engagement by making the surveys more interactive (e.g. fun feedback to participants as they progress) would be nice. Some periodic messages of encouragement like “Great job!” “Keep it up!”

Survey Distribution and Reminders

Respondents found it advantageous that REDCap included multiple ways to invite patients, such as emails or embedded links (4 codes). REDCap surveys were easy to distribute (11 codes) and could be automated and scheduled on a timeline easily. One participant commented on this aspect, “It can send surveys to participants directly, and on a schedule when the project is longitudinal.” REDCap’s ability to send patients custom links was an advantage respondents liked (3 codes): “For online surveys: able [to] send individualized email links...automated email with message that has piping upon completion.” One respondent pointed out that there was “ no scheduling component for visits ” and suggested this feature. One respondent suggested the ability to send attachments with automatic notifications.

In addition, the ability to send completion reminder emails to patients was reported to reduce the burden on clinic staff while engaging patients (7 codes). Reminders also allowed the study staff members to follow up on incomplete surveys but 1 respondent mentioned that this was challenging while respondents suggested for improvements in customizing reminders and enhanced tracking for incomplete surveys longitudinally (3 codes):

If there was a more efficient way to upload and manage patient invitations, as well as identify which patients have completed the survey within previous xx months therefore a new survey invitation does not need to be sent.

Respondents noted patients sometimes missed invitations and reminders because email service providers blocked the emails (2 codes): “We have had email providers block REDCap emails, specifically Yahoo.com email.” There was also confusion about the email sender as the emails were “from” REDCap instead of the study staff (2 codes):

From my experience... The emails that are sent out to respondents are not user friendly. The ‘From’ text box comes from REDCap, not from my email address.

In addition, this respondent noted the emails were not user-friendly, sometimes arriving with broken links going to patient’s junk mail, and requiring patients create a completely new log-in to complete a survey. One respondent suggested REDCap may “make it easier to send mass emails that are individually linked with the patient’s profile; create a prettier or more visually appealing interface for patients.” Furthermore, integrations to link communications to personal calendars were thought to be beneficial (3 codes). Respondents wanted a way to automatically opt out patients from surveys that were being distributed over a period (2 codes). One participant stated they, “would really like to be able to set a flag for opt-out subject [s] when distributing surveys over a period of time. We currently have to remove their emails to prevent future distribution.”

Respondents commented on the “Save and Return” feature (25 codes), which allows patients to leave and return using a unique code to complete the survey at a later time. Although REDCap’s Save and Return feature existed, respondents noted that this feature was often difficult to use (15 codes). They reported that patients may forget or not save their return code or may not know how to return to the survey, resulting in incomplete data or delay in data collection. One respondent commented, “It is not always obvious how to ‘save and return later’ if that is an option or even be aware that that is an option.” Respondents suggested improvements (9 codes) to send the unique save and return code via emails, with reminders and save in invitation logs such that the study staff could provide it to patients if needed. In addition, respondents suggested that improving user-friendliness and patient awareness of this feature could increase response rates and data completion. A participant noted the following:

If they [patients] don’t complete the survey the first time they often forget their return code and lose it. It would really help if the reminder emails had the return code, of if it could be included on the survery [sic] invitation log page that would make it much easier to find and give to the patient.

Results and Data

Respondents liked that REDCap made data exportation easy for storage and analysis purposes (8 codes). Not only was it easy to export data out of the REDCap survey tool, it also made the analysis of the data much easier for the study staff, even those with minimal statistics training. As 1 respondent put it, “[REDCap has a] good translation into a dataset [and] easy statistics for those with minimal statistical training.” Respondents (4 codes) pointed out the need for improving data exports and seamless communication with third-party solutions to send and receive information:

Being able to send to communicate and receive information from other software programs like Clinical Conductor for Demographic information and seamless data uploads.

Respondents perceived that it is easy to create reports and monitor patient responses on REDCap and review specific data points (5 codes). Some respondents provided suggestions to edit charts and graphics as well as being able to share user- or survey-specific data (3 codes). For example, 1 respondent mentioned the following:

Ability for researchers to edit/modify graphics that can be automatically displayed with reports within redcap. This would facilitate researchers’ ability to use those charts.”

Another respondent mentioned, “built in tools to share summary-level data (you vs the whole study) or findings.” While respondents perceived that REDCap allows capturing accurate and complete high-quality data (3 codes), 1 respondent mentioned the following:

As with every self-service data entry portal accuracy of self-service data entry is wildly unreliable. There is real value to having a trained rep assisting the client enter information, when possible.

Training and Support

Respondents reported that REDCap’s active online community and support allowed REDCap users (including administrators and researchers) to find information and answers on how to manage, design, and conduct surveys (8 codes): “...it has a huge user base and a great consortium full of all the information you need to begin administering [surveys].” Respondents mentioned needing REDCap or IT support for patients to complete consent forms or surveys (12 codes). Although support existed for survey designers and administrators, it did not extend to patients completing surveys. Respondents suggested REDCap needed a way to educate or support patients in completing surveys (10 codes) and obtain help via on-demand messaging to study staff members (2 codes). As 1 participant suggested, REDCap should allow patients to “Click icon and get video explaining any information on a field . ” Another participant asked that REDCap have the following:

Dedicated instrument defined support button at the top that takes participants to a page made by the study team where we can put in a zoom room link monitored by study staff, phone numbers, or some pointers on definitions/examples on the instrument.

Respondents also suggested a need for obtaining standardized patient feedback surveys to better engage them and understand their experience (6 codes).

Although some respondents mentioned REDCap required minimal training to get started (7 codes), some respondents (especially REDCap administrators) mentioned the need for training survey designers to set up REDCap tools and surveys to design high-quality surveys (7 codes). When asked about challenges, 1 participant mentioned the following:

Lack of resources for support (in person- phone) and functionality. It is not always easy and takes a lot of time to build tools. Not able to use to its fullest capacity or correctly—basically training ourselves. Library or community network does not help either. Not knowing how to set up properly more complicating functions inhibits usage.

Respondents suggested more information and mandatory training for survey builders, including better guidelines and training videos to enhance builder and patient experience (8 codes). Respondents also perceived that patients taking surveys often do not understand how to fill out surveys or certain questions (8 codes) and having expert survey designers and well-designed surveys could alleviate these concerns (1 code).

Respondents often noted challenges of access to the internet and devices (51 codes) as well as technology literacy (33 codes):

Patients [without] a computer, device, or smart phone may not be able to use REDCap.

As REDCap is web based, data collection could be difficult in rural and low-resource areas due to lack of access to technology (4 codes), such as a computer or reliable internet connection. Another participant noted, “I do work in global health, so our colleagues in resource-limited settings have challenges with the internet connection.”

They also noted REDCap’s ability to integrate with other technologies, such as messaging tools (eg, Twilio) as well as open application programmable interface to be beneficial (11 codes). In comparison, 1 participant noted as a challenge that, “integrating the ReCap [sic] extract with Epic [EHR] data. But once the system is setup it’s easy to maintain.” Respondents suggested integrations with other clinical trial management systems for seamless data transfers and EHRs to conduct surveys or autopopulate patient medical information:

The only other thing that would be super cool is if it could blow surveys into EPIC for documentation when needed.

Respondents also referred to the informed consent capabilities of REDCap (8 codes). Even though they noted the consent module to be advantageous to obtain remote informed consents especially after the COVID-19 pandemic (5 codes), respondents suggested more enhancements, such as a 1-step consent process (3 codes).

Respondents commented positively on the security and compliance of REDCap (15 codes). They reported that HIPAA compliance and the ability to store patient data securely are important advantages of REDCap. One participant commented that “all client data can be stored in one HIPAA compliant platform.”

Respondents mentioned mistrust of technology (3 codes) could make patients feel uncomfortable sharing medical information on web-based platforms. One respondent commented that surveys requiring password protection are difficult for patients. They also provided enhancement suggestions (2 codes) related to maintaining HIPAA compliance, enhancing security, and assuring patients that their health information is safe and secure with REDCap.

Platform Features

Respondents found REDCap advantageous in enabling researchers to collect and patients to provide health data remotely (23 codes):“It has made it much easier for patients to submit their questionnaires and information using an online platform,” especially during and after the COVID-19 pandemic.

Respondents perceived REDCap as a comprehensive or versatile (5 codes) data collection solution noting the following: “It provides us a comprehensive tool for collecting, tracking, and managing patient data and outreach.” They also noted administration and maintenance (3 codes) to be advantageous as REDCap allows “ being able to maintain administrative research tasks together with the data collection .” They noted REDCap’s offline data collection (using REDCap mobile apps) to be challenging (3 codes) and suggested that the offline feature should be improved for better data collection experience (2 codes). In addition, respondents noted the familiarity with REDCap among researchers (2 codes) and seamlessness (1 code) for the study personnel to be advantageous.

In addition, REDCap being available for free to REDCap consortium members was sought to be beneficial (10 codes). While some respondents noted REDCap being simpler and easier than other commercial platforms and paper forms (3 codes), some also noted that REDCap’s interface was not easy to use or user-friendly compared to modern data collection tools (5 codes).

Respondents did not provide inputs with respect to advantages, disadvantages, and enhancement suggestions stating lack of experience or ability to provide inputs or not using REDCap for patient data collection (81 codes). Some nonsensical or unrelated comments lacking information context or irrelevant responses were excluded from the analysis. For example, when asked about enhancement suggestions for REDCap, 1 participant responded, “To REDCap or??.”

This study aimed to identify the advantages, challenges, and future opportunities for enhancements from the perspectives of REDCap administrators and researchers. To the best of our knowledge, this is one of the early studies of user perspectives on REDCap services and features. We believe that the findings of this study will aid REDCap developers and consortium users in better understanding stakeholder needs to enhance and customize REDCap features as well as researchers in improved survey development and data collection.

Principal Findings

Respondents had overwhelmingly positive perceptions of REDCap’s survey design and data collection interface. The vast majority of respondents agreed or strongly agreed that data collected via REDCap were accurate (188/207, 90.8%), reliable (182/207, 87.9%), and complete (166/207, 80.2%). They found REDCap advantageous as it is free for its consortium members, secure, and easy to use. Respondents also perceived REDCap as easy and flexible to create and customize surveys including a variety of response and validation options, which make data collection easier for survey takers. However, respondents pointed out that poor survey design—often attributed to human factors (eg, lengthy forms and lack of knowledge among study staff) or technology limitations (eg, restrictions in survey and visual formatting in REDCap)—could result in poor patient experience and, ultimately, response and completion rates. Optimal design of survey forms is critical for assuring patient comprehension of the forms and accurate data collection [ 27 , 28 ]. Furthermore, direct investigations of REDCap user experiences and preferences could allow better understanding of the need for study staff and patient education. In addition, further research related to user needs for survey development and optimization can lead to enhancing their experience of developing high-quality surveys. One respondent pointed out the following:

It [REDCap] needs a much better understanding of how users engage the questions on a form (e.g., sit and watch users and staff try to figure out acceptable data type entries!). Needs a solid revamping in how it works “out front” and to run a series of user groups—patient and staff.

Although respondents appreciated the availability of REDCap’s community support for administrators and study staff, they pointed out that REDCap has room for improvement in this realm: the tool is not simple to learn, and there is a need for more training of study staff to help develop efficient, unambiguous survey instruments that can enhance patient experience. Poorly designed surveys and questions could potentially lead to incomplete responses and inaccurate data. Respondents pointed out the need for supporting patients, especially to ensure they understand the questions and can obtain help when needed in filling out surveys. Direct help from study staff members to fill out surveys or having the ability to directly send a message to study staff could alleviate misunderstandings and errors in completing surveys. Previous research suggests that the ability to obtain clarifications about survey questions can enhance response accuracy [ 28 ]. Further research and availability of resources are necessary to guide study staff members in creating well-designed instruments. In addition, understanding the factors affecting patients’ experience in completing REDCap surveys and reasons for misunderstanding and errors could also enhance the REDCap experience and health data collection processes.

Opinions on patient experience and usability were more mixed. Most respondents agreed or strongly agreed that patients found REDCap easy to use (90.4%), able to be completed without assistance (79.8%), and able to be completed in a timely manner (87.5%). These strongly positive perceptions of REDCap usability are consistent with a prior study in which 6 out of 7 participants needed no help using REDCap, achieved 71% to 100% task completion, and provided 89% positive reaction words [ 13 ]. Qualitative outcomes showed that respondents perceived REDCap made it convenient for patients to provide data remotely without having to log in or remember credentials. Although they commented that patients can complete REDCap surveys using a device of choice (such as a laptop or mobile), technology access and technology literacy appeared to be a concern. Living in rural or low-income areas also presented issues for survey access. Respondents noted low-resource areas without stable internet access meant data collection was not reliable. Lack of internet access not only meant surveys could not be accessed but also meant the data collection process could be interrupted. REDCap’s MyCap and REDCap Mobile app can allow study staff and patients to complete the collection of data offline, but they were also deemed challenging due to the lack of features compared with the web interface. In a study by Doyle et al [ 19 ], the REDCap mobile interface was less favorably received by participants. Similarly, REDCap’s Save and Return feature allows users to complete surveys at a later time, which could be helpful during poor internet access; however, respondents recommended enhancements in the feature to improve patient experience, specifically an easier way for patients to remember and retrieve the return code. One participant noted this difficulty that patients face in attempting to use the feature:

If they don’t complete the survey the first time, they often forget their return code and lose it. It would really help if the reminder emails had the return code, or if it could be included on the survey invitation [log-in] page...

It is imperative to better understand patient and research participant experience with REDCap in completing surveys via larger and direct studies.

This study identified opportunities to improve the usability of REDCap. Respondents suggested enhancements in the patient-facing survey user interface to be in line with present EDC tools on the market, wanting a sleeker, modern, and cleaner looking interface. A variety of EDC tools are available for health care and non–health care data collection providing modern, device-friendly, and intuitive user interfaces to promote patient engagement [ 29 - 32 ]. In recent years, virtual conversational agents or chatbots have emerged as intuitive and engaging mediums for data collection. Modern data collection tools allow survey designers to develop chatbot-based interactions to collect health data mimicking human-to-human conversations. Studies have shown that individuals prefer chatbot-based conversational data collection experience in comparison to traditional web-based forms [ 33 , 34 ]. Visual and graphical enhancements in REDCap appearance of surveys, patient communication, and researcher interface could support modernization of REDCap-based surveys, thus providing study staff and patients with clear and effective experience of health data collection.

Respondents wanted the mobile interface updated to look more like other commercial products, such as Qualtrics or SurveyMonkey. As more individuals are using mobile devices to obtain health information, it is of great importance to enhance their experience with mobile data collection [ 35 ]. They also suggested that the mobile apps have similar features as the web-based REDCap. Other requests included REDCap to support more languages or a translation service, where surveys could be translated to patients’ preferred languages. Though it has some language capabilities, including Spanish, respondents wanted more language options built into REDCap. In addition, there was concern about the literacy of patients leading to suggestions for REDCap to include tools allowing patients with various literacy levels to access surveys. Respondents suggested inclusion of voice capabilities and more multimedia and gamification features in response options, such as a picture interface where patients could locate their pain visually for researchers. Inclusion of these features could further enhance the experience among patients with higher accessibility needs and low literacy. We also noted that some respondents suggested features that were available within REDCap at the time of conducting the survey. Suggestions included availability of REDCap’s mobile version, embedded fields for responses, and integrations with messaging services such as Twilio. This again points out the need for education among study staff and organizational administrators to enable the optimal and effective use of REDCap features.

Limitations

This study is not without limitations. Although we recruited over 200 respondents, the sample size is small in comparison with the existing user base. We recruited fewer researchers (25/207, 12.1%) than administrators (150/207, 72.5%) who may be more directly involved in survey design and data collection. We also did not ask for participants’ training and experience with REDCap. Future studies should focus on better understanding user perspectives (especially researchers) while also considering the type and amount of REDCap training received by the user. We asked individuals’ opinions that are valuable but may be subject to bias, incomplete recall, or lack of information. For example, we asked information about their institution’s REDCap use, but we did not include a response option or decline responding if they did not have accurate information. We also used REDCap as the platform to conduct our survey, which may have potentially biased responses by familiarizing participants with REDCap more than necessary. Participants’ free-ended responses may have been influenced by how our study was designed or how the features were used. Future, more direct studies are warranted to better understand preferences. We recruited respondents from current REDCap consortium members, who may be more likely to believe REDCap is highly usable, as they may act as REDCap champions within institutions. We may be missing critical information by not capturing the perspectives of people who are not frequent users or consortium members. Future research should capture opinions of novice or past REDCap users. We also did not ask for information about participants’ institutions, REDCap versions and plug-ins used, or institutional policies and customizations. It is possible that participant feedback may be related to institutional requirements or policies. Furthermore, we asked researcher and administrator opinions on patient experience. However, we did not directly assess the patient or research participant experience. Understanding patient experience is important to study in future research. In addition, a comparison of the REDCap experience with other EDC platforms could provide a better understanding of study staff and patient needs. A recent study compared individuals’ experience in completing health forms using REDCap versus a chatbot platform. The results revealed that over 69% of participants preferred a chatbot for data collection with higher usability and net promotor scores for the chatbot [ 33 ]. The chatbot provided superior engagement and interactivity and was perceived as more intuitive than a standard, web-based REDCap interface. Future studies should look into better understanding study staff and patient needs to optimize survey development and data collection experience.

Conclusions

This pilot study aimed to assess stakeholder perspectives on experience with REDCap as an electronic health data collection tool. The findings revealed researchers and administrators perceive REDCap as a valued, low-cost resource that enables them to remotely collect and report health data in a secure and easy way. They also indicated a generally positive health data collection experience by clinical research and care staff members and patients. Although, with the advancements in data collection technologies and availability of interactive and intuitive user interfaces, there is a critical opportunity to enhance the REDCap experience to meet the needs of its vast user base of researchers and patients.

Acknowledgments

Research reported in this publication was supported by the National Library of Medicine of the National Institutes of Health (award number 1R41LM013419-01). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. BB was funded by the National Institute of Mental Health (grant K23MH118482). BW was funded by the National Library of Medicine (grant R41LM013419).

Conflicts of Interest

BW is a shareholder of Doxy.me Inc and Dokbot LLC. All other authors are employees of Doxy.me Inc, a commercial telemedicine company. All authors declare no other conflicts of interest.

Qualitative codebook.

• Caron-Flinterman JF, Broerse JE, Bunders JF. The experiential knowledge of patients: a new resource for biomedical research? Soc Sci Med. Jun 2005;60(11):2575-2584. [ CrossRef ] [ Medline ]
• Hanley B, Truesdale A, King A, Elbourne D, Chalmers I. Involving consumers in designing, conducting, and interpreting randomised controlled trials: questionnaire survey. BMJ. Mar 03, 2001;322(7285):519-523. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Saczynski JS, McManus DD, Goldberg RJ. Commonly used data-collection approaches in clinical research. Am J Med. Nov 2013;126(11):946-950. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Milinovich A, Kattan MW. Extracting and utilizing electronic health data from Epic for research. Ann Transl Med. Feb 2018;6(3):42. [ FREE Full text ] [ CrossRef ] [ Medline ]
• van Velthoven MH, Mastellos N, Majeed A, O'Donoghue J, Car J. Feasibility of extracting data from electronic medical records for research: an international comparative study. BMC Med Inform Decis Mak. Jul 13, 2016;16(1):90. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Sacristan JA, Aguaron A, Avendaño C, Garrido P, Carrion J, Gutierrez A, et al. Patient involvement in clinical research: why, when, and how. Patient Prefer Adherence. Apr 2016;10:631-640. [ CrossRef ]
• van der Scheer L, Garcia E, van der Laan AL, van der Burg S, Boenink M. The benefits of patient involvement for translational research. Health Care Anal. Sep 24, 2017;25(3):225-241. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Summary of the HIPAA privacy rule. U.S. Department of Health and Human Services. URL: https://www.hhs.gov/hipaa/for-professionals/privacy/laws-regulations/index.html [accessed 2021-12-09]
• REDCap homepage. REDCap. URL: https://www.project-redcap.org/ [accessed 2022-05-09]
• Patridge EF, Bardyn TP. Research electronic data capture (REDCap). J Med Libr Assoc. Jan 12, 2018;106(1). [ FREE Full text ] [ CrossRef ]
• REDCap (Research Electronic Data Capture) - Harvard Catalyst. Harvard Catalyst. URL: https://catalyst.harvard.edu/redcap/ [accessed 2024-06-05]
• Reichold M, Heß M, Kolominsky-Rabas P, Gräßel E, Prokosch HU. Usability evaluation of an offline electronic data capture app in a prospective multicenter dementia registry (digiDEM Bayern): mixed method study. JMIR Form Res. Nov 03, 2021;5(11):e31649. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Stambler DM, Feddema E, Riggins O, Campeau K, Breuch LA, Kessler MM, et al. REDCap delivery of a web-based intervention for patients with voice disorders: usability study. JMIR Hum Factors. Mar 25, 2022;9(1):e26461. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kianersi S, Luetke M, Ludema C, Valenzuela A, Rosenberg M. Use of research electronic data capture (REDCap) in a COVID-19 randomized controlled trial: a practical example. BMC Med Res Methodol. Aug 21, 2021;21(1):175. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Tamuhla T, Tiffin N, Allie T. An e-consent framework for tiered informed consent for human genomic research in the global south, implemented as a REDCap template. BMC Med Ethics. Nov 24, 2022;23(1):119. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Wong TC, Captur G, Valeti U, Moon J, Schelbert EB. Feasibility of the REDCap platform for single center and collaborative multicenter CMR research. J Cardiovasc Magn Reson. Jan 16, 2014;16(Supplement 1):P89. [ CrossRef ]
• Chen C, Turner SP, Sholle ET, Brown SW, Blau VL, Brouwer JP, et al. Evaluation of a REDCap-based workflow for supporting federal guidance for electronic informed consent. AMIA Jt Summits Transl Sci Proc. 2019;2019:163-172. [ FREE Full text ] [ Medline ]
• Lee CA, Gamino D, Lore M, Donelson C, Windsor LC. Use of research electronic data capture (REDCap) in a sequential multiple assignment randomized trial (SMART): a practical example of automating double randomization. BMC Med Res Methodol. Jul 06, 2023;23(1):162. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Doyle S, Pavlos R, Carlson SJ, Barton K, Bhuiyan M, Boeing B, et al. Efficacy of digital health tools for a pediatric patient registry: semistructured interviews and interface usability testing with parents and clinicians. JMIR Form Res. Jan 17, 2022;6(1):e29889. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Garcia KK, Abrahão AA. Research development using REDCap software. Healthc Inform Res. Oct 2021;27(4):341-349. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Partners - REDCap. REDCap. URL: https://projectredcap.org/partners/ [accessed 2023-04-21]
• Harris PA, Taylor R, Minor BL, Elliott V, Fernandez M, O'Neal L, et al. The REDCap consortium: building an international community of software platform partners. J Biomed Inform. Jul 2019;95:103208. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. Apr 2009;42(2):377-381. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Braun V, Clarke V. Thematic analysis. In: Cooper H, Camic PM, Long DL, Panter AT, Rindskopf D, Sher KJ, editors. APA Handbook of Research Methods in Psychology, Vol. 2: Research Designs: Quantitative, Qualitative, Neuropsychological, and Biological. Washington, DC. American Psychological Association; 2012:57-71.
• Bernard HR. Research Methods in Anthropology: Qualitative and Quantitative Approaches. Lanham, MD. Rowman & Littlefield Publishers; 2018.
• Bernard HR, Wutich A, Ryan GW. Analyzing Qualitative Data: Systematic Approaches. Thousand Oaks, CA. SAGE Publications; 2016.
• Bargas-Avila JA, Brenzikofer O, Roth SP, Tuch AN, Orsini S, Opwis K. Simple but crucial user interfaces in the world wide web: introducing 20 guidelines for usable web form design. In: Mátrai R, editor. User Interfaces. London, UK. IntechOpen; May 1, 2010.
• Conrad FG, Schober MF. Clarifying survey questions when respondents don't know they need clarification. National Center for Education Statistics. 2001. URL: https://nces.ed.gov/FCSM/pdf/2001FCSM_Conrad.pdf [accessed 2023-04-24]
• ClinCapture homepage. ClinCapture. URL: https://www.clincapture.com/ [accessed 2023-05-09]
• Pawelek J, Baca-Motes K, Pandit JA, Berk BB, Ramos E. The power of patient engagement with electronic health records as research participants. JMIR Med Inform. Jul 08, 2022;10(7):e39145. [ FREE Full text ] [ CrossRef ] [ Medline ]
• SurveyMonkey homepage. SurveyMonkey. URL: https://www.surveymonkey.com/ [accessed 2023-05-09]
• Qualtrics XM: the leading experience management software. Qualtrics XM. URL: https://www.qualtrics.com/ [accessed 2023-05-09]
• Soni H, Ivanova J, Wilczewski H, Bailey A, Ong T, Narma A, et al. Virtual conversational agents versus online forms: patient experience and preferences for health data collection. Front Digit Health. Oct 13, 2022;4:954069. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Ponathil A, Ozkan F, Welch B, Bertrand J, Chalil Madathil K. Family health history collected by virtual conversational agents: an empirical study to investigate the efficacy of this approach. J Genet Couns. Dec 03, 2020;29(6):1081-1092. [ CrossRef ] [ Medline ]
• Heimlich R. More use cell phones to get health information. Pew Research Center. Nov 14, 2012. URL: https://www.pewresearch.org/fact-tank/2012/11/14/more-use-cell-phones-to-get-health-information/ [accessed 2022-04-18]

Abbreviations

Edited by C Lovis; submitted 09.06.23; peer-reviewed by S Wang, C Chen; comments to author 12.03.24; revised version received 10.04.24; accepted 04.05.24; published 25.06.24.

©Hiral Soni, Julia Ivanova, Hattie Wilczewski, Triton Ong, J Nalubega Ross, Alexandra Bailey, Mollie Cummins, Janelle Barrera, Brian Bunnell, Brandon Welch. Originally published in JMIR Medical Informatics (https://medinform.jmir.org), 25.06.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Medical Informatics, is properly cited. The complete bibliographic information, a link to the original publication on https://medinform.jmir.org/, as well as this copyright and license information must be included.