literature review of skill

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• How to Write a Literature Review | Guide, Examples, & Templates

How to Write a Literature Review | Guide, Examples, & Templates

Published on January 2, 2023 by Shona McCombes . Revised on September 11, 2023.

What is a literature review? A literature review is a survey of scholarly sources on a specific topic. It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research that you can later apply to your paper, thesis, or dissertation topic .

There are five key steps to writing a literature review:

• Search for relevant literature
• Evaluate sources
• Identify themes, debates, and gaps
• Outline the structure
• Write your literature review

A good literature review doesn’t just summarize sources—it analyzes, synthesizes , and critically evaluates to give a clear picture of the state of knowledge on the subject.

Instantly correct all language mistakes in your text

Upload your document to correct all your mistakes in minutes

Table of contents

What is the purpose of a literature review, examples of literature reviews, step 1 – search for relevant literature, step 2 – evaluate and select sources, step 3 – identify themes, debates, and gaps, step 4 – outline your literature review’s structure, step 5 – write your literature review, free lecture slides, other interesting articles, frequently asked questions, introduction.

• Quick Run-through
• Step 1 & 2

When you write a thesis , dissertation , or research paper , you will likely have to conduct a literature review to situate your research within existing knowledge. The literature review gives you a chance to:

• Demonstrate your familiarity with the topic and its scholarly context
• Develop a theoretical framework and methodology for your research
• Position your work in relation to other researchers and theorists
• Show how your research addresses a gap or contributes to a debate
• Evaluate the current state of research and demonstrate your knowledge of the scholarly debates around your topic.

Writing literature reviews is a particularly important skill if you want to apply for graduate school or pursue a career in research. We’ve written a step-by-step guide that you can follow below.

Don't submit your assignments before you do this

The academic proofreading tool has been trained on 1000s of academic texts. Making it the most accurate and reliable proofreading tool for students. Free citation check included.

Try for free

Writing literature reviews can be quite challenging! A good starting point could be to look at some examples, depending on what kind of literature review you’d like to write.

• Example literature review #1: “Why Do People Migrate? A Review of the Theoretical Literature” ( Theoretical literature review about the development of economic migration theory from the 1950s to today.)
• Example literature review #2: “Literature review as a research methodology: An overview and guidelines” ( Methodological literature review about interdisciplinary knowledge acquisition and production.)
• Example literature review #3: “The Use of Technology in English Language Learning: A Literature Review” ( Thematic literature review about the effects of technology on language acquisition.)
• Example literature review #4: “Learners’ Listening Comprehension Difficulties in English Language Learning: A Literature Review” ( Chronological literature review about how the concept of listening skills has changed over time.)

You can also check out our templates with literature review examples and sample outlines at the links below.

Download Word doc Download Google doc

Before you begin searching for literature, you need a clearly defined topic .

If you are writing the literature review section of a dissertation or research paper, you will search for literature related to your research problem and questions .

Make a list of keywords

Start by creating a list of keywords related to your research question. Include each of the key concepts or variables you’re interested in, and list any synonyms and related terms. You can add to this list as you discover new keywords in the process of your literature search.

• Social media, Facebook, Instagram, Twitter, Snapchat, TikTok
• Body image, self-perception, self-esteem, mental health
• Generation Z, teenagers, adolescents, youth

Search for relevant sources

Use your keywords to begin searching for sources. Some useful databases to search for journals and articles include:

• Your university’s library catalogue
• Google Scholar
• Project Muse (humanities and social sciences)
• Medline (life sciences and biomedicine)
• EconLit (economics)
• Inspec (physics, engineering and computer science)

You can also use boolean operators to help narrow down your search.

Make sure to read the abstract to find out whether an article is relevant to your question. When you find a useful book or article, you can check the bibliography to find other relevant sources.

You likely won’t be able to read absolutely everything that has been written on your topic, so it will be necessary to evaluate which sources are most relevant to your research question.

For each publication, ask yourself:

• What question or problem is the author addressing?
• What are the key concepts and how are they defined?
• What are the key theories, models, and methods?
• Does the research use established frameworks or take an innovative approach?
• What are the results and conclusions of the study?
• How does the publication relate to other literature in the field? Does it confirm, add to, or challenge established knowledge?
• What are the strengths and weaknesses of the research?

Make sure the sources you use are credible , and make sure you read any landmark studies and major theories in your field of research.

You can use our template to summarize and evaluate sources you’re thinking about using. Click on either button below to download.

Take notes and cite your sources

As you read, you should also begin the writing process. Take notes that you can later incorporate into the text of your literature review.

It is important to keep track of your sources with citations to avoid plagiarism . It can be helpful to make an annotated bibliography , where you compile full citation information and write a paragraph of summary and analysis for each source. This helps you remember what you read and saves time later in the process.

To begin organizing your literature review’s argument and structure, be sure you understand the connections and relationships between the sources you’ve read. Based on your reading and notes, you can look for:

• Trends and patterns (in theory, method or results): do certain approaches become more or less popular over time?
• Themes: what questions or concepts recur across the literature?
• Debates, conflicts and contradictions: where do sources disagree?
• Pivotal publications: are there any influential theories or studies that changed the direction of the field?
• Gaps: what is missing from the literature? Are there weaknesses that need to be addressed?

This step will help you work out the structure of your literature review and (if applicable) show how your own research will contribute to existing knowledge.

• Most research has focused on young women.
• There is an increasing interest in the visual aspects of social media.
• But there is still a lack of robust research on highly visual platforms like Instagram and Snapchat—this is a gap that you could address in your own research.

There are various approaches to organizing the body of a literature review. Depending on the length of your literature review, you can combine several of these strategies (for example, your overall structure might be thematic, but each theme is discussed chronologically).

Chronological

The simplest approach is to trace the development of the topic over time. However, if you choose this strategy, be careful to avoid simply listing and summarizing sources in order.

Try to analyze patterns, turning points and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred.

If you have found some recurring central themes, you can organize your literature review into subsections that address different aspects of the topic.

For example, if you are reviewing literature about inequalities in migrant health outcomes, key themes might include healthcare policy, language barriers, cultural attitudes, legal status, and economic access.

Methodological

If you draw your sources from different disciplines or fields that use a variety of research methods , you might want to compare the results and conclusions that emerge from different approaches. For example:

• Look at what results have emerged in qualitative versus quantitative research
• Discuss how the topic has been approached by empirical versus theoretical scholarship
• Divide the literature into sociological, historical, and cultural sources

Theoretical

A literature review is often the foundation for a theoretical framework . You can use it to discuss various theories, models, and definitions of key concepts.

You might argue for the relevance of a specific theoretical approach, or combine various theoretical concepts to create a framework for your research.

Like any other academic text , your literature review should have an introduction , a main body, and a conclusion . What you include in each depends on the objective of your literature review.

The introduction should clearly establish the focus and purpose of the literature review.

Depending on the length of your literature review, you might want to divide the body into subsections. You can use a subheading for each theme, time period, or methodological approach.

As you write, you can follow these tips:

• Summarize and synthesize: give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: don’t just paraphrase other researchers — add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically evaluate: mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: use transition words and topic sentences to draw connections, comparisons and contrasts

In the conclusion, you should summarize the key findings you have taken from the literature and emphasize their significance.

When you’ve finished writing and revising your literature review, don’t forget to proofread thoroughly before submitting. Not a language expert? Check out Scribbr’s professional proofreading services !

This article has been adapted into lecture slides that you can use to teach your students about writing a literature review.

Scribbr slides are free to use, customize, and distribute for educational purposes.

Open Google Slides Download PowerPoint

If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.

• Sampling methods
• Simple random sampling
• Stratified sampling
• Cluster sampling
• Likert scales
• Reproducibility

 Statistics

• Null hypothesis
• Statistical power
• Probability distribution
• Effect size
• Poisson distribution

Research bias

• Optimism bias
• Cognitive bias
• Implicit bias
• Hawthorne effect
• Anchoring bias
• Explicit bias

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

There are several reasons to conduct a literature review at the beginning of a research project:

• To familiarize yourself with the current state of knowledge on your topic
• To ensure that you’re not just repeating what others have already done
• To identify gaps in knowledge and unresolved problems that your research can address
• To develop your theoretical framework and methodology
• To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

The literature review usually comes near the beginning of your thesis or dissertation . After the introduction , it grounds your research in a scholarly field and leads directly to your theoretical framework or methodology .

A literature review is a survey of credible sources on a topic, often used in dissertations , theses, and research papers . Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other  academic texts , with an introduction , a main body, and a conclusion .

An  annotated bibliography is a list of  source references that has a short description (called an annotation ) for each of the sources. It is often assigned as part of the research process for a  paper .  

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, September 11). How to Write a Literature Review | Guide, Examples, & Templates. Scribbr. Retrieved September 9, 2024, from https://www.scribbr.com/dissertation/literature-review/

Is this article helpful?

Shona McCombes

Other students also liked, what is a theoretical framework | guide to organizing, what is a research methodology | steps & tips, how to write a research proposal | examples & templates, "i thought ai proofreading was useless but..".

I've been using Scribbr for years now and I know it's a service that won't disappoint. It does a good job spotting mistakes”

• StudySkills@Sheffield
• Academic writing skills
• Critical writing

How to write a literature review

Are you writing a literature review as part of a final year project, dissertation, or thesis, or as a standalone piece of work? This page will work through a process of organising and synthesising your sources and then writing a clear and critical final review.

What is a literature review?

A literature review is an account of the current thinking in a specific area of study. Its purpose is to introduce the reader to what has gone before and often to provide you with a foundation that you can build on with your own research. This traditional form of review is sometimes also referred to as a narrative review.

A literature review will often form a section or chapter of a larger piece of research work, such as a dissertation, thesis, or final year project.  It can also be a standalone piece of work.  

A literature review will usually do some or all of the following:

• Introduce the reader to a specific area of interest.
• Organise relevant sources thematically, starting with the more general, broader themes and narrowing towards the most specific themes.
• Introduce key theories relevant to the area of study.
• Define your understanding of important terms or language used in the research.
• Include only the most relevant, important or influential sources, carefully selected. It is about quality not quantity!
• Identify gaps or limitations in existing research.

Considering a body of scholarship as a whole (or in relation to each of your themes) will allow you to 'synthesise' multiple sources and produce an overall summary.

Developing a literature review will help you to develop a level of expertise in your chosen area. By consulting and including a unique combination of sources, you will be able to formulate an informed and original perspective.  Where relevant, this can drive forward your ongoing research.

Writing a Literature Review workshop: book here

A systematic review is a research methodology, often following a standardised and replicable search method and reporting structure that is specific to your discipline. Visit our guidance on systematic reviews for more information.

Organising your sources

As you encounter more and more relevant sources, you will face an ever-expanding amount of reading for yourself. It would take years to read through all of the literature in a specific field from start to finish.

Academic reading, and particularly the process of 'reading around' a topic, is about selective, or targeted reading. Visit our Reading and understanding information Hub to explore approaches to reading for different purposes.

Creating a Literature Matrix can help you to identify the key things that you want to take away from each source. A literature matrix is a simple spreadsheet where you select column titles to suit the aims of your literature review. Are you interested in the research methodology, the scale of the research, the main conclusions, or something else entirely?

Once you have scanned through a source and pulled out the points you are interested in, you can move onto the next source. Organising your reading in this way will also allow you to identify key themes that are emerging in your reading, which you will be able to use later on to plan your review.

You may want to use a reference management tool to help organise and produce your bibliography. Visit the University of Sheffield Library Reference Management pages here .

Make a copy of our Literature matrix template (Google Sheet) and add/delete columns based on the information you want to collect during your search.  Using a spreadsheet means that you can filter and sort your sources, for example, into chronological order, or alphabetically by author.

This downloadable example literature matrix shows how you can lay out your columns.

Synthesising your sources

Once you have a number of sources to work with, you will start to identify key themes emerging. At this point you can start to organise your sources systematically to develop and explore those themes. Can you organise your themes from the broadest to the narrowest and most specific?

A synthesis matrix will help you to identify a thematic structure for your literature review and to understand how the sources that you have found relate to one another. A synthesis matrix is a further spreadsheet that organises your sources by theme and includes a synthesis column, where you can begin to draw out comparisons between the sources. 

Once you have identified a number of sources for each theme in your matrix, you should be able to identify the following:

• Do the sources build on or develop one another? This may be a chronological process.
• Do the sources challenge or contradict one another? Do they reveal a debate within the field?
• Do the sources identify an area of particular interest or a gap in the field?
• Do the sources help to fill in gaps or complete a bigger picture?

Your synthesis column provides an opportunity for you to comment on multiple sources considered as a whole. It is a space for your critical voice and interpretation, which is a key part of writing a successful literature review.

Make a copy of our synthesis matrix (Google Sheet) to organise your themes and plan how the relevant sources can be synthesised.

Download a completed example synthesis matrix from NC State University (PDF, 34Kb)

Visit our Producing a literature review interactive tutorial - for further guidance.

Writing your review

Once you have done the background reading and organised your sources using a synthesis matrix, the job of writing your review is simply about adding flesh to the bones. You will need to write your review as a narrative account, but you can use your matrix as a framework to help you do so.

A literature review will usually follow a simple structure:

• Introduction: what is the overall topic area and how have you broken your review down into themes?
• Theme 1: the broadest, most top-level area (perhaps including some background theory that may have influenced your thinking).
• Theme 2, theme 3, theme 4, etc. Your themes should get progressively more specific and closer to the focus of your research.
• Conclusion: how has this informed your thinking and (if the review is part of a bigger project) what are your research aims and objectives? 

Your review may be broken down by section headings or be a continuous flow with themes clearly separated in a paragraph structure. Each section or paragraph will describe that theme and finish by summarising your overview of a theme (the synthesis part of the matrix above, which includes your critical analysis). 

Our web page How to structure a paragrap h has further guidance to ensure your paragraphs are clear and contain your synthesis and critical analysis.

For advice and feedback on your own review, including referencing, synthesis and academic arguments, please book a writing advisory service appointment.

Make an appointment (student login required)

• How to plan an effective information search
• How to plan a dissertation or final year project
• How to write critically

Use your mySkills portfolio to discover your skillset, reflect on your development, and record your progress.

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

What is a literature review?

A literature review is a critical analysis of the literature related to your research topic. It evaluates and critiques the literature to establish a theoretical framework for your research topic and/or identify a gap in the existing research that your research will address.

A literature review is not a summary of the literature. You need to engage deeply and critically with the literature. Your literature review should show your understanding of the literature related to your research topic and lead to presenting a rationale for your research.

A literature review focuses on:

• the context of the topic
• key concepts, ideas, theories and methodologies
• key researchers, texts and seminal works
• major issues and debates
• identifying conflicting evidence
• the main questions that have been asked around the topic
• the organisation of knowledge on the topic
• definitions, particularly those that are contested
• showing how your research will advance scholarly knowledge (generally referred to as identifying the ‘gap’).

This module will guide you through the functions of a literature review; the typical process of conducting a literature review (including searching for literature and taking notes); structuring your literature review within your thesis and organising its internal ideas; and styling the language of your literature review.

The purposes of a literature review

A literature review serves two main purposes:

1) To show awareness of the present state of knowledge in a particular field, including:

• seminal authors
• the main empirical research
• theoretical positions
• controversies
• breakthroughs as well as links to other related areas of knowledge.

2) To provide a foundation for the author’s research. To do that, the literature review needs to:

• help the researcher define a hypothesis or a research question, and how answering the question will contribute to the body of knowledge;
• provide a rationale for investigating the problem and the selected methodology;
• provide a particular theoretical lens, support the argument, or identify gaps.

Before you engage further with this module, try the quiz below to see how much you already know about literature reviews.

Research and Writing Skills for Academic and Graduate Researchers Copyright © 2022 by RMIT University is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License , except where otherwise noted.

Share This Book

• LEARNING SKILLS
• Writing a Dissertation or Thesis

Researching and Writing a Literature Review

Search SkillsYouNeed:

Learning Skills:

• A - Z List of Learning Skills
• What is Learning?
• Learning Approaches
• Learning Styles
• 8 Types of Learning Styles
• Understanding Your Preferences to Aid Learning
• Lifelong Learning
• Decisions to Make Before Applying to University
• Top Tips for Surviving Student Life
• Living Online: Education and Learning
• 8 Ways to Embrace Technology-Based Learning Approaches
• Critical Thinking Skills
• Critical Thinking and Fake News
• Understanding and Addressing Conspiracy Theories
• Critical Analysis
• Study Skills
• Exam Skills
• How to Write a Research Proposal
• Ethical Issues in Research
• Dissertation: The Introduction
• Writing your Methodology
• Dissertation: Results and Discussion
• Dissertation: Conclusions and Extras

Writing Your Dissertation or Thesis eBook

Part of the Skills You Need Guide for Students .

• Research Methods
• Teaching, Coaching, Mentoring and Counselling
• Employability Skills for Graduates

Subscribe to our FREE newsletter and start improving your life in just 5 minutes a day.

You'll get our 5 free 'One Minute Life Skills' and our weekly newsletter.

We'll never share your email address and you can unsubscribe at any time.

A literature review demonstrates that you have read around your topic and have a broad understanding of previous research, including its limitations.

In the literature review, you summarise the main viewpoints and important facts that you encountered in your reading as they relate to your chosen topic . You will also use the literature review to justify the value of doing research on your topic by showing what is already known, what is not yet known, and how it is relevant.

Your literature review should not simply be descriptive but should also provide a critical analysis of the body of work, and demonstrate that you understand how it fits together as a whole and how your own research fits with previous studies.

A key aspect of a literature review is what sources you select to include, and which you exclude.

Finding Sources

Thanks to the internet, literature searches are now relatively easy, and can be done from the comfort of your own laptop without needing to go anywhere near a library.

However, a word of warning is in order here. The ease with which anybody can access and publish to the internet means that many items published online have not been scrutinised by anybody other than the author.

In other words, content has not necessarily been checked, you have no way of knowing whether the author’s facts and claims are at all accurate and you could easily land yourself in trouble by blindly following or citing from online sources. 

Furthermore, because items on the internet are frequently changed, you may find that something you read yesterday is no longer available in the same form today. However, internet sources can be very useful for up-to-date information, especially current affairs or ongoing or very recent research.

Blogs and sites like the encyclopaedia Wikipedia are particularly prone to these problems.  For these reasons, a general rule of thumb is that you should only rely on internet resources from the websites of organisations whose information you already know to be reputable, like SkillsYouNeed.

See our page: Assessing Internet Information for more.

Do not underestimate how much physical libraries and librarians may be able to help you.

Librarians are usually hugely experienced in using all the search tools and databases, and can often show you much quicker ways of doing things, as well as tips and tricks to help you refine your search.

Furthermore, libraries may have copies of books and academic journals that are not available online. So a trip to your library may prove to be very helpful.

If you haven’t already done so, get yourself an ATHENS account through your university and/or school library. Spend time working out which of the available databases are going to be most useful for your topic, including asking the librarians for advice.

A simple way to get started with finding appropriate materials is simply to ask people who are likely to know.

You might for example ask your tutor or supervisor, or an expert or practitioner working on your chosen topic. Often, they will be able to give you some very helpful ideas about where to begin your reading.

However, be aware that some professionals may find constant requests for information intrudes on their time. Always be courteous and sensitive to the level of demand you may be making on someone’s time.

See our page: Sources of Information for more about the types of resources that you might use and how to access them.

Choosing and Refining your Search Terms

Your search terms are one of the most important elements of finding the right sources for your research project and developing them is an ongoing process.

It’s a good idea to start with a phrase that you think others will have used about the topic, perhaps that you have identified from your lectures and/or earlier study. You will probably find that your first few searches don’t turn up much that’s useful.

Use the one or two articles that you find that are on the right lines to identify alternative search terms, and continue to search until you turn up useful articles.

You can also use a tool such as Google Adword Keyword Research Tool to identify phrases and keywords that are similar to your chosen term(s). This tool is usually used by internet marketing professionals to help them find keywords similar to their own but can be useful for academic research too.

If you’re really struggling to find articles on the right topic, but you’re certain that they must be out there, drop your supervisor a note asking about possible search terms. Tell them what you’ve already used, and ask them for a few alternatives to get you started. However, this should be a last resort, as you don’t want to demonstrate your ignorance too obviously!

Finally, keep searching. You need to read a lot of sources to find the most relevant and will probably end up discarding more than half of what you read. Use abstracts to decide which articles are worth reading, and don’t read those that aren’t relevant: keep checking back to your research questions and decide whether each article is useful. If not, move on.

Critical Reading

Your literature review should not only show that you have been reading a range of materials related to your topic, but also that you have been reading them critically and have thought about the wider contexts and how they apply to your own area of research.

Critical reading is a skill that, like any other skill, is acquired with practice.

In essence, reading critically means that you do not take the claims at face value: you question the basis for claims, why the author may have done and said things in the particular way he or she did, what the wider context is, and whose interests are being served by the claims you encounter.

See our page, Critical Reading for more information.

How Many Sources?

Your university or college supervisor will be able to give you an idea of how many sources you should include in your literature review.

You will probably need to read at least double that number to find enough that are suitable for inclusion. You should also try to find several different sorts of sources: books, journal articles, dissertations, conference papers, working papers, and so on.

You need to make sure that you identify the key texts for the subject. Check a few references, and see which texts are cited most often, or ask the librarians how to use the databases to check how often each article is cited. A good way to identify when you have read enough is if your reading keeps turning up the same points and you’re not learning anything new.

A Note on Dates

There are some theories or articles which are so important in a particular field that they need to be cited, however long ago they were originally published. But those apart, you should generally prefer more recent sources published in the last five or ten years. As a rough guide, the balance of publication dates should be about two thirds from the last 10 years, and no more than one third older than that.

Writing your Literature Review

In general, your literature review should start with one or two broad paragraphs, demonstrating your understanding of the breadth of your area of study.

You should then discuss the literature that deals with your area of research and, finally, consider and critique the studies that are most directly relevant.

You should spend most time on the latter.

Writing your literature review should be an iterative process.

The best way to do it is probably to summarise each source as you go along, referencing it carefully, and grouping your sources by themes.

You will almost certainly find that the themes develop as you go along, and so do your search terms. Use headings to store your summaries and then write a more polished section under that heading when you have enough sources to be able to ‘compare and contrast’ opposing views, and particularly to draw out areas where there is disagreement and/or conflicting evidence as these are the most fruitful for further research.

Where there are gaps, you can then go back and search for more sources on that area. The best literature reviews are not only descriptive, but draw together similar thinking and provide a critical analysis of the previous research, including highlighting really good studies, or identifying flaws and gaps.

To make sure that you carrying out a critical analysis, make sure that you ask yourself the question ‘ Do I agree with this viewpoint? Why? ’, and also consider whether the methods used are strong or weak and why. This will also help you to decide on your own methodology.

Another way of checking whether you are evaluating or merely describing is to look at whether you have discussed work chronologically (likely to be descriptive) or in terms of whether there is general agreement on a topic (much more likely to be evaluative).

Checklist of Questions for Critical Reading

Ask yourself the following questions to decide whether or not a particular piece of work is worth including in your literature review.

• Who is the author? What can I find out about him/her? Has he/she written other books, articles etc.?
• What is the author’s position in the research process, e.g., gender, class, politics, life experience, relationship to research participants?
• Where and when was the document produced? What type of document is it?
• Is it reporting original research that the author has done, or is it presenting second-hand information about a topic?
• Is it formal or informal?
• Is it 'authoritative' (e.g., academic, scientific) or 'popular' (newspaper or magazine article)?
• How has it been produced? Is it glossy, with lots of pictures, diagrams, etc.?
• If it is contained on a website, is the website from a reputable organisation, or is the document drawn from some other reputable source?

The Message

• What is being said?
• What is not being said?
• How is the argument presented? Why?
• What use has been made of diagrams, pictures etc.?
• Who was or is the intended audience?
• Whose interests are being served by this message? Are there political implications, for instance?
• What evidence is presented to support the claims that are made?
• Does the evidence actually support the claims? Is the evidence presented in enough detail for you to make up your own mind whether you agree with the claims?
• Are there errors or inconsistencies?
• What is the significance to my topic and the research that I wish to carry out?

Your literature review should also demonstrate how your study does or will relate to previous work, and how it either fills gaps, or responds to calls for further work.

Your literature review will help you to refine your research question. It should also help you to explain how your methodology fits with previous work, and help you to identify and evaluate possible research methods.

A Note on Tense

When you are describing someone’s findings or opinions, it is probably best to use the past tense.

For example:

“Jones (2001) argued that…”.

Many authors of academic papers prefer the present tense when describing opinions or views (“Jones (2001) argues that…”). However, it is always possible that Jones has subsequently changed his/her view, and therefore the past tense is preferable.

The past tense is always going to be correct for something that was expressed in the past; the present tense may no longer be true.

Citations and References

Your university will almost certainly have a preferred style for citations and references that you will need to use. Make sure you understand how this works before you start writing your literature review and use it consistently throughout.

Keep your references up to date as you go, and make sure that you always cite the reference as you write: it’s much easier than trying to build a reference list at the end.

See our page on Academic Referencing for more information

For scientific subjects, Vancouver (numerical) referencing is often preferred.

However, it is much harder to check that your references are correct using this system. It is therefore better to use a (name, date) system of citations until you are certain that you have finished revising the document.

Alternatively, use a system of end-notes which will automatically update the numbering if you move a citation as you will otherwise end up hopelessly confused.

Draft, Draft and Redraft

Finally, once you have written each section by theme, go back and read the whole thing to check that the sections flow logically one from another, and that the whole literature review reads sensibly and coherently.

As with any essay or extended piece of writing, editing and redrafting will improve the quality of your writing, as will asking someone else to read it over and check for errors or inconsistencies.

You should also do a search to check for consistent use of British or American spellings (-ise and -ize, for example), double spaces after words, and double/single inverted commas around quotations. You might think such details are less important than the content, but the marker may not share your view.

Continue to: Writing a Research Proposal Writing a Dissertation: The Introduction

See also: Writing a Methodology Dissertation Results and Discussion Dissertation Conclusions and Extra Sections

Systematic Literature Review of E-Learning Capabilities to Enhance Organizational Learning

• Open access
• Published: 01 February 2021
• Volume 24 , pages 619–635, ( 2022 )

Cite this article

You have full access to this open access article

• Michail N. Giannakos 1 ,
• Patrick Mikalef 1 &
• Ilias O. Pappas   ORCID: orcid.org/0000-0001-7528-3488 1 , 2  

23k Accesses

39 Citations

Explore all metrics

E-learning systems are receiving ever increasing attention in academia, business and public administration. Major crises, like the pandemic, highlight the tremendous importance of the appropriate development of e-learning systems and its adoption and processes in organizations. Managers and employees who need efficient forms of training and learning flow within organizations do not have to gather in one place at the same time or to travel far away to attend courses. Contemporary affordances of e-learning systems allow users to perform different jobs or tasks for training courses according to their own scheduling, as well as to collaborate and share knowledge and experiences that result in rich learning flows within organizations. The purpose of this article is to provide a systematic review of empirical studies at the intersection of e-learning and organizational learning in order to summarize the current findings and guide future research. Forty-seven peer-reviewed articles were collected from a systematic literature search and analyzed based on a categorization of their main elements. This survey identifies five major directions of the research on the confluence of e-learning and organizational learning during the last decade. Future research should leverage big data produced from the platforms and investigate how the incorporation of advanced learning technologies (e.g., learning analytics, personalized learning) can help increase organizational value.

Similar content being viewed by others

Technology-Enhanced Organizational Learning: A Systematic Literature Review

Knowledge Transfer Through E-learning: Case of Tunisian Post

Organizational e-Learning Systems’ Success in Industry

Explore related subjects.

• Artificial Intelligence

Avoid common mistakes on your manuscript.

1 Introduction

E-learning covers the integration of information and communication technology (ICT) in environments with the main goal of fostering learning (Rosenberg and Foshay 2002 ). The term “e-learning” is often used as an umbrella term to portray several modes of digital learning environments (e.g., online, virtual learning environments, social learning technologies). Digitalization seems to challenge numerous business models in organizations and raises important questions about the meaning and practice of learning and development (Dignen and Burmeister 2020 ). Among other things, the digitalization of resources and processes enables flexible ways to foster learning across an organization’s different sections and personnel.

Learning has long been associated with formal or informal education and training. However organizational learning is much more than that. It can be defined as “a learning process within organizations that involves the interaction of individual and collective (group, organizational, and inter-organizational) levels of analysis and leads to achieving organizations’ goals” (Popova-Nowak and Cseh 2015 ) with a focus on the flow of knowledge across the different organizational levels (Oh 2019 ). Flow of knowledge or learning flow is the way in which new knowledge flows from the individual to the organizational level (i.e., feed forward) and vice versa (i.e., feedback) (Crossan et al. 1999 ; March 1991 ). Learning flow and the respective processes constitute the cornerstone of an organization’s learning activities (e.g., from physical training meetings to digital learning resources), they are directly connected to the psycho-social experiences of an organization’s members, and they eventually lead to organizational change (Crossan et al. 2011 ). The overall organizational learning is extremely important in an organization because it is associated with the process of creating value from an organizations’ intangible assets. Moreover, it combines notions from several different domains, such as organizational behavior, human resource management, artificial intelligence, and information technology (El Kadiri et al. 2016 ).

A growing body of literature lies at the intersection of e-learning and organizational learning. However, there is limited work on the qualities of e-learning and the potential of its qualities to enhance organizational learning (Popova-Nowak and Cseh 2015 ). Blockages and disruptions in the internal flow of knowledge is a major reason why organizational change initiatives often fail to produce their intended results (Dee and Leisyte 2017 ). In recent years, several models of organizational learning have been published (Berends and Lammers 2010 ; Oh 2019 ). However, detailed empirical studies indicate that learning does not always proceed smoothly in organizations; rather, the learning meets interruptions and breakdowns (Engeström et al. 2007 ).

Discontinuities and disruptions are common phenomena in organizational learning (Berends and Lammers 2010 ), and they stem from various causes. For example, organizational members’ low self-esteem, unsupportive technology and instructors (Garavan et al. 2019 ), and even crises like the Covid-19 pandemic can result in demotivated learners and overall unwanted consequences for their learning (Broadbent 2017 ). In a recent conceptual article, Popova-Nowak and Cseh ( 2015 ) emphasized that there is a limited use of multidisciplinary perspectives to investigate and explain the processes and importance of utilizing the available capabilities and resources and of creating contexts where learning is “attractive to individual agents so that they can be more engaged in exploring ways in which they can contribute through their learning to the ongoing renewal of organizational routines and practices” (Antonacopoulou and Chiva 2007 , p. 289).

Despite the importance of e-learning, the lack of systematic reviews in this area significantly hinders research on the highly promising value of e-learning capabilities for efficiently supporting organizational learning. This gap leaves practitioners and researchers in uncharted territories when faced with the task of implementing e-learning designs or deciding on their digital learning strategies to enhance the learning flow of their organizations. Hence, in order to derive meaningful theoretical and practical implications, as well as to identify important areas for future research, it is critical to understand how the core capabilities pertinent to e-learning possess the capacity to enhance organizational learning.

In this paper, we define e-learning enhanced organizational learning (eOL) as the utilization of digital technologies to enhance the process of improving actions through better knowledge and understanding in an organization. In recent years, a significant body of research has focused on the intersection of e-learning and organizational learning (e.g., Khandakar and Pangil 2019 ; Lin et al. 2019 ; Menolli et al. 2020 ; Turi et al. 2019 ; Xiang et al. 2020 ). However, there is a lack of systematic work that summarizes and conceptualizes the results in order to support organizations that want to move from being information-based enterprises to being knowledge-based ones (El Kadiri et al. 2016 ). In particular, recent technological advances have led to an increase in research that leverages e-learning capacities to support organizational learning, from virtual reality (VR) environments (Costello and McNaughton 2018 ; Muller Queiroz et al. 2018 ) to mobile computing applications (Renner et al. 2020 ) to adaptive learning and learning analytics (Zhang et al. 2019 ). These studies support different skills, consider different industries and organizations, and utilize various capacities while focusing on various learning objectives (Garavan et al. 2019 ). Our literature review aims to tease apart these particularities and to investigate how these elements have been utilized over the past decade in eOL research. Therefore, in this review we aim to answer the following research questions (RQs):

RQ1: What is the status of research at the intersection of e-learning and organizational learning, seen through the lens of areas of implementation (e.g., industries, public sector), technologies used, and methodologies (e.g., types of data and data analysis techniques employed)?

RQ2: How can e-learning be leveraged to enhance the process of improving actions through better knowledge and understanding in an organization?

Our motivation for this work is based on the emerging developments in the area of learning technologies that have created momentum for their adoption by organizations. This paper provides a review of research on e-learning capabilities to enhance organizational learning with the purpose of summarizing the findings and guiding future studies. This study can provide a springboard for other scholars and practitioners, especially in the area of knowledge-based enterprises, to examine e-learning approaches by taking into consideration the prior and ongoing research efforts. Therefore, in this paper we present a systematic literature review (SLR) (Kitchenham and Charters 2007 ) on the confluence of e-learning and organizational learning that uncovers initial findings on the value of e-learning to support organizational learning while also delineating several promising research streams.

The rest of this paper is organized as follows. In the next section, we present the related background work. The third section describes the methodology used for the literature review and how the studies were selected and analyzed. The fourth section presents the research findings derived from the data analysis based on the specific areas of focus. In the fifth section, we discuss the findings, the implications for practice and research, and the limitations of the selected methodological approach. In the final section, we summarize the conclusions from the study and make suggestions for future work.

2 Background and Related Work

2.1 e-learning systems.

E-learning systems provide solutions that deliver knowledge and information, facilitate learning, and increase performance by developing appropriate knowledge flow inside organizations (Menolli et al. 2020 ). Putting into practice and appropriately managing technological solutions, processes, and resources are necessary for the efficient utilization of e-learning in an organization (Alharthi et al. 2019 ). Examples of e-learning systems that have been widely adopted by various organizations are Canvas, Blackboard, and Moodle. Such systems provide innovative services for students, employees, managers, instructors, institutions, and other actors to support and enhance the learning processes and facilitate efficient knowledge flow (Garavan et al. 2019 ). Functionalities, such as creating modules to organize mini course information and learning materials or communication channels such as chat, forums, and video exchange, allow instructors and managers to develop appropriate training and knowledge exchange (Wang et al. 2011 ). Nowadays, the utilization of various e-learning capabilities is a commodity for supporting organizational and workplace learning. Such learning refers to training or knowledge development (also known in the literature as learning and development, HR development, and corporate training: Smith and Sadler-Smith 2006 ; Garavan et al. 2019 ) that takes place in the context of work.

Previous studies have focused on evaluating e-learning systems that utilize various models and frameworks. In particular, the development of maturity models, such as the e-learning capability maturity model (eLCMM), addresses technology-oriented concerns (Hammad et al. 2017 ) by overcoming the limitations of the domain-specific models (e.g., game-based learning: Serrano et al.  2012 ) or more generic lenses such as the e-learning maturity model (Marshall 2006 ). The aforementioned models are very relevant since they focus on assessing the organizational capabilities for sustainably developing, deploying, and maintaining e-learning. In particular, the eLCMM focuses on assessing the maturity of adopting e-learning systems and adds a feedback building block for improving learners’ experiences (Hammad et al. 2017 ). Our proposed literature review builds on the previously discussed models, lenses, and empirical studies, and it provides a review of research on e-learning capabilities with the aim of enhancing organizational learning in order to complement the findings of the established models and guide future studies.

E-learning systems can be categorized into different types, depending on their functionalities and affordances. One very popular e-learning type is the learning management system (LMS), which includes a virtual classroom and collaboration capabilities and allows the instructor to design and orchestrate a course or a module. An LMS can be either proprietary (e.g., Blackboard) or open source (e.g., Moodle). These two types differ in their features, costs, and the services they provide; for example, proprietary systems prioritize assessment tools for instructors, whereas open-source systems focus more on community development and engagement tools (Alharthi et al. 2019 ). In addition to LMS, e-learning systems can be categorized based on who controls the pace of learning; for example, an institutional learning environment (ILE) is provided by the organization and is usually used for instructor-led courses, while a personal learning environment (PLE) is proposed by the organization and is managed personally (i.e., learner-led courses). Many e-learning systems use a hybrid version of ILE and PLE that allows organizations to have either instructor-led or self-paced courses.

Besides the controlled e-learning systems, organizations have been using environments such as social media (Qi and Chau 2016 ), massive open online courses (MOOCs) (Weinhardt and Sitzmann 2018 ) and other web-based environments (Wang et al. 2011 ) to reinforce their organizational learning potential. These systems have been utilized through different types of technology (e.g., desktop applications, mobile) that leverage the various capabilities offered (e.g., social learning, VR, collaborative systems, smart and intelligent support) to reinforce the learning and knowledge flow potential of the organization. Although there is a growing body of research on e-learning systems for organizational learning due to the increasingly significant role of skills and expertise development in organizations, the role and alignment of the capabilities of the various e-learning systems with the expected competency development remains underexplored.

2.2 Organizational Learning

There is a large body of research on the utilization of technologies to improve the process and outcome dimensions of organizational learning (Crossan et al. 1999 ). Most studies have focused on the learning process and on the added value that new technologies can offer by replacing some of the face-to-face processes with virtual processes or by offering new, technology-mediated phases to the process (Menolli et al. 2020 ; Lau 2015 ) highlighted how VR capabilities can enhance organizational learning, describing the new challenges and frameworks needed in order to effectively utilize this potential. In the same vein, Zhang et al. ( 2017 ) described how VR influences reflective thinking and considered its indirect value to overall learning effectiveness. In general, contemporary research has investigated how novel technologies and approaches have been utilized to enhance organizational learning, and it has highlighted both the promises and the limitations of the use of different technologies within organizations.

In many organizations, alignment with the established infrastructure and routines, and adoption by employees are core elements for effective organizational learning (Wang et al. 2011 ). Strict policies, low digital competence, and operational challenges are some of the elements that hinder e-learning adoption by organizations (Garavan et al. 2019 ; Wang 2018 ) demonstrated the importance of organizational, managerial, and job support for utilizing individual and social learning in order to increase the adoption of organizational learning. Other studies have focused on the importance of communication through different social channels to develop understanding of new technology, to overcome the challenges employees face when engaging with new technology, and, thereby, to support organizational learning (Menolli et al. 2020 ). By considering the related work in the area of organizational learning, we identified a gap in aligning an organization’s learning needs with the capabilities offered by the various technologies. Thus, systematic work is needed to review e-learning capabilities and how these capabilities can efficiently support organizational learning.

2.3 E-learning Systems to Enhance Organizational Learning

When considering the interplay between e-learning systems and organizational learning, we observed that a major challenge for today’s organizations is to switch from being information-based enterprises to become knowledge-based enterprises (El Kadiri et al. 2016 ). Unidirectional learning flows, such as formal and informal training, are important but not sufficient to cover the needs that enterprises face (Manuti et al. 2015 ). To maintain enterprises’ competitiveness, enterprise staff have to operate in highly intense information and knowledge-oriented environments. Traditional learning approaches fail to substantiate learning flow on the basis of daily evidence and experience. Thus, novel, ubiquitous, and flexible learning mechanisms are needed, placing humans (e.g., employees, managers, civil servants) at the center of the information and learning flow and bridging traditional learning with experiential, social, and smart learning.

Organizations consider lack of skills and competences as being the major knowledge-related factors hampering innovation (El Kadiri et al. 2016 ). Thus, solutions need to be implemented that support informal, day-to-day, and work training (e.g., social learning, collaborative learning, VR/AR solutions) in order to develop individual staff competences and to upgrade the competence affordances at the organizational level. E-learning-enhanced organizational learning has been delivered primarily in the form of web-based learning (El Kadiri et al. 2016 ). More recently, the TEL tools portfolio has rapidly expanded to make more efficient joint use of novel learning concepts, methodologies, and technological enablers to achieve more direct, effective, and lasting learning impacts. Virtual learning environments, mobile-learning solutions, and AR/VR technologies and head-mounted displays have been employed so that trainees are empowered to follow their own training pace, learning topics, and assessment tests that fit their needs (Costello and McNaughton 2018 ; Mueller et al. 2011 ; Muller Queiroz et al. 2018 ). The expanding use of social networking tools has also brought attention to the contribution of social and collaborative learning (Hester et al. 2016 ; Wei and Ram 2016 ).

Contemporary learning systems supporting adaptive, personalized, and collaborative learning expand the tools available in eOL and contribute to the adoption, efficiency, and general prospects of the introduction of TEL in organizations (Cheng et al. 2011 ). In recent years, eOL has emphasized how enterprises share knowledge internally and externally, with particular attention being paid to systems that leverage collaborative learning and social learning functionalities (Qi and Chau 2016 ; Wang  2011 ). This is the essence of computer-supported collaborative learning (CSCL). The CSCL literature has developed a framework that combines individual learning, organizational learning, and collaborative learning, facilitated by establishing adequate learning flows and emerges effective learning in an enterprise learning (Goggins et al. 2013 ), in Fig.  1 .

Representation of the combination of enterprise learning and knowledge flows. (adapted from Goggins et al. 2013 )

Establishing efficient knowledge and learning flows is a primary target for future data-driven enterprises (El Kadiri et al. 2016 ). Given the involved knowledge, the human resources, and the skills required by enterprises, there is a clear need for continuous, flexible, and efficient learning. This can be met by contemporary learning systems and practices that provide high adoption, smooth usage, high satisfaction, and close alignment with the current practices of an enterprise. Because the required competences of an enterprise evolve, the development of competence models needs to be agile and to leverage state-of-the art technologies that align with the organization’s processes and models. Therefore, in this paper we provide a review of the eOL research in order to summarize the findings, identify the various capabilities of eOL, and guide the development of organizational learning in future enterprises as well as in future studies.

3 Methodology

To answer our research questions, we conducted an SLR, which is a means of evaluating and interpreting all available research relevant to a particular research question, topic area, or phenomenon of interest. A SLR has the capacity to present a fair evaluation of a research topic by using a trustworthy, rigorous, and auditable methodology (Kitchenham and Charters 2007 ). The guidelines used (Kitchenham and Charters 2007 ) were derived from three existing guides adopted by medical researchers. Therefore, we adopted SLR guidelines that follow transparent and widely accepted procedures (especially in the area of software engineering and information systems, as well as in e-learning), minimize potential bias (researchers), and support reproducibility (Kitchenham and Charters 2007 ). Besides the minimization of bias and support for reproducibility, an SLR allows us to provide information about the impact of some phenomenon across a wide range of settings, contexts, and empirical methods. Another important advantage is that, if the selected studies give consistent results, SLRs can provide evidence that the phenomenon is robust and transferable (Kitchenham and Charters 2007 ).

3.1 Article Collection

Several procedures were followed to ensure a high-quality review of the literature of eOL. A comprehensive search of peer-reviewed articles was conducted in February 2019 (short papers, posters, dissertations, and reports were excluded), based on a relatively inclusive range of key terms: “organizational learning” & “elearning”, “organizational learning” & “e-learning”, “organisational learning” & “elearning”, and “organisational learning” & “e-learning”. Publications were selected from 2010 onwards, because we identified significant advances since 2010 (e.g., MOOCs, learning analytics, personalized learning) in the area of learning technologies. A wide variety of databases were searched, including SpringerLink, Wiley, ACM Digital Library, IEEE Xplore, Science Direct, SAGE, ERIC, AIS eLibrary, and Taylor & Francis. The selected databases were aligned with the SLR guidelines (Kitchenham and Charters 2007 ) and covered the major venues in IS and educational technology (e.g., a basket of eight IS journals, the top 20 journals in the Google Scholar IS subdiscipline, and the top 20 journals in the Google Scholar Educational Technology subdiscipline). The search process uncovered 2,347 peer-reviewed articles.

3.2 Inclusion and Exclusion Criteria

The selection phase determines the overall validity of the literature review, and thus it is important to define specific inclusion and exclusion criteria. As Dybå and Dingsøyr ( 2008 ) specified, the quality criteria should cover three main issues – namely, rigor, credibility, and relevance – that need to be considered when evaluating the quality of the selected studies. We applied eight quality criteria informed by the proposed Critical Appraisal Skills Programme (CASP) and related works (Dybå and Dingsøyr 2008 ). Table 1 presents these criteria.

Therefore, studies were eligible for inclusion if they were focused on eOL. The aforementioned criteria were applied in stages 2 and 3 of the selection process (see Fig.  2 ), when we assessed the papers based on their titles and abstracts, and read the full papers. From March 2020, we performed an additional search (stage 4) following the same process for papers published after the initial search period (i.e., 2010–February 2019). The additional search returned seven papers. Figure 2 summarizes the stages of the selection process.

Stages of the selection process

3.3 Analysis

Each collected study was analyzed based on the following elements: study design (e.g., experiment, case study), area (e.g., IT, healthcare), technology (e.g., wiki, social media), population (e.g., managers, employees), sample size, unit of analysis (individual, firm), data collections (e.g., surveys, interviews), research method, data analysis, and the main research objective of the study. It is important to highlight that the articles were coded based on the reported information, that different authors reported information at different levels of granularity (e.g., an online system vs. the name of the system), and that in some cases the information was missing from the paper. Overall, we endeavored to code the articles as accurately and completely as possible.

The coding process was iterative with regular consensus meetings between the two researchers involved. The primary coder prepared the initial coding for a number of articles and both coders reviewed and agreed on the coding in order to reach the final codes presented in the Appendix . Disagreements between the coders and inexplicit aspects of the reviewed papers were discussed and resolved in regular consensus meetings. Although this process did not provide reliability indices (e.g., Cohen’s kappa), it did provide certain reliability in terms of consistency of the coding and what Krippendorff ( 2018 ) stated as the reliability of “the degree to which members of a designated community concur on the readings, interpretations, responses to, or uses of given texts or data”, which is considered acceptable research practice (McDonald et al. 2019 ).

In this section, we present the detailed results of the analysis of the 47 papers. Analysis of the studies was performed using non-statistical methods that considered the variables reported in the Appendix . This section is followed by an analysis and discussion of the categories.

4.1 Sample Size and Population Involved

The categories related to the sample of the articles and included the number of participants in each study (size), their position (e.g., managers, employees), and the area/topic covered by the study. The majority of the studies involved employees (29), with few studies involving managers (6), civil servants (2), learning specialists (2), clients, and researchers. Regarding the sample size, approximately half of the studies (20) were conducted with fewer than 100 participants; some (12) can be considered large-scale studies (more than 300 participants); and only a few (9) can be considered small scale (fewer than 20 participants). In relation to the area/topic of the study, most studies (11) were conducted in the context of the IT industry, but there was also good coverage of other important areas (i.e., healthcare, telecommunications, business, public sector). Interestingly, several studies either did not define the area or were implemented in a generic context (sector-agnostic studies, n = 10), and some studies were implemented in a multi-sector context (e.g., participants from different sections or companies, n = 4).

4.2 Research Methods

When assessing the status of research for an area, one of the most important aspects is the methodology used. By “method” in the Appendix , we refer to the distinction between quantitative, qualitative, and mixed methods research. In addition to the method, in our categorization protocol we also included “study design” to refer to the distinction between survey studies (i.e., those that gathered data by asking a group of participants), experiments (i.e., those that created situations to record beneficial data), and case studies (i.e., those that closely studied a group of individuals).

Based on this categorization, the Appendix shows that the majority of the papers were quantitative (34) and qualitative (7), with few studies (6) utilizing mixed methods. Regarding the study design, most of the studies were survey studies (26), 13 were case studies, and fewer were experiments (8). For most studies, the individual participant (40) was the unit of analysis, with few studies having the firm as the unit of analysis, and only one study using the training session as a unit of analysis. Regarding the measures used in the studies, most utilized surveys (39), with 11 using interviews, and only a few studies using field notes from focus groups (2) and log files from the systems (2). Only eight studies involved researchers using different measures to triangulate or extend their findings. Most articles used structural equation modeling (SEM) (17) to analyze their data, with 13 studies employing descriptive statistics, seven using content analysis, nine using regression analysis or analyses of variances/covariance, and one study using social network analysis (SNA).

4.3 Technologies

Concerning the technology used, most of the studies (17) did not study a specific system, referring instead in their investigation to a generic e-learning or technological solution. Several studies (9) named web-based learning environments, without describing the functionalities of the identified system. Other studies focused on online learning environments (4), collaborative learning systems (3), social learning systems (3), smart learning systems (2), podcasting (2), with the rest of the studies using a specific system (e.g., a wiki, mobile learning, e-portfolios, Second Life, web application).

4.4 Research Objectives

The research objectives of the studies could be separated into six main categories. The first category focuses on the intention of the employees to use the technology (9); the second focuses on the performance of the employees (8); the third focuses on the value/outcome for the organization (4); the fourth focuses on the actual usage of the system (7); the fifth focuses on employees’ satisfaction (4); and the sixth focuses on the ability of the proposed system to foster learning (9). In addition to these six categories, we also identified studies that focused on potential barriers for eOL in organizations (Stoffregen et al. 2016 ), the various benefits associated with the successful implementation of eOL (Liu et al. 2012 ), the feasibility of eOL (Kim et al. 2014 ; Mueller et al. 2011 ), and the alignment of the proposed innovation with the other processes and systems in the organization (Costello and McNaughton 2018 ).

4.5 E-learning Capabilities in Various Organizations and for Various Objectives

The technology used has an inherent role for both the organization and the expected eOL objective. E-learning systems are categorized based on their functionalities and affordances. Based on the information reported in the selected papers, we ranked them based on the different technologies and functionalities (e.g., collaborative, online, smart). To do so, we focused on the main elements described in the selected paper; for instance, a paper that described the system as wiki-based or indicated that the system was Second Life was ranked as such, rather than being added to collaborative systems or social learning respectively. We did this because we wanted to capture all the available information since it gave us additional insights (e.g., Second Life is both a social and a VR system).

To investigate the connection between the various technologies used to enhance organizational learning and their application in the various organizations, we utilized the coding (see Appendix ) and mapped the various e-learning technologies (or their affordances) with the research industries to which they applied (Fig.  3 ). There was occasionally a lack of detailed information about the capabilities of the e-learning systems applied (e.g., generic, or a web application, or an online system), which limited the insights. Figure 3 provides a useful mapping of the confluence of e-learning technologies and their application in the various industries.

Association of the different e-learning technologies with the industries to which they are applied in the various studies. Note: The size of the circles depicts the frequency of studies, with the smallest circle representing one study and the largest representing six studies. The mapping is extracted from the data in the Appendix , which outlines the papers that belong in each of the circles

To investigate the connection between the various technologies used to enhance organizational learning and their intended objectives, we utilized the coding of the articles (see Appendix ) and mapped the various e-learning technologies (or their affordances) with the intended objectives, as reported in the various studies (Fig.  4 ). The results in Fig.  4 show the objectives that are central in eOL research (e.g., performance, fostering learning, adoption, and usage) as well as those objectives on which few studies have focused (e.g., alignment, feasibility, behavioral change). In addition, the results also indicate the limited utilization of the various e-learning capabilities (e.g., social, collaborative, smart) to achieve objectives connected with those capabilities (e.g., social learning and behavioral change, collaborative learning, and barriers).

Association of the different e-learning technologies with the objectives investigated in the various studies. Note: The size of the circles depicts the frequency of studies, with the smallest circle representing one study and the largest representing five studies. The mapping is extracted from the data in the Appendix , which outlines the papers that belong in each of the circles

5 5. Discussion

After reviewing the 47 identified articles in the area of eOL, we can observe that all the works acknowledge the importance of the affordances offered by different e-learning technologies (e.g., remote collaboration, anytime anywhere), the importance of the relationship between eOL and employees’ satisfaction and performance, and the benefits associated with organizational value and outcome. Most of the studies agree that eOL provides employees, managers, and even clients with opportunities to learn in a more differentiated manner, compared to formal and face-to-face learning. However, how the organization adopts and puts into practice these capabilities to leverage them and achieve its goals are complex and challenging procedures that seem to be underexplored.

Several studies (Lee et al. 2015a ; Muller Queiroz et al. 2018 ; Tsai et al. 2010 ) focused on the positive effect of perceived managerial support, perceived usefulness, perceived ease of use, and other technology acceptance model (TAM) constructs of the e-learning system in supporting all three levels of learning (i.e., individual, collaborative, and organizational). Another interesting dimension highlighted by many studies (Choi and Ko 2012 ; Khalili et al. 2012 ; Yanson and Johnson 2016 ) is the role of socialization in the adoption and usage of the e-learning systems that offer these capabilities. Building connections and creating a shared learning space in the e-learning system is challenging but also critical for the learners (Yanson and Johnson 2016 ). This is consistent with the expectancy-theoretical explanation of how social context impacts on employees’ motivation to participate in learning (Lee et al. 2015a ; Muller Queiroz et al. 2018 ).

The organizational learning literature suggests that e-learning may be more appropriate for the acquisition of certain types of knowledge than others (e.g., procedural vs. declarative, or hard-skills vs. soft-skills); however, there is no empirical evidence for this (Yanson and Johnson 2016 ). To advance eOL research, there is a need for a significant move to address complex, strategic skills by including learning and development professionals (Garavan et al. 2019 ) and by developing strategic relationships. Another important element is to utilize e-learning technology that addresses and integrates organizational, individual, and social perspectives in eOL (Wang  2011 ). This is also identified in our literature review since we found only limited specialized e-learning systems in domain areas that have traditionally benefited from such technology. For instance, although there were studies that utilized VR environments (Costello and McNaughton 2018 ; Muller Queiroz et al. 2018 ) and video-based learning systems (Wei et al. 2013 ; Wei and Ram 2016 ), there was limited focus in contemporary eOL research on how specific affordances of the various environments that are used in organizations (e.g., Carnetsoft, Outotec HSC, and Simscale for simulations of working environments; or Raptivity, YouTube, and FStoppers to gain specific skills and how-to knowledge) can benefit the intended goals or be integrated with the unique qualities of the organization (e.g., IT, healthcare).

For the design and the development of the eOL approach, the organization needs to consider the alignment of individual learning needs, organizational objectives, and the necessary resources (Wang  2011 ). To achieve this, it is advisable for organizations to define the expected objectives, catalogue the individual needs, and select technologies that have the capacity to support and enrich learners with self-directed and socially constructed learning practices in the organization (Wang  2011 ). This needs to be done by taking into consideration that on-demand eOL is gradually replacing the classic static eOL curricula and processes (Dignen and Burmeister 2020 ).

Another important dimension of eOL research is the lenses used to approach effectiveness. The selected papers approached effectiveness with various objectives, such as fostering learning, usage of the e-learning system, employees’ performance, and the added organizational value (see Appendix ). To measure these indices, various metrics (quantitative, qualitative, and mixed) have been applied. The qualitative dimensions emphasize employees’ satisfaction and system usage (e.g., Menolli et al. 2020 ; Turi et al. 2019 ), as well as managers’ perceived gained value and benefits (e.g., Lee et al. 2015b ; Xiang et al. 2020 ) and firms’ perceived effective utilization of eOL resources (López-Nicolás and Meroño-Cerdán 2011 ). The quantitative dimensions focus on usage, feasibility, and experience at different levels within an organization, based on interviews, focus groups, and observations (Costello and McNaughton 2018 ; Michalski 2014 ; Stoffregen et al. 2016 ). However, it is not always clear the how eOL effectiveness has been measured, nor the extent to which eOL is well aligned with and is strategically impactful on delivering the strategic agenda of the organization (Garavan et al. 2019 ).

Research on digital technologies is developing rapidly, and big data and business analytics have the potential to pave the way for organizations’ digital transformation and sustainable development (Mikalef et al. 2018 ; Pappas et al. 2018 ); however, our review finds surprisingly limited use of big data and analytics in eOL. Despite contemporary e-learning systems adopting data-driven mechanisms, as well as advances in learning analytics (Siemens and Long 2011 ), the results of our analysis indicate that learner-generated data in the context of eOL are used in only a few studies to extract very limited insights with respect to the effectiveness of eOL and the intended objectives of the respective study (Hung et al. 2015 ; Renner et al. 2020 ; Rober and Cooper 2011 ). Therefore, eOL research needs to focus on data-driven qualities that will allow future researchers to gain deeper insights into which capabilities need to be developed to monitor the effectiveness of the various practices and technologies, their alignment with other functions of the organization, and how eOL can be a strategic and impactful vehicle for materializing the strategic agenda of the organization.

5.1 Status of eOL Research

The current review suggests that, while the efficient implementation of eOL entails certain challenges, there is also a great potential for improving employees’ performance as well as overall organizational outcome and value. There are also opportunities for improving organizations’ learning flow, which might not be feasible with formal learning and training. In order to construct the main research dimensions of eOL research and to look more deeply at the research objectives of the studies (the information we coded as objectives in the Appendix ), we performed a content analysis and grouped the research objectives. This enabled us to summarize the contemporary research on eOL according to five major categories, each of which is describes further below. As the research objectives of the published work shows, the research on eOL conducted during the last decade has particularly focused on the following five directions.

Investigating the capabilities of different technologies in different organizations.

Research has particularly focused on how easy the technology is to use, on how useful it is, or on how well aligned/integrated it is with other systems and processes within the organization. In addition, studies have used different learning technologies (e.g., smart, social, personalized) to enhance organizational learning in different contexts and according to different needs. However, most works have focused on affordances such as remote training and the development of static courses or modules to share information with learners. Although a few studies have utilized contemporary e-learning systems (see Appendix ), even in these studies there is a lack of alignment between the capabilities of those systems (e.g., open online course, adaptive support, social and collaborative learning) and the objectives and strategy of the organization (e.g., organizational value, fostering learning).

Enriching the learning flow and learning potential in different levels within an organization.

The reviewed work has emphasized how different factors contribute to different levels of organizational learning, and it has focused on practices that address individual, collaborative, and organizational learning within the structure of the organization. In particular, most of the reviewed studies recognize that organizational learning occurs at multiple levels: individual, team (or group), and organization. In other words, although each of the studies carried out an investigation within a given level (except for Garavan et al. 2019 ), there is a recognition and discussion of the different levels. Therefore, the results align with the 4I framework of organizational learning that recognizes how learning across the different levels is linked by social and psychological processes: intuiting, interpreting, integrating, and institutionalizing (the 4Is) (Crossan et al. 1999 ). However, most of the studies focused on the institutionalizing-intuiting link (i.e., top-down feedback); moreover, no studies focused on contemporary learning technologies and processes that strengthen the learning flow (e.g., self-regulated learning).

Identifying critical aspects for effective eOL.

There is a considerable amount of predominantly qualitative studies that focus on potential barriers to eOL implementation as well as on the risks and requirements associated with the feasibility and successful implementation of eOL. In the same vein, research has emphasized the importance of alignment of eOL (both in processes and in technologies) within the organization. These critical aspects for effective eOL are sometimes the main objectives of the studies (see Appendix ). However, most of the elements relating to the effectiveness of eOL were measured with questionnaires and interviews with employees and managers, and very little work was conducted on how to leverage the digital technologies employed in eOL, big data, and analytics in order to monitor the effectiveness of eOL.

Implementing employee-centric eOL.

In most of the studies, the main objective was to increase employees’ adoption, satisfaction, and usage of the e-learning system. In addition, several studies focused on the e-learning system’s ability to improve employees’ performance, increase the knowledge flow in the organization, and foster learning. Most of the approaches were employee-centric, with a small amount of studies focusing on managers and the firm in general. However, employees were seen as static entities within the organization, with limited work investigating how eOL-based training exposes employees to new knowledge, broadens their skills repertoire, and has tremendous potential for fostering innovation (Lin and Sanders 2017 ).

Achieving goals associated with the value creation of the organization.

A considerable number of studies utilized the firm (rather than the individual employee) as the unit of analysis. Such studies focused on how the implementation of eOL can increase employee performance, organizational value, and customer value. Although this is extremely helpful in furthering knowledge about eOL technologies and practices, a more granular investigation of the different e-learning systems and processes to address the various goals and strategies of the organization would enable researchers to extract practical insights on the design and implementation of eOL.

5.2 Research Agenda

By conducting an SLR and documenting the eOL research of the last decade, we have identified promising themes of research that have the potential to further eOL research and practice. To do so, we define a research agenda consisting of five thematic areas of research, as depicted in the research framework in Fig.  5 , and we provide some suggestions on how researchers could approach these challenges. In this visualization of the framework, on the left side we present the organizations as they were identified from our review (i.e., area/topic category in the Appendix ) and the multiple levels where organizational learning occurs (Costello and McNaughton 2018 ). On the right side, we summarize the objectives as they were identified from our review (i.e., the objectives category in the Appendix ). In the middle, we depict the orchestration that was conducted and how potential future research on eOL can improve the orchestration of the various elements and accelerate the achievement of the intended objectives. In particular, our proposed research agenda includes five research themes discussed in the following subsections.

E-learning capabilities to enhance organizational research agenda

5.2.1 Theme 1: Couple E-learning Capabilities With the Intended Goals

The majority of the eOL studies either investigated a generic e-learning system using the umbrella term “e-learning” or did not provide enough details about the functionalities of the system (in most cases, it was simply defined as an online or web system). This indicates the very limited focus of the eOL research on the various capabilities of e-learning systems. In other words, the literature has been very detailed on the organizational value and employees’ acceptance of the technology, but less detailed on the capabilities of this technology that needs to be put into place to achieve the intended goals and strategic agenda. However, the capabilities of the e-learning systems and their use are not one-size-fits-all, and the intended goals (to obtain certain skills and competences) and employees’ needs and backgrounds play a determining role in the selection of the e-learning system (Al-Fraihat et al. 2020 ).

Only in a very few studies (Mueller et al. 2011 ; Renner et al. 2020 ) were the capabilities of the e-learning solutions (e.g., mobile learning, VR) utilized, and the results were found to significantly contribute to the intended goals. The intended knowledge can be procedural, declarative, general competence (e.g., presentation, communication, or leadership skills) or else, and its particularities and the pedagogical needs of the intended knowledge (e.g., a need for summative/formative feedback or for social learning support) should guide the selection of the e-learning system and the respective capabilities. Therefore, future research needs to investigate how the various capabilities offered by contemporary learning systems (e.g., assessment mechanisms, social learning, collaborative learning, personalized learning) can be utilized to adequately reinforce the intended goals (e.g., to train personnel to use a new tool, to improve presentation skills).

5.2.2 Theme 2: Embrace the Particularities of the Various Industries

Organizational learning entails sharing knowledge and enabling opportunities for growth at the individual, group, team, and organizational levels. Contemporary e-learning systems provide the medium to substantiate the necessary knowledge flow within organizations and to support employees’ overall learning. From the selected studies, we can infer that eOL research is either conducted in an industry-agnostic context (either generic or it was not properly reported) or there is a focus on the IT industry (see Appendix ). However, when looking at the few studies that provide results from different industries (Garavan et al. 2019 ; Lee et al. 2014 ), companies indicate that there are different practices, processes, and expectations, and that employees have different needs and perceptions with regards to e-learning systems and eOL in general. Such particularities influence the perceived dimensions of a learning organization. Some industries noted that eOL promoted the development of their learning organizations, whereas others reported that eOL did not seem to contribute to their development as a learning organization (Yoo and Huang 2016 ). Therefore, it is important that the implementation of organizational learning embraces the particularities of the various industries and future research needs to identify how the industry-specific characteristics can inform the design and development of organizational learning in promoting an organization’s goals and agenda.

5.2.3 Theme 3: Utilize E-learning Capabilities to Implement Employee-centric Approaches

For efficient organizational learning to be implemented, the processes and technologies need to recognize that learning is linked by social and psychological processes (Crossan et al. 1999 ). This allows employees to develop learning in various forms (e.g., social, emotional, personalized) and to develop elements such as self-awareness, self-control, and interpersonal skills that are vital for the organization. Looking at the contemporary eOL research, we notice that the exploration of e-learning capabilities to nurture the aforementioned elements and support employee-centric approaches is very limited (e.g., personalized technologies, adaptive assessment). Therefore, future research needs to collect data to understand how e-learning capabilities can be utilized in relation to employees’ needs and perceptions in order to provide solutions (e.g., collaborative, social, adaptive) that are employee-centric and focused on development, and that have the potential to move away from standard one-size-fits-all e-learning solutions to personalized and customized systems and processes.

5.2.4 Theme 4: Employ Analytics-enabled eOL

There is a lot of emphasis on measuring, via various qualitative and quantitative metrics, the effectiveness of eOL implemented at different levels in organizations. However, most of these metrics come from surveys and interviews that capture employees’ and managers’ perceptions of various aspects of eOL (e.g., fostering of learning, organizational value, employees’ performance), and very few studies utilize analytics (Hung et al. 2015 ; Renner et al. 2020 ; Rober and Cooper 2011 ). Given how digital technologies, big data, and business analytics pave the way towards organizations’ digital transformation and sustainable development (Mikalef et al. 2018 ; Pappas et al. 2018 ), and considering the learning analytics affordances of contemporary e-learning systems (Siemens and Long 2011 ), future work needs to investigate how learner/employee-generated data can be employed to inform practice and devise more accurate and temporal effectiveness metrics when measuring the importance and impact of eOL.

5.2.5 Theme 5: Orchestrate the Employees’ Needs, Resources, and Objectives in eOL Implementation

While considerable effort has been directed towards the various building blocks of eOL implementation, such as resources (intangible, tangible, and human skills) and employees’ needs (e.g., vision, growth, skills development), little is known so far about the processes and structures necessary for orchestrating those elements in order to achieve an organization’s intended goals and to materialize its overall agenda. In other words, eOL research has been very detailed on some of the elements that constitute efficient eOL, but less so on the interplay of those elements and how they need to be put into place. Prior literature on strategic resource planning has shown that competence in orchestrating such elements is a prerequisite to successfully increasing business value (Wang et al. 2012 ). Therefore, future research should not only investigate each of these elements in silos, but also consider their interplay, since it is likely that organizations with similar resources will exert highly varied levels in each of these elements (e.g., analytics-enabled, e-learning capabilities) to successfully materialize their goals (e.g., increase value, improve the competence base of their employees, modernize their organization).

5.3 Implications

Several implications for eOL have been revealed in this literature review. First, most studies agree that employees’ or trainees’ experience is extremely important for the successful implementation of eOL. Thus, keeping them in the design and implementation cycle of eOL will increase eOL adoption and satisfaction as well as reduce the risks and barriers. Another important implication addressed by some studies relates to the capabilities of the e-learning technologies, with easy-to-use, useful, and social technologies resulting in more efficient eOL (e.g., higher adoption and performance). Thus, it is important for organizations to incorporate these functionalities in the platform and reinforce them with appropriate content and support. This should not only benefit learning outcomes, but also provide the networking opportunities for employees to broaden their personal networks, which are often lost when companies move from face-to-face formal training to e-learning-enabled organizational learning.

5.4 Limitations

This review has some limitations. First, we had to make some methodological decisions (e.g., selection of databases, the search query) that might lead to certain biases in the results. However, tried to avoid such biases by considering all the major databases and following the steps indicated by Kitchenham and Charters ( 2007 ). Second, the selection of empirical studies and coding of the papers might pose another possible bias. However, the focus was clearly on the empirical evidence, the terminology employed (“e-learning”) is an umbrella term that covers the majority of the work in the area, and the coding of papers was checked by two researchers. Third, some elements of the papers were not described accurately, leading to some missing information in the coding of the papers. However, the amount of missing information was very small and could not affect the results significantly. Finally, we acknowledge that the selected methodology (Kitchenham and Charters 2007 ) includes potential biases (e.g., false negatives and false positives), and that different, equally valid methods (e.g., Okoli and Schabram 2010 ) might have been used and have resulted in slightly different outcomes. Nevertheless, despite the limitations of the selected methodology, it is a well-accepted and widely used literature review method in both software engineering and information systems (Boell and Cecez-Kecmanovic 2014 ), providing certain assurance of the results.

6 Conclusions and Future Work

We have presented an SLR of 47 contributions in the field of eOL over the last decade. With respect to RQ1, we analyzed the papers from different perspectives, such as research methodology, technology, industries, employees, and intended outcomes in terms of organizational value, employees’ performance, usage, and behavioral change. The detailed landscape is depicted in the Appendix and Figs.  3 and 4 ; with the results indicating the limited utilization of the various e-learning capabilities (e.g., social, collaborative) to achieve objectives connected with those capabilities (e.g., social learning and behavioral change, collaborative learning and overcoming barriers).

With respect to RQ2, we categorized the main findings of the selected papers into five areas that reflect the status of eOL research, and we have discussed the challenges and opportunities emerging from the current review. In addition, we have synthesized the extracted challenges and opportunities and proposed a research agenda consisting of five elements that provide suggestions on how researchers could approach these challenges and exploit the opportunities. Such an agenda will strengthen how e-learning can be leveraged to enhance the process of improving actions through better knowledge and understanding in an organization.

A number of suggestions for further research have emerged from reviewing prior and ongoing work on eOL. One recommendation for future researchers is to clearly describe the eOL approach by providing detailed information about the technologies and materials used, as well as the organizations. This will allow meta-analyses to be conducted and it will also identify the potential effects of a firm’s size or area on the performance and other aspects relating to organizational value. Future work should also focus on collecting and triangulating different types of data from different sources (e.g., systems’ logs). The reviewed studies were conducted mainly by using survey data, and they made limited use of data coming from the platforms; thus, the interpretations and triangulation between the different types of collected data were limited.

Al-Fraihat, D., Joy, M., & Sinclair, J. (2020). Evaluating E-learning systems success: An empirical study. Computers in Human Behavior, 102 , 67–86.

Article   Google Scholar  

Alharthi, A. D., Spichkova, M., & Hamilton, M. (2019). Sustainability requirements for eLearning systems: A systematic literature review and analysis. Requirements Engineering, 24 (4), 523–543.

Alsabawy, A. Y., Cater-Steel, A., & Soar, J. (2013). IT infrastructure services as a requirement for e-learning system success. Computers & Education, 69 , 431–451.

Antonacopoulou, E., & Chiva, R. (2007). The social complexity of organizational learning: The dynamics of learning and organizing. Management Learning, 38 , 277–295.

Berends, H., & Lammers, I. (2010). Explaining discontinuity in organizational learning: A process analysis. Organization Studies, 31 (8), 1045–1068.

Boell, S. K., & Cecez-Kecmanovic, D. (2014). A hermeneutic approach for conducting literature reviews and literature searches. Communications of the Association for Information Systems, 34 (1), 12.

Google Scholar  

Bologa, R., & Lupu, A. R. (2014). Organizational learning networks that can increase the productivity of IT consulting companies. A case study for ERP consultants. Expert Systems with Applications, 41 (1), 126–136.

Broadbent, J. (2017). Comparing online and blended learner’s self-regulated learning strategies and academic performance. The Internet and Higher Education, 33 , 24–32.

Cheng, B., Wang, M., Moormann, J., Olaniran, B. A., & Chen, N. S. (2012). The effects of organizational learning environment factors on e-learning acceptance. Computers & Education, 58 (3), 885–899.

Cheng, B., Wang, M., Yang, S. J., & Peng, J. (2011). Acceptance of competency-based workplace e-learning systems: Effects of individual and peer learning support. Computers & Education, 57 (1), 1317–1333.

Choi, S., & Ko, I. (2012). Leveraging electronic collaboration to promote interorganizational learning. International Journal of Information Management, 32 (6), 550–559.

Costello, J. T., & McNaughton, R. B. (2018). Integrating a dynamic capabilities framework into workplace e-learning process evaluations. Knowledge and Process Management, 25 (2), 108–125.

Crossan, M. M., Lane, H. W., & White, R. E. (1999). An organizational learning framework: From intuition to institution. Academy of Management Review, 24 , 522–537.

Crossan, M. M., Maurer, C. C., & White, R. E. (2011). Reflections on the 2009 AMR decade award: Do we have a theory of organizational learning? Academy of Management Review, 36 (3), 446–460.

Dee, J., & Leisyte, L. (2017). Knowledge sharing and organizational change in higher education. The Learning Organization, 24 (5), 355–365. https://doi.org/10.1108/TLO-04-2017-0034

Dignen, B., & Burmeister, T. (2020). Learning and development in the organizations of the future. Three pillars of organization and leadership in disruptive times (pp. 207–232). Cham: Springer.

Chapter   Google Scholar  

Dybå, T., & Dingsøyr, T. (2008). Empirical studies of agile software development: A systematic review. Information and Software Technology, 50 (9–10), 833–859.

El Kadiri, S., Grabot, B., Thoben, K. D., Hribernik, K., Emmanouilidis, C., Von Cieminski, G., & Kiritsis, D. (2016). Current trends on ICT technologies for enterprise information systems. Computers in Industry, 79 , 14–33.

Engeström, Y., Kerosuo, H., & Kajamaa, A. (2007). Beyond discontinuity: Expansive organizational learning remembered. Management Learning, 38 (3), 319–336.

Gal, E., & Nachmias, R. (2011). Online learning and performance support in organizational environments using performance support platforms. Performance Improvement, 50 (8), 25–32.

Garavan, T. N., Heneghan, S., O’Brien, F., Gubbins, C., Lai, Y., Carbery, R., & Grant, K. (2019). L&D professionals in organisations: much ambition, unfilled promise. European Journal of Training and Development, 44 (1), 1–86.

Goggins, S. P., Jahnke, I., & Wulf, V. (2013). Computer-supported collaborative learning at the workplace . New York: Springer.

Book   Google Scholar  

Hammad, R., Odeh, M., & Khan, Z. (2017). ELCMM: An e-learning capability maturity model. In Proceedings of the 15th International Conference (e-Society 2017) (pp. 169–178).

Hester, A. J., Hutchins, H. M., & Burke-Smalley, L. A. (2016). Web 2.0 and transfer: Trainers’ use of technology to support employees’ learning transfer on the job. Performance Improvement Quarterly, 29 (3), 231–255.

Hung, Y. H., Lin, C. F., & Chang, R. I. (2015). Developing a dynamic inference expert system to support individual learning at work. British Journal of Educational Technology, 46 (6), 1378–1391.

Iris, R., & Vikas, A. (2011). E-Learning technologies: A key to dynamic capabilities. Computers in Human Behavior, 27 (5), 1868–1874.

Jia, H., Wang, M., Ran, W., Yang, S. J., Liao, J., & Chiu, D. K. (2011). Design of a performance-oriented workplace e-learning system using ontology. Expert Systems with Applications, 38 (4), 3372–3382.

Joo, Y. J., Lim, K. Y., & Park, S. Y. (2011). Investigating the structural relationships among organisational support, learning flow, learners’ satisfaction and learning transfer in corporate e-learning. British Journal of Educational Technology, 42 (6), 973–984.

Kaschig, A., Maier, R., Sandow, A., Lazoi, M., Barnes, S. A., Bimrose, J., … Schmidt, A. (2010). Knowledge maturing activities and practices fostering organisational learning: results of an empirical study. In European Conference on Technology Enhanced Learning (pp. 151–166). Berlin: Springer.

Khalili, A., Auer, S., Tarasowa, D., & Ermilov, I. (2012). SlideWiki: Elicitation and sharing of corporate knowledge using presentations. International Conference on Knowledge Engineering and Knowledge Management (pp. 302–316). Berlin: Springer.

Khandakar, M. S. A., & Pangil, F. (2019). Relationship between human resource management practices and informal workplace learning. Journal of Workplace Learning, 31 (8), 551–576.

Kim, M. K., Kim, S. M., & Bilir, M. K. (2014). Investigation of the dimensions of workplace learning environments (WLEs): Development of the WLE measure. Performance Improvement Quarterly, 27 (2), 35–57.

Kitchenham, B., & Charters, S. (2007). Guidelines for performing systematic literature reviews in software engineering. Technical Report EBSE-2007-01, 2007 . https://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=35909B1B280E2032BF116BDC9DCB71EA? .

Krippendorff, K. (2018). Content analysis: an introduction to its methodology. Thousand Oaks: Sage Publications.

Lai, H. J. (2017). Examining civil servants’ decisions to use Web 2.0 tools for learning, based on the decomposed theory of planned behavior. Interactive Learning Environments, 25 (3), 295–305.

Lau, K. (2015). Organizational learning goes virtual? A study of employees’ learning achievement in stereoscopic 3D virtual reality. The Learning Organization, 22 (5), 289–303.

Lee, J., Choi, M., & Lee, H. (2015a). Factors affecting smart learning adoption in workplaces: Comparing large enterprises and SMEs. Information Technology and Management, 16 (4), 291–302.

Lee, J., Kim, D. W., & Zo, H. (2015b). Conjoint analysis on preferences of HRD managers and employees for effective implementation of m-learning: The case of South Korea. Telematics and Informatics, 32 (4), 940–948.

Lee, J., Zo, H., & Lee, H. (2014). Smart learning adoption in employees and HRD managers. British Journal of Educational Technology, 45 (6), 1082–1096.

Lin, C. H., & Sanders, K. (2017). HRM and innovation: A multi-level organizational learning perspective. Human Resource Management Journal, 27 (2), 300–317.

Lin, C. Y., Huang, C. K., & Zhang, H. (2019). Enhancing employee job satisfaction via e-learning: The mediating role of an organizational learning culture. International Journal of Human–Computer Interaction, 35 (7), 584–595.

Liu, Y. C., Huang, Y. A., & Lin, C. (2012). Organizational factors’ effects on the success of e-learning systems and organizational benefits: An empirical study in Taiwan. The International Review of Research in Open and Distributed Learning, 13 (4), 130–151.

López-Nicolás, C., & Meroño-Cerdán, ÁL. (2011). Strategic knowledge management, innovation and performance. International Journal of Information Management, 31 (6), 502–509.

Manuti, A., Pastore, S., Scardigno, A. F., Giancaspro, M. L., & Morciano, D. (2015). Formal and informal learning in the workplace: A research review. International Journal of Training and Development, 19 (1), 1–17.

March, J. G. (1991). Exploration and exploitation in organizational learning. Organization Science, 2 (1), 71–87.

Marshall, S. (2006). New Zealand Tertiary Institution E-learning Capability: Informing and Guiding eLearning Architectural Change and Development. Report to the ministry of education . NZ: Victoria University of Wellington.

McDonald, N., Schoenebeck, S., & Forte, A. (2019). Reliability and inter-rater reliability in qualitative research: Norms and guidelines for CSCW and HCI practice. In Proceedings of the ACM on Human–Computer Interaction, 3(CSCW) (pp. 1–23).

Menolli, A., Tirone, H., Reinehr, S., & Malucelli, A. (2020). Identifying organisational learning needs: An approach to the semi-automatic creation of course structures for software companies. Behaviour & Information Technology, 39 (11), 1140–1155.

Michalski, M. P. (2014). Symbolic meanings and e-learning in the workplace: The case of an intranet-based training tool. Management Learning, 45 (2), 145–166.

Mikalef, P., Pappas, I. O., Krogstie, J., & Giannakos, M. (2018). Big data analytics capabilities: A systematic literature review and research agenda. Information Systems and e-Business Management, 16 (3), 547–578.

Mitić, S., Nikolić, M., Jankov, J., Vukonjanski, J., & Terek, E. (2017). The impact of information technologies on communication satisfaction and organizational learning in companies in Serbia. Computers in Human Behavior, 76 , 87–101.

Mueller, J., Hutter, K., Fueller, J., & Matzler, K. (2011). Virtual worlds as knowledge management platform—A practice-perspective. Information Systems Journal, 21 (6), 479–501.

Muller Queiroz, A. C., Nascimento, M., Tori, A., Alejandro, R. Brashear, Veloso, T., de Melo, V., de Souza Meirelles, F., & da Silva Leme, M. I. (2018). Immersive virtual environments in corporate education and training. In AMCIS. https://aisel.aisnet.org/amcis2018/Education/Presentations/12/ .

Navimipour, N. J., & Zareie, B. (2015). A model for assessing the impact of e-learning systems on employees’ satisfaction. Computers in Human Behavior, 53 , 475–485.

Oh, S. Y. (2019). Effects of organizational learning on performance: The moderating roles of trust in leaders and organizational justice. Journal of Knowledge Management, 23, 313–331.

Okoli, C., & Schabram, K. (2010). A guide to conducting a systematic literature review of information systems research. Sprouts: Working Papers on Information Systems, 10 (26), 1–46.

Pappas, I. O., Mikalef, P., Giannakos, M. N., Krogstie, J., & Lekakos, G. (2018). Big data and business analytics ecosystems: paving the way towards digital transformation and sustainable societies. Information Systems and e-Business Management, 16, 479–491.

Popova-Nowak, I. V., & Cseh, M. (2015). The meaning of organizational learning: A meta-paradigm perspective. Human Resource Development Review, 14 (3), 299–331.

Qi, C., & Chau, P. Y. (2016). An empirical study of the effect of enterprise social media usage on organizational learning. In Pacific Asia Conference on Information Systems (PACIS'16). Proceedings , Paper 330. http://aisel.aisnet.org/pacis2016/330 .

Renner, B., Wesiak, G., Pammer-Schindler, V., Prilla, M., Müller, L., Morosini, D., … Cress, U. (2020). Computer-supported reflective learning: How apps can foster reflection at work. Behaviour & Information Technology, 39 (2), 167–187.

Rober, M. B., & Cooper, L. P. (2011, January). Capturing knowledge via an” Intrapedia”: A case study. In 2011 44th Hawaii International Conference on System Sciences (pp. 1–10). New York: IEEE.

Rosenberg, M. J., & Foshay, R. (2002). E-learning: Strategies for delivering knowledge in the digital age. Performance Improvement, 41 (5), 50–51.

Serrano, Á., Marchiori, E. J., del Blanco, Á., Torrente, J., & Fernández-Manjón, B. (2012). A framework to improve evaluation in educational games. The IEEE Global Engineering Education Conference (pp. 1–8). Marrakesh, Morocco.

Siadaty, M., Jovanović, J., Gašević, D., Jeremić, Z., & Holocher-Ertl, T. (2010). Leveraging semantic technologies for harmonization of individual and organizational learning. In European Conference on Technology Enhanced Learning (pp. 340–356). Berlin: Springer.

Siemens, G., & Long, P. (2011). Penetrating the fog: Analytics in learning and education. EDUCAUSE Review, 46 (5), 30.

Škerlavaj, M., Dimovski, V., Mrvar, A., & Pahor, M. (2010). Intra-organizational learning networks within knowledge-intensive learning environments. Interactive Learning Environments, 18 (1), 39–63.

Smith, P. J., & Sadler-Smith, E. (2006). Learning in organizations: Complexities and diversities . London: Routledge.

Stoffregen, J. D., Pawlowski, J. M., Ras, E., Tobias, E., Šćepanović, S., Fitzpatrick, D., … Friedrich, H. (2016). Barriers to open e-learning in public administrations: A comparative case study of the European countries Luxembourg, Germany, Montenegro and Ireland. Technological Forecasting and Social Change, 111 , 198–208.

Subramaniam, R., & Nakkeeran, S. (2019). Impact of corporate e-learning systems in enhancing the team performance in virtual software teams. In Smart Technologies and Innovation for a Sustainable Future (pp. 195–204). Berlin: Springer.

Tsai, C. H., Zhu, D. S., Ho, B. C. T., & Wu, D. D. (2010). The effect of reducing risk and improving personal motivation on the adoption of knowledge repository system. Technological Forecasting and Social Change, 77 (6), 840–856.

Turi, J. A., Sorooshian, S., & Javed, Y. (2019). Impact of the cognitive learning factors on sustainable organizational development. Heliyon, 5 (9), e02398.

Wang, M. (2011). Integrating organizational, social, and individual perspectives in Web 2.0-based workplace e-learning. Information Systems Frontiers, 13 (2), 191–205.

Wang, M. (2018). Effects of individual and social learning support on employees’ acceptance of performance-oriented e-learning. In E-Learning in the Workplace (pp. 141–159). Springer. https://doi.org/10.1007/978-3-319-64532-2_13 .

Wang, M., Ran, W., Liao, J., & Yang, S. J. (2010). A performance-oriented approach to e-learning in the workplace. Journal of Educational Technology & Society, 13 (4), 167–179.

Wang, M., Vogel, D., & Ran, W. (2011). Creating a performance-oriented e-learning environment: A design science approach. Information & Management, 48 (7), 260–269.

Wang, N., Liang, H., Zhong, W., Xue, Y., & Xiao, J. (2012). Resource structuring or capability building? An empirical study of the business value of information technology. Journal of Management Information Systems, 29 (2), 325–367.

Wang, S., & Wang, H. (2012). Organizational schemata of e-portfolios for fostering higher-order thinking. Information Systems Frontiers, 14 (2), 395–407.

Wei, K., & Ram, J. (2016). Perceived usefulness of podcasting in organizational learning: The role of information characteristics. Computers in Human Behavior, 64 , 859–870.

Wei, K., Sun, H., & Li, H. (2013). On the driving forces of diffusion of podcasting in organizational settings: A case study and propositions. In PACIS 2013. Proceedings , 217. http://aisel.aisnet.org/pacis2013/217 .

Weinhardt, J. M., & Sitzmann, T. (2018). Revolutionizing training and education? Three questions regarding massive open online courses (MOOCs). Human Resource Management Review, 29 (2), 218–225.

Xiang, Q., Zhang, J., & Liu, H. (2020). Organisational improvisation as a path to new opportunity identification for incumbent firms: An organisational learning view. Innovation, 22 (4), 422–446. https://doi.org/10.1080/14479338.2020.1713001 .

Yanson, R., & Johnson, R. D. (2016). An empirical examination of e-learning design: The role of trainee socialization and complexity in short term training. Computers & Education, 101 , 43–54.

Yoo, S. J., & Huang, W. D. (2016). Can e-learning system enhance learning culture in the workplace? A comparison among companies in South Korea. British Journal of Educational Technology, 47 (4), 575–591.

Zhang, X., Jiang, S., Ordóñez de Pablos, P., Lytras, M. D., & Sun, Y. (2017). How virtual reality affects perceived learning effectiveness: A task–technology fit perspective. Behaviour & Information Technology, 36 (5), 548–556.

Zhang, X., Meng, Y., de Pablos, P. O., & Sun, Y. (2019). Learning analytics in collaborative learning supported by Slack: From the perspective of engagement. Computers in Human Behavior, 92 , 625–633.

Download references

Open Access funding provided by NTNU Norwegian University of Science and Technology (incl St. Olavs Hospital - Trondheim University Hospital).

Author information

Authors and affiliations.

Norwegian University of Science and Technology, Trondheim, Norway

Michail N. Giannakos, Patrick Mikalef & Ilias O. Pappas

University of Agder, Kristiansand, Norway

Ilias O. Pappas

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Ilias O. Pappas .

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Giannakos, M.N., Mikalef, P. & Pappas, I.O. Systematic Literature Review of E-Learning Capabilities to Enhance Organizational Learning. Inf Syst Front 24 , 619–635 (2022). https://doi.org/10.1007/s10796-020-10097-2

Download citation

Accepted : 09 December 2020

Published : 01 February 2021

Issue Date : April 2022

DOI : https://doi.org/10.1007/s10796-020-10097-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Organizational learning
• Literature review
• Learning environments
• Find a journal
• Publish with us
• Track your research

What is a Literature Review? How to Write It (with Examples)

A literature review is a critical analysis and synthesis of existing research on a particular topic. It provides an overview of the current state of knowledge, identifies gaps, and highlights key findings in the literature. 1 The purpose of a literature review is to situate your own research within the context of existing scholarship, demonstrating your understanding of the topic and showing how your work contributes to the ongoing conversation in the field. Learning how to write a literature review is a critical tool for successful research. Your ability to summarize and synthesize prior research pertaining to a certain topic demonstrates your grasp on the topic of study, and assists in the learning process. 

Table of Contents

What is the purpose of literature review , a. habitat loss and species extinction: , b. range shifts and phenological changes: , c. ocean acidification and coral reefs: , d. adaptive strategies and conservation efforts: .

• Choose a Topic and Define the Research Question: 
• Decide on the Scope of Your Review: 
• Select Databases for Searches: 
• Conduct Searches and Keep Track: 
• Review the Literature: 
• Organize and Write Your Literature Review: 
• How to write a literature review faster with Paperpal? 

Frequently asked questions 

What is a literature review .

A well-conducted literature review demonstrates the researcher’s familiarity with the existing literature, establishes the context for their own research, and contributes to scholarly conversations on the topic. One of the purposes of a literature review is also to help researchers avoid duplicating previous work and ensure that their research is informed by and builds upon the existing body of knowledge.

A literature review serves several important purposes within academic and research contexts. Here are some key objectives and functions of a literature review: 2  

1. Contextualizing the Research Problem: The literature review provides a background and context for the research problem under investigation. It helps to situate the study within the existing body of knowledge. 

2. Identifying Gaps in Knowledge: By identifying gaps, contradictions, or areas requiring further research, the researcher can shape the research question and justify the significance of the study. This is crucial for ensuring that the new research contributes something novel to the field.

Find academic papers related to your research topic faster. Try Research on Paperpal

3. Understanding Theoretical and Conceptual Frameworks: Literature reviews help researchers gain an understanding of the theoretical and conceptual frameworks used in previous studies. This aids in the development of a theoretical framework for the current research. 

4. Providing Methodological Insights: Another purpose of literature reviews is that it allows researchers to learn about the methodologies employed in previous studies. This can help in choosing appropriate research methods for the current study and avoiding pitfalls that others may have encountered. 

5. Establishing Credibility: A well-conducted literature review demonstrates the researcher’s familiarity with existing scholarship, establishing their credibility and expertise in the field. It also helps in building a solid foundation for the new research. 

6. Informing Hypotheses or Research Questions: The literature review guides the formulation of hypotheses or research questions by highlighting relevant findings and areas of uncertainty in existing literature. 

Literature review example 

Let’s delve deeper with a literature review example: Let’s say your literature review is about the impact of climate change on biodiversity. You might format your literature review into sections such as the effects of climate change on habitat loss and species extinction, phenological changes, and marine biodiversity. Each section would then summarize and analyze relevant studies in those areas, highlighting key findings and identifying gaps in the research. The review would conclude by emphasizing the need for further research on specific aspects of the relationship between climate change and biodiversity. The following literature review template provides a glimpse into the recommended literature review structure and content, demonstrating how research findings are organized around specific themes within a broader topic. 

Literature Review on Climate Change Impacts on Biodiversity:  

Climate change is a global phenomenon with far-reaching consequences, including significant impacts on biodiversity. This literature review synthesizes key findings from various studies: 

Climate change-induced alterations in temperature and precipitation patterns contribute to habitat loss, affecting numerous species (Thomas et al., 2004). The review discusses how these changes increase the risk of extinction, particularly for species with specific habitat requirements. 

Observations of range shifts and changes in the timing of biological events (phenology) are documented in response to changing climatic conditions (Parmesan & Yohe, 2003). These shifts affect ecosystems and may lead to mismatches between species and their resources. 

The review explores the impact of climate change on marine biodiversity, emphasizing ocean acidification’s threat to coral reefs (Hoegh-Guldberg et al., 2007). Changes in pH levels negatively affect coral calcification, disrupting the delicate balance of marine ecosystems. 

Recognizing the urgency of the situation, the literature review discusses various adaptive strategies adopted by species and conservation efforts aimed at mitigating the impacts of climate change on biodiversity (Hannah et al., 2007). It emphasizes the importance of interdisciplinary approaches for effective conservation planning. 

Strengthen your literature review with factual insights. Try Research on Paperpal for free!

How to write a good literature review 

Writing a literature review involves summarizing and synthesizing existing research on a particular topic. A good literature review format should include the following elements. 

Introduction: The introduction sets the stage for your literature review, providing context and introducing the main focus of your review. 

• Opening Statement: Begin with a general statement about the broader topic and its significance in the field. 
• Scope and Purpose: Clearly define the scope of your literature review. Explain the specific research question or objective you aim to address. 
• Organizational Framework: Briefly outline the structure of your literature review, indicating how you will categorize and discuss the existing research. 
• Significance of the Study: Highlight why your literature review is important and how it contributes to the understanding of the chosen topic. 
• Thesis Statement: Conclude the introduction with a concise thesis statement that outlines the main argument or perspective you will develop in the body of the literature review. 

Body: The body of the literature review is where you provide a comprehensive analysis of existing literature, grouping studies based on themes, methodologies, or other relevant criteria. 

• Organize by Theme or Concept: Group studies that share common themes, concepts, or methodologies. Discuss each theme or concept in detail, summarizing key findings and identifying gaps or areas of disagreement. 
• Critical Analysis: Evaluate the strengths and weaknesses of each study. Discuss the methodologies used, the quality of evidence, and the overall contribution of each work to the understanding of the topic. 
• Synthesis of Findings: Synthesize the information from different studies to highlight trends, patterns, or areas of consensus in the literature. 
• Identification of Gaps: Discuss any gaps or limitations in the existing research and explain how your review contributes to filling these gaps. 
• Transition between Sections: Provide smooth transitions between different themes or concepts to maintain the flow of your literature review. 

Write and Cite as yo u go with Paperpal Research. Start now for free!

Conclusion: The conclusion of your literature review should summarize the main findings, highlight the contributions of the review, and suggest avenues for future research. 

• Summary of Key Findings: Recap the main findings from the literature and restate how they contribute to your research question or objective. 
• Contributions to the Field: Discuss the overall contribution of your literature review to the existing knowledge in the field. 
• Implications and Applications: Explore the practical implications of the findings and suggest how they might impact future research or practice. 
• Recommendations for Future Research: Identify areas that require further investigation and propose potential directions for future research in the field. 
• Final Thoughts: Conclude with a final reflection on the importance of your literature review and its relevance to the broader academic community. 

Conducting a literature review 

Conducting a literature review is an essential step in research that involves reviewing and analyzing existing literature on a specific topic. It’s important to know how to do a literature review effectively, so here are the steps to follow: 1  

Choose a Topic and Define the Research Question:  

• Select a topic that is relevant to your field of study. 
• Clearly define your research question or objective. Determine what specific aspect of the topic do you want to explore? 

Decide on the Scope of Your Review:  

• Determine the timeframe for your literature review. Are you focusing on recent developments, or do you want a historical overview? 
• Consider the geographical scope. Is your review global, or are you focusing on a specific region? 
• Define the inclusion and exclusion criteria. What types of sources will you include? Are there specific types of studies or publications you will exclude? 

Select Databases for Searches:  

• Identify relevant databases for your field. Examples include PubMed, IEEE Xplore, Scopus, Web of Science, and Google Scholar. 
• Consider searching in library catalogs, institutional repositories, and specialized databases related to your topic. 

Conduct Searches and Keep Track:  

• Develop a systematic search strategy using keywords, Boolean operators (AND, OR, NOT), and other search techniques. 
• Record and document your search strategy for transparency and replicability. 
• Keep track of the articles, including publication details, abstracts, and links. Use citation management tools like EndNote, Zotero, or Mendeley to organize your references. 

Review the Literature:  

• Evaluate the relevance and quality of each source. Consider the methodology, sample size, and results of studies. 
• Organize the literature by themes or key concepts. Identify patterns, trends, and gaps in the existing research. 
• Summarize key findings and arguments from each source. Compare and contrast different perspectives. 
• Identify areas where there is a consensus in the literature and where there are conflicting opinions. 
• Provide critical analysis and synthesis of the literature. What are the strengths and weaknesses of existing research? 

Organize and Write Your Literature Review:  

• Literature review outline should be based on themes, chronological order, or methodological approaches. 
• Write a clear and coherent narrative that synthesizes the information gathered. 
• Use proper citations for each source and ensure consistency in your citation style (APA, MLA, Chicago, etc.). 
• Conclude your literature review by summarizing key findings, identifying gaps, and suggesting areas for future research. 

Whether you’re exploring a new research field or finding new angles to develop an existing topic, sifting through hundreds of papers can take more time than you have to spare. But what if you could find science-backed insights with verified citations in seconds? That’s the power of Paperpal’s new Research feature!  

How to write a literature review faster with Paperpal?  

Paperpal, an AI writing assistant, integrates powerful academic search capabilities within its writing platform. With the Research | Cite feature, you get 100% factual insights, with citations backed by 250M+ verified research articles, directly within your writing interface. It also allows you auto-cite references in 10,000+ styles and save relevant references in your Citation Library. By eliminating the need to switch tabs to find answers to all your research questions, Paperpal saves time and helps you stay focused on your writing.   

Here’s how to use the Research feature:  

• Ask a question: Get started with a new document on paperpal.com. Click on the “Research | Cite” feature and type your question in plain English. Paperpal will scour over 250 million research articles, including conference papers and preprints, to provide you with accurate insights and citations. 

• Review and Save: Paperpal summarizes the information, while citing sources and listing relevant reads. You can quickly scan the results to identify relevant references and save these directly to your built-in citations library for later access. 
• Cite with Confidence: Paperpal makes it easy to incorporate relevant citations and references in 10,000+ styles into your writing, ensuring your arguments are well-supported by credible sources. This translates to a polished, well-researched literature review. 

The literature review sample and detailed advice on writing and conducting a review will help you produce a well-structured report. But remember that a good literature review is an ongoing process, and it may be necessary to revisit and update it as your research progresses. By combining effortless research with an easy citation process, Paperpal Research streamlines the literature review process and empowers you to write faster and with more confidence. Try Paperpal Research now and see for yourself.  

A literature review is a critical and comprehensive analysis of existing literature (published and unpublished works) on a specific topic or research question and provides a synthesis of the current state of knowledge in a particular field. A well-conducted literature review is crucial for researchers to build upon existing knowledge, avoid duplication of efforts, and contribute to the advancement of their field. It also helps researchers situate their work within a broader context and facilitates the development of a sound theoretical and conceptual framework for their studies.

Literature review is a crucial component of research writing, providing a solid background for a research paper’s investigation. The aim is to keep professionals up to date by providing an understanding of ongoing developments within a specific field, including research methods, and experimental techniques used in that field, and present that knowledge in the form of a written report. Also, the depth and breadth of the literature review emphasizes the credibility of the scholar in his or her field.  

Before writing a literature review, it’s essential to undertake several preparatory steps to ensure that your review is well-researched, organized, and focused. This includes choosing a topic of general interest to you and doing exploratory research on that topic, writing an annotated bibliography, and noting major points, especially those that relate to the position you have taken on the topic. 

Literature reviews and academic research papers are essential components of scholarly work but serve different purposes within the academic realm. 3 A literature review aims to provide a foundation for understanding the current state of research on a particular topic, identify gaps or controversies, and lay the groundwork for future research. Therefore, it draws heavily from existing academic sources, including books, journal articles, and other scholarly publications. In contrast, an academic research paper aims to present new knowledge, contribute to the academic discourse, and advance the understanding of a specific research question. Therefore, it involves a mix of existing literature (in the introduction and literature review sections) and original data or findings obtained through research methods. 

Literature reviews are essential components of academic and research papers, and various strategies can be employed to conduct them effectively. If you want to know how to write a literature review for a research paper, here are four common approaches that are often used by researchers.  Chronological Review: This strategy involves organizing the literature based on the chronological order of publication. It helps to trace the development of a topic over time, showing how ideas, theories, and research have evolved.  Thematic Review: Thematic reviews focus on identifying and analyzing themes or topics that cut across different studies. Instead of organizing the literature chronologically, it is grouped by key themes or concepts, allowing for a comprehensive exploration of various aspects of the topic.  Methodological Review: This strategy involves organizing the literature based on the research methods employed in different studies. It helps to highlight the strengths and weaknesses of various methodologies and allows the reader to evaluate the reliability and validity of the research findings.  Theoretical Review: A theoretical review examines the literature based on the theoretical frameworks used in different studies. This approach helps to identify the key theories that have been applied to the topic and assess their contributions to the understanding of the subject.  It’s important to note that these strategies are not mutually exclusive, and a literature review may combine elements of more than one approach. The choice of strategy depends on the research question, the nature of the literature available, and the goals of the review. Additionally, other strategies, such as integrative reviews or systematic reviews, may be employed depending on the specific requirements of the research.

The literature review format can vary depending on the specific publication guidelines. However, there are some common elements and structures that are often followed. Here is a general guideline for the format of a literature review:  Introduction:   Provide an overview of the topic.  Define the scope and purpose of the literature review.  State the research question or objective.  Body:   Organize the literature by themes, concepts, or chronology.  Critically analyze and evaluate each source.  Discuss the strengths and weaknesses of the studies.  Highlight any methodological limitations or biases.  Identify patterns, connections, or contradictions in the existing research.  Conclusion:   Summarize the key points discussed in the literature review.  Highlight the research gap.  Address the research question or objective stated in the introduction.  Highlight the contributions of the review and suggest directions for future research.

Both annotated bibliographies and literature reviews involve the examination of scholarly sources. While annotated bibliographies focus on individual sources with brief annotations, literature reviews provide a more in-depth, integrated, and comprehensive analysis of existing literature on a specific topic. The key differences are as follows: 

References 

• Denney, A. S., & Tewksbury, R. (2013). How to write a literature review.  Journal of criminal justice education ,  24 (2), 218-234. 
• Pan, M. L. (2016).  Preparing literature reviews: Qualitative and quantitative approaches . Taylor & Francis. 
• Cantero, C. (2019). How to write a literature review.  San José State University Writing Center . 

Paperpal is a comprehensive AI writing toolkit that helps students and researchers achieve 2x the writing in half the time. It leverages 22+ years of STM experience and insights from millions of research articles to provide in-depth academic writing, language editing, and submission readiness support to help you write better, faster.  

Get accurate academic translations, rewriting support, grammar checks, vocabulary suggestions, and generative AI assistance that delivers human precision at machine speed. Try for free or upgrade to Paperpal Prime starting at US$19 a month to access premium features, including consistency, plagiarism, and 30+ submission readiness checks to help you succeed.  

Experience the future of academic writing – Sign up to Paperpal and start writing for free!  

Related Reads:

• Empirical Research: A Comprehensive Guide for Academics 
• How to Write a Scientific Paper in 10 Steps 
• How Long Should a Chapter Be?
• How to Use Paperpal to Generate Emails & Cover Letters?

6 Tips for Post-Doc Researchers to Take Their Career to the Next Level

Self-plagiarism in research: what it is and how to avoid it, you may also like, academic integrity vs academic dishonesty: types & examples, dissertation printing and binding | types & comparison , what is a dissertation preface definition and examples , the ai revolution: authors’ role in upholding academic..., the future of academia: how ai tools are..., how to write a research proposal: (with examples..., how to write your research paper in apa..., how to choose a dissertation topic, how to write a phd research proposal, how to write an academic paragraph (step-by-step guide).

• Peterborough

Writing Literature Reviews

What is a literature review, establish direction, read sources analytically, synthesize the sources.

• Develop a Position on the Body of Work

Attribution of Sources

Literature reviews may be written as separate assignments or as a section within a longer paper or thesis chapter. Writing a literature review requires you to demonstrate many essential academic skills: to perform research, to read carefully, to analyze and synthesize theories and findings, and to make an organized argument about the body of research on a topic. Furthermore, by assigning a literature review, your instructor expects that you will develop detailed knowledge about an important topic and you will recognize the dynamic nature of scholarship within a particular field of study.

In a literature review, you must survey a range of sources on a particular topic and analyze how scholars have examined and explained this particular topic. You must present a focused and organized argument about the state of knowledge on this topic, considering how research has progressed, evaluating debates, and identifying unanswered questions.

1. Narrow your focus.

Because you need to discuss a wide range of sources in a literature review, you may find it difficult to narrow your focus for an effective critique and understanding of research on the topic. Indeed, you may need to consider theoretical frameworks, methodology, and a variety of data sources or evidence alongside the findings and conclusions put forth by the authors of the articles and books that inform your review. It is important to consider the purpose of your literature review to ensure you ask appropriate questions of your topic.

2. Understand its purpose.

In a literature review that stands alone, your argument may be related to the development of a body of knowledge, the significance of debates within the literature, or future directions for research. The literature review should always offer an analytical discussion of the value or meaning of a body of knowledge. Later in this document, we offer strategies to encourage this thinking and organize your ideas.

For a thesis or major paper in an advanced course, a literature review establishes a framework for original research. In this case, you should also be sure to consider the questions that have not been asked or the perspective that have not been considered. It is important to explain how the literature informs your original work and how your work builds on the body of knowledge that exists. 

Become familiar with particular conventions for your discipline by reading published work. For a history course, you may write a historiography, which is a literature review. In psychology, you need to include particular details about the studies you review, so look to published meta-analyses or reviews for direction.

3. Organize your ideas.

Successful literature reviews offer a clear thesis or position about the current state of research on the topic usually organized comparatively, chronologically or thematically. A literature review should not be a collection of disconnected paragraphs that summarize different articles.

A literature review can be organized in many different ways depending on the discipline as well as the nature of the topic and the purpose of the review. A review might be organized chronologically to show how scholars’ views on a topic have changed over time; it could also be organized thematically or by methodology to group together scholars who share similar views or methods. A comparative organization offers an opportunity to assess sides of a debate or different theoretical lenses.

Read each source closely and carefully to determine key ideas from the source and to analyze the source independently of other sources. Consider the following questions and categories, which some students find it helpful to organize these questions in a chart:

• Identify the author(s). Expert or authority on the subject?
• Date of publication. Revised edition? When originally published?
• What kind of research is it? Empirical, review, argumentative, meta-analysis? How does this form affect the findings or relevance of this work?
• What is the research question and rationale for that question? Is the rationale valid?
• Is there a particular theoretical framework that informs this source? How does this framework influence its argument or findings?
• Is there a particular methodological framework that informs this source? How does this framework influence its argument or findings?
• How does the evidence or data support the arguments or findings of the source?
• What are the implications of the argument or findings?

Although your personal response to the literature may lead you to important insights, your assessment of the research cannot be driven by your biases or assumptions. Instead, you must inquire further when you have an immediate reaction to a piece of evidence or the basis for an argument. Consider the root of the reaction to ground your analysis and to understand bigger questions or trends in the literature.

Consider relationships between your sources and how these relationships influence the current state of knowledge on the topic.  You may find it helpful to organize your notes in particular folders, create a table, or use colour coding to make these relationships visible. The following questions can encourage synthesis:

• On what points do sources agree? What are the key concepts or theories that are accepted on this topic? How does one source build upon knowledge established by another?
• What are points of contention? Are there important debates on this topic? What lenses or frameworks inform the different sides of these debates? What do these debates reveal? Is one position more effective/persuasive? Why?
• How have ideas developed in different times, places, or fields?
• Where are current directions in research heading?

Develop a position on the body of work

A literature review must have a focused perspective. Take time to consider the sources to develop a cohesive and insightful discussion about the body of scholarship. Again, use inquiry to spark your thinking; you may find one or more of these questions to be helpful:

• What evidence, methods, or perspectives have influenced the current understanding of this topic?
• What is the significance of the existing knowledge or debates?
• What is the direction of scholarship? How has it changed? How is it changing?
• What has not been addressed? Why does it need be addressed?

Specificity is necessary for clear communication. In particular, you must clearly identify the authors of works to differentiate their findings, approaches, and perspectives. Whether you are summarizing an argument or analyzing its logic, include the author’s name in the sentence (alongside appropriate citations). For example, “Black (2011) argues that print media has an important place in the 21st century, but this position is not common; in fact, research by Khan and Stewart (2010), Li and Korchev (2012), and Atkins et. al (2012) demonstrates that print media’s influence is limited to small audiences.” This example also shows how to pull sources together into a cohesive point, rather than separately summarizing them.

Remember, there is no one model for a successful literature review; your direction is determined by your critical reading of the sources, your purpose for the review, and the conventions of your field of study. Be sure to ask your course instructor for advice if you are unsure of your approach to this type of work.

Literature Review: Sample

This is an excerpt from a literature review in a fourth-year educational sociology paper. Note how the author integrates arguments by different authors within a discussion about one point to demonstrate the debates that exist in the literature. Furthermore, this author puts forth a position about these debates.

[Topic sentence describing the point of the paragraph] The most significant element of anti-racist education is the recognition of race and racism. Social work theorist, Dorothy Chave Herberg (1993) examines the divergent Canadian understandings of racism. She notes there are some who directly address racism and those who do not recognize its existence in present-day Canada. Chave Herberg, Enid Lee (1985), and sociologist Carl James (1994) emphasize the necessity of discussing race and racism. [Making connections between sources] Chave Herberg argues that Canadians must recognize their own history of racism as she identifies and condemns a cultural condition which seems to inhibit discussions about race. Lee notes that many people feel uncomfortable talking about racism and believe that silence will diminish its impact on society. However, she argues that it is only through discussion that racism can be understood, and action for its abolition can begin. James urges all teachers to recognize the racial differences of students. He shows that without a recognition of their differences, students are marginalised and silenced, but when difference is validated by a teacher, students are welcomed into a positive learning environment. Conversely, Education professor Keith McLeod (1994) stresses the importance of non-racialism. [Illustrating debates within the field] He argues that students have difficulty developing positive relationships in an environment where difference is emphasized, but by recognizing the common traits shared by people, students can learn to “cope” with their differences and discrimination (McLeod, 1994, p. 19). It is interesting to note that McLeod believes it is sufficient for students to “cope” with their differences and discrimination. He does not discuss how they understand their difference, nor how they are motivated to challenge discrimination. McLeod fails to recognize the foundation of the arguments of Chave Herberg, Lee, and James: critical analysis of difference (race) and discrimination (racism) is fundamental to anti-racism education. [Author’s position on topic]

To read this content please select one of the options below:

Please note you do not have access to teaching notes, skill development research in india: a systematic literature review and future research agenda.

Benchmarking: An International Journal

ISSN : 1463-5771

Article publication date: 25 June 2019

Issue publication date: 13 September 2019

The purpose of this paper is to conduct an in-depth analysis of the literature and provide new insights on skill development research in India. Objectives in the study are twofold. First is to conduct descriptive analysis in terms of structural dimensions such as geographical context, focus on skill development, research methods, economic sector, type of training and origin of the study. The second objective is to conduct thematic analysis on importance of skill development, institutional and regulatory mechanisms, skill development as an antecedent for technology adoption, role of skill development in women empowerment, integrating skill development with secondary education and labour market reforms for skill development.

Design/methodology/approach

Systematic literature review method is employed to review 45 articles on skill development research in India conducted between 2004 and 2017.

The review discusses about the importance of skill development, the role of institutions such as NSDA and PMKVJ, skill development as a necessity in the advent of technological changes, an effective measure to empower women in the country and the need of integrating skill development with secondary education. Labour market reforms are required to overcome challenges such as mismatch between theory and practice, low quality of skills assimilated, the mismatch between demand and supply of skilled labour force, low level of in-house training, low cooperation from students and employees due to lack of incentives and lack of qualified teachers.

Originality/value

This study is the first to offer a systematic literature review on skill development. The study provides insights into the concept of skill development, specifically for academicians to carry out research in a niche area and for government authorities in policy formulation.

• Systematic literature review
• Technology adoption
• Skill development
• Demographic dividend

Cabral, C. and Dhar, R.L. (2019), "Skill development research in India: a systematic literature review and future research agenda", Benchmarking: An International Journal , Vol. 26 No. 7, pp. 2242-2266. https://doi.org/10.1108/BIJ-07-2018-0211

Emerald Publishing Limited

Copyright © 2019, Emerald Publishing Limited

Related articles

All feedback is valuable.

Please share your general feedback

Report an issue or find answers to frequently asked questions

Contact Customer Support

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Clinics (Sao Paulo)

Approaching literature review for academic purposes: The Literature Review Checklist

Debora f.b. leite.

I Departamento de Ginecologia e Obstetricia, Faculdade de Ciencias Medicas, Universidade Estadual de Campinas, Campinas, SP, BR

II Universidade Federal de Pernambuco, Pernambuco, PE, BR

III Hospital das Clinicas, Universidade Federal de Pernambuco, Pernambuco, PE, BR

Maria Auxiliadora Soares Padilha

Jose g. cecatti.

A sophisticated literature review (LR) can result in a robust dissertation/thesis by scrutinizing the main problem examined by the academic study; anticipating research hypotheses, methods and results; and maintaining the interest of the audience in how the dissertation/thesis will provide solutions for the current gaps in a particular field. Unfortunately, little guidance is available on elaborating LRs, and writing an LR chapter is not a linear process. An LR translates students’ abilities in information literacy, the language domain, and critical writing. Students in postgraduate programs should be systematically trained in these skills. Therefore, this paper discusses the purposes of LRs in dissertations and theses. Second, the paper considers five steps for developing a review: defining the main topic, searching the literature, analyzing the results, writing the review and reflecting on the writing. Ultimately, this study proposes a twelve-item LR checklist. By clearly stating the desired achievements, this checklist allows Masters and Ph.D. students to continuously assess their own progress in elaborating an LR. Institutions aiming to strengthen students’ necessary skills in critical academic writing should also use this tool.

INTRODUCTION

Writing the literature review (LR) is often viewed as a difficult task that can be a point of writer’s block and procrastination ( 1 ) in postgraduate life. Disagreements on the definitions or classifications of LRs ( 2 ) may confuse students about their purpose and scope, as well as how to perform an LR. Interestingly, at many universities, the LR is still an important element in any academic work, despite the more recent trend of producing scientific articles rather than classical theses.

The LR is not an isolated section of the thesis/dissertation or a copy of the background section of a research proposal. It identifies the state-of-the-art knowledge in a particular field, clarifies information that is already known, elucidates implications of the problem being analyzed, links theory and practice ( 3 - 5 ), highlights gaps in the current literature, and places the dissertation/thesis within the research agenda of that field. Additionally, by writing the LR, postgraduate students will comprehend the structure of the subject and elaborate on their cognitive connections ( 3 ) while analyzing and synthesizing data with increasing maturity.

At the same time, the LR transforms the student and hints at the contents of other chapters for the reader. First, the LR explains the research question; second, it supports the hypothesis, objectives, and methods of the research project; and finally, it facilitates a description of the student’s interpretation of the results and his/her conclusions. For scholars, the LR is an introductory chapter ( 6 ). If it is well written, it demonstrates the student’s understanding of and maturity in a particular topic. A sound and sophisticated LR can indicate a robust dissertation/thesis.

A consensus on the best method to elaborate a dissertation/thesis has not been achieved. The LR can be a distinct chapter or included in different sections; it can be part of the introduction chapter, part of each research topic, or part of each published paper ( 7 ). However, scholars view the LR as an integral part of the main body of an academic work because it is intrinsically connected to other sections ( Figure 1 ) and is frequently present. The structure of the LR depends on the conventions of a particular discipline, the rules of the department, and the student’s and supervisor’s areas of expertise, needs and interests.

Interestingly, many postgraduate students choose to submit their LR to peer-reviewed journals. As LRs are critical evaluations of current knowledge, they are indeed publishable material, even in the form of narrative or systematic reviews. However, systematic reviews have specific patterns 1 ( 8 ) that may not entirely fit with the questions posed in the dissertation/thesis. Additionally, the scope of a systematic review may be too narrow, and the strict criteria for study inclusion may omit important information from the dissertation/thesis. Therefore, this essay discusses the definition of an LR is and methods to develop an LR in the context of an academic dissertation/thesis. Finally, we suggest a checklist to evaluate an LR.

WHAT IS A LITERATURE REVIEW IN A THESIS?

Conducting research and writing a dissertation/thesis translates rational thinking and enthusiasm ( 9 ). While a strong body of literature that instructs students on research methodology, data analysis and writing scientific papers exists, little guidance on performing LRs is available. The LR is a unique opportunity to assess and contrast various arguments and theories, not just summarize them. The research results should not be discussed within the LR, but the postgraduate student tends to write a comprehensive LR while reflecting on his or her own findings ( 10 ).

Many people believe that writing an LR is a lonely and linear process. Supervisors or the institutions assume that the Ph.D. student has mastered the relevant techniques and vocabulary associated with his/her subject and conducts a self-reflection about previously published findings. Indeed, while elaborating the LR, the student should aggregate diverse skills, which mainly rely on his/her own commitment to mastering them. Thus, less supervision should be required ( 11 ). However, the parameters described above might not currently be the case for many students ( 11 , 12 ), and the lack of formal and systematic training on writing LRs is an important concern ( 11 ).

An institutional environment devoted to active learning will provide students the opportunity to continuously reflect on LRs, which will form a dialogue between the postgraduate student and the current literature in a particular field ( 13 ). Postgraduate students will be interpreting studies by other researchers, and, according to Hart (1998) ( 3 ), the outcomes of the LR in a dissertation/thesis include the following:

• To identify what research has been performed and what topics require further investigation in a particular field of knowledge;
• To determine the context of the problem;
• To recognize the main methodologies and techniques that have been used in the past;
• To place the current research project within the historical, methodological and theoretical context of a particular field;
• To identify significant aspects of the topic;
• To elucidate the implications of the topic;
• To offer an alternative perspective;
• To discern how the studied subject is structured;
• To improve the student’s subject vocabulary in a particular field; and
• To characterize the links between theory and practice.

A sound LR translates the postgraduate student’s expertise in academic and scientific writing: it expresses his/her level of comfort with synthesizing ideas ( 11 ). The LR reveals how well the postgraduate student has proceeded in three domains: an effective literature search, the language domain, and critical writing.

Effective literature search

All students should be trained in gathering appropriate data for specific purposes, and information literacy skills are a cornerstone. These skills are defined as “an individual’s ability to know when they need information, to identify information that can help them address the issue or problem at hand, and to locate, evaluate, and use that information effectively” ( 14 ). Librarian support is of vital importance in coaching the appropriate use of Boolean logic (AND, OR, NOT) and other tools for highly efficient literature searches (e.g., quotation marks and truncation), as is the appropriate management of electronic databases.

Language domain

Academic writing must be concise and precise: unnecessary words distract the reader from the essential content ( 15 ). In this context, reading about issues distant from the research topic ( 16 ) may increase students’ general vocabulary and familiarity with grammar. Ultimately, reading diverse materials facilitates and encourages the writing process itself.

Critical writing

Critical judgment includes critical reading, thinking and writing. It supposes a student’s analytical reflection about what he/she has read. The student should delineate the basic elements of the topic, characterize the most relevant claims, identify relationships, and finally contrast those relationships ( 17 ). Each scientific document highlights the perspective of the author, and students will become more confident in judging the supporting evidence and underlying premises of a study and constructing their own counterargument as they read more articles. A paucity of integration or contradictory perspectives indicates lower levels of cognitive complexity ( 12 ).

Thus, while elaborating an LR, the postgraduate student should achieve the highest category of Bloom’s cognitive skills: evaluation ( 12 ). The writer should not only summarize data and understand each topic but also be able to make judgments based on objective criteria, compare resources and findings, identify discrepancies due to methodology, and construct his/her own argument ( 12 ). As a result, the student will be sufficiently confident to show his/her own voice .

Writing a consistent LR is an intense and complex activity that reveals the training and long-lasting academic skills of a writer. It is not a lonely or linear process. However, students are unlikely to be prepared to write an LR if they have not mastered the aforementioned domains ( 10 ). An institutional environment that supports student learning is crucial.

Different institutions employ distinct methods to promote students’ learning processes. First, many universities propose modules to develop behind the scenes activities that enhance self-reflection about general skills (e.g., the skills we have mastered and the skills we need to develop further), behaviors that should be incorporated (e.g., self-criticism about one’s own thoughts), and each student’s role in the advancement of his/her field. Lectures or workshops about LRs themselves are useful because they describe the purposes of the LR and how it fits into the whole picture of a student’s work. These activities may explain what type of discussion an LR must involve, the importance of defining the correct scope, the reasons to include a particular resource, and the main role of critical reading.

Some pedagogic services that promote a continuous improvement in study and academic skills are equally important. Examples include workshops about time management, the accomplishment of personal objectives, active learning, and foreign languages for nonnative speakers. Additionally, opportunities to converse with other students promotes an awareness of others’ experiences and difficulties. Ultimately, the supervisor’s role in providing feedback and setting deadlines is crucial in developing students’ abilities and in strengthening students’ writing quality ( 12 ).

HOW SHOULD A LITERATURE REVIEW BE DEVELOPED?

A consensus on the appropriate method for elaborating an LR is not available, but four main steps are generally accepted: defining the main topic, searching the literature, analyzing the results, and writing ( 6 ). We suggest a fifth step: reflecting on the information that has been written in previous publications ( Figure 2 ).

First step: Defining the main topic

Planning an LR is directly linked to the research main question of the thesis and occurs in parallel to students’ training in the three domains discussed above. The planning stage helps organize ideas, delimit the scope of the LR ( 11 ), and avoid the wasting of time in the process. Planning includes the following steps:

• Reflecting on the scope of the LR: postgraduate students will have assumptions about what material must be addressed and what information is not essential to an LR ( 13 , 18 ). Cooper’s Taxonomy of Literature Reviews 2 systematizes the writing process through six characteristics and nonmutually exclusive categories. The focus refers to the reviewer’s most important points of interest, while the goals concern what students want to achieve with the LR. The perspective assumes answers to the student’s own view of the LR and how he/she presents a particular issue. The coverage defines how comprehensive the student is in presenting the literature, and the organization determines the sequence of arguments. The audience is defined as the group for whom the LR is written.
• Designating sections and subsections: Headings and subheadings should be specific, explanatory and have a coherent sequence throughout the text ( 4 ). They simulate an inverted pyramid, with an increasing level of reflection and depth of argument.
• Identifying keywords: The relevant keywords for each LR section should be listed to guide the literature search. This list should mirror what Hart (1998) ( 3 ) advocates as subject vocabulary . The keywords will also be useful when the student is writing the LR since they guide the reader through the text.
• Delineating the time interval and language of documents to be retrieved in the second step. The most recently published documents should be considered, but relevant texts published before a predefined cutoff year can be included if they are classic documents in that field. Extra care should be employed when translating documents.

Second step: Searching the literature

The ability to gather adequate information from the literature must be addressed in postgraduate programs. Librarian support is important, particularly for accessing difficult texts. This step comprises the following components:

• Searching the literature itself: This process consists of defining which databases (electronic or dissertation/thesis repositories), official documents, and books will be searched and then actively conducting the search. Information literacy skills have a central role in this stage. While searching electronic databases, controlled vocabulary (e.g., Medical Subject Headings, or MeSH, for the PubMed database) or specific standardized syntax rules may need to be applied.

In addition, two other approaches are suggested. First, a review of the reference list of each document might be useful for identifying relevant publications to be included and important opinions to be assessed. This step is also relevant for referencing the original studies and leading authors in that field. Moreover, students can directly contact the experts on a particular topic to consult with them regarding their experience or use them as a source of additional unpublished documents.

Before submitting a dissertation/thesis, the electronic search strategy should be repeated. This process will ensure that the most recently published papers will be considered in the LR.

• Selecting documents for inclusion: Generally, the most recent literature will be included in the form of published peer-reviewed papers. Assess books and unpublished material, such as conference abstracts, academic texts and government reports, are also important to assess since the gray literature also offers valuable information. However, since these materials are not peer-reviewed, we recommend that they are carefully added to the LR.

This task is an important exercise in time management. First, students should read the title and abstract to understand whether that document suits their purposes, addresses the research question, and helps develop the topic of interest. Then, they should scan the full text, determine how it is structured, group it with similar documents, and verify whether other arguments might be considered ( 5 ).

Third step: Analyzing the results

Critical reading and thinking skills are important in this step. This step consists of the following components:

• Reading documents: The student may read various texts in depth according to LR sections and subsections ( defining the main topic ), which is not a passive activity ( 1 ). Some questions should be asked to practice critical analysis skills, as listed below. Is the research question evident and articulated with previous knowledge? What are the authors’ research goals and theoretical orientations, and how do they interact? Are the authors’ claims related to other scholars’ research? Do the authors consider different perspectives? Was the research project designed and conducted properly? Are the results and discussion plausible, and are they consistent with the research objectives and methodology? What are the strengths and limitations of this work? How do the authors support their findings? How does this work contribute to the current research topic? ( 1 , 19 )
• Taking notes: Students who systematically take notes on each document are more readily able to establish similarities or differences with other documents and to highlight personal observations. This approach reinforces the student’s ideas about the next step and helps develop his/her own academic voice ( 1 , 13 ). Voice recognition software ( 16 ), mind maps ( 5 ), flowcharts, tables, spreadsheets, personal comments on the referenced texts, and note-taking apps are all available tools for managing these observations, and the student him/herself should use the tool that best improves his/her learning. Additionally, when a student is considering submitting an LR to a peer-reviewed journal, notes should be taken on the activities performed in all five steps to ensure that they are able to be replicated.

Fourth step: Writing

The recognition of when a student is able and ready to write after a sufficient period of reading and thinking is likely a difficult task. Some students can produce a review in a single long work session. However, as discussed above, writing is not a linear process, and students do not need to write LRs according to a specific sequence of sections. Writing an LR is a time-consuming task, and some scholars believe that a period of at least six months is sufficient ( 6 ). An LR, and academic writing in general, expresses the writer’s proper thoughts, conclusions about others’ work ( 6 , 10 , 13 , 16 ), and decisions about methods to progress in the chosen field of knowledge. Thus, each student is expected to present a different learning and writing trajectory.

In this step, writing methods should be considered; then, editing, citing and correct referencing should complete this stage, at least temporarily. Freewriting techniques may be a good starting point for brainstorming ideas and improving the understanding of the information that has been read ( 1 ). Students should consider the following parameters when creating an agenda for writing the LR: two-hour writing blocks (at minimum), with prespecified tasks that are possible to complete in one section; short (minutes) and long breaks (days or weeks) to allow sufficient time for mental rest and reflection; and short- and long-term goals to motivate the writing itself ( 20 ). With increasing experience, this scheme can vary widely, and it is not a straightforward rule. Importantly, each discipline has a different way of writing ( 1 ), and each department has its own preferred styles for citations and references.

Fifth step: Reflecting on the writing

In this step, the postgraduate student should ask him/herself the same questions as in the analyzing the results step, which can take more time than anticipated. Ambiguities, repeated ideas, and a lack of coherence may not be noted when the student is immersed in the writing task for long periods. The whole effort will likely be a work in progress, and continuous refinements in the written material will occur once the writing process has begun.

LITERATURE REVIEW CHECKLIST

In contrast to review papers, the LR of a dissertation/thesis should not be a standalone piece or work. Instead, it should present the student as a scholar and should maintain the interest of the audience in how that dissertation/thesis will provide solutions for the current gaps in a particular field.

A checklist for evaluating an LR is convenient for students’ continuous academic development and research transparency: it clearly states the desired achievements for the LR of a dissertation/thesis. Here, we present an LR checklist developed from an LR scoring rubric ( 11 ). For a critical analysis of an LR, we maintain the five categories but offer twelve criteria that are not scaled ( Figure 3 ). The criteria all have the same importance and are not mutually exclusive.

First category: Coverage

1. justified criteria exist for the inclusion and exclusion of literature in the review.

This criterion builds on the main topic and areas covered by the LR ( 18 ). While experts may be confident in retrieving and selecting literature, postgraduate students must convince their audience about the adequacy of their search strategy and their reasons for intentionally selecting what material to cover ( 11 ). References from different fields of knowledge provide distinct perspective, but narrowing the scope of coverage may be important in areas with a large body of existing knowledge.

Second category: Synthesis

2. a critical examination of the state of the field exists.

A critical examination is an assessment of distinct aspects in the field ( 1 ) along with a constructive argument. It is not a negative critique but an expression of the student’s understanding of how other scholars have added to the topic ( 1 ), and the student should analyze and contextualize contradictory statements. A writer’s personal bias (beliefs or political involvement) have been shown to influence the structure and writing of a document; therefore, the cultural and paradigmatic background guide how the theories are revised and presented ( 13 ). However, an honest judgment is important when considering different perspectives.

3. The topic or problem is clearly placed in the context of the broader scholarly literature

The broader scholarly literature should be related to the chosen main topic for the LR ( how to develop the literature review section). The LR can cover the literature from one or more disciplines, depending on its scope, but it should always offer a new perspective. In addition, students should be careful in citing and referencing previous publications. As a rule, original studies and primary references should generally be included. Systematic and narrative reviews present summarized data, and it may be important to cite them, particularly for issues that should be understood but do not require a detailed description. Similarly, quotations highlight the exact statement from another publication. However, excessive referencing may disclose lower levels of analysis and synthesis by the student.

4. The LR is critically placed in the historical context of the field

Situating the LR in its historical context shows the level of comfort of the student in addressing a particular topic. Instead of only presenting statements and theories in a temporal approach, which occasionally follows a linear timeline, the LR should authentically characterize the student’s academic work in the state-of-art techniques in their particular field of knowledge. Thus, the LR should reinforce why the dissertation/thesis represents original work in the chosen research field.

5. Ambiguities in definitions are considered and resolved

Distinct theories on the same topic may exist in different disciplines, and one discipline may consider multiple concepts to explain one topic. These misunderstandings should be addressed and contemplated. The LR should not synthesize all theories or concepts at the same time. Although this approach might demonstrate in-depth reading on a particular topic, it can reveal a student’s inability to comprehend and synthesize his/her research problem.

6. Important variables and phenomena relevant to the topic are articulated

The LR is a unique opportunity to articulate ideas and arguments and to purpose new relationships between them ( 10 , 11 ). More importantly, a sound LR will outline to the audience how these important variables and phenomena will be addressed in the current academic work. Indeed, the LR should build a bidirectional link with the remaining sections and ground the connections between all of the sections ( Figure 1 ).

7. A synthesized new perspective on the literature has been established

The LR is a ‘creative inquiry’ ( 13 ) in which the student elaborates his/her own discourse, builds on previous knowledge in the field, and describes his/her own perspective while interpreting others’ work ( 13 , 17 ). Thus, students should articulate the current knowledge, not accept the results at face value ( 11 , 13 , 17 ), and improve their own cognitive abilities ( 12 ).

Third category: Methodology

8. the main methodologies and research techniques that have been used in the field are identified and their advantages and disadvantages are discussed.

The LR is expected to distinguish the research that has been completed from investigations that remain to be performed, address the benefits and limitations of the main methods applied to date, and consider the strategies for addressing the expected limitations described above. While placing his/her research within the methodological context of a particular topic, the LR will justify the methodology of the study and substantiate the student’s interpretations.

9. Ideas and theories in the field are related to research methodologies

The audience expects the writer to analyze and synthesize methodological approaches in the field. The findings should be explained according to the strengths and limitations of previous research methods, and students must avoid interpretations that are not supported by the analyzed literature. This criterion translates to the student’s comprehension of the applicability and types of answers provided by different research methodologies, even those using a quantitative or qualitative research approach.

Fourth category: Significance

10. the scholarly significance of the research problem is rationalized.

The LR is an introductory section of a dissertation/thesis and will present the postgraduate student as a scholar in a particular field ( 11 ). Therefore, the LR should discuss how the research problem is currently addressed in the discipline being investigated or in different disciplines, depending on the scope of the LR. The LR explains the academic paradigms in the topic of interest ( 13 ) and methods to advance the field from these starting points. However, an excess number of personal citations—whether referencing the student’s research or studies by his/her research team—may reflect a narrow literature search and a lack of comprehensive synthesis of ideas and arguments.

11. The practical significance of the research problem is rationalized

The practical significance indicates a student’s comprehensive understanding of research terminology (e.g., risk versus associated factor), methodology (e.g., efficacy versus effectiveness) and plausible interpretations in the context of the field. Notably, the academic argument about a topic may not always reflect the debate in real life terms. For example, using a quantitative approach in epidemiology, statistically significant differences between groups do not explain all of the factors involved in a particular problem ( 21 ). Therefore, excessive faith in p -values may reflect lower levels of critical evaluation of the context and implications of a research problem by the student.

Fifth category: Rhetoric

12. the lr was written with a coherent, clear structure that supported the review.

This category strictly relates to the language domain: the text should be coherent and presented in a logical sequence, regardless of which organizational ( 18 ) approach is chosen. The beginning of each section/subsection should state what themes will be addressed, paragraphs should be carefully linked to each other ( 10 ), and the first sentence of each paragraph should generally summarize the content. Additionally, the student’s statements are clear, sound, and linked to other scholars’ works, and precise and concise language that follows standardized writing conventions (e.g., in terms of active/passive voice and verb tenses) is used. Attention to grammar, such as orthography and punctuation, indicates prudence and supports a robust dissertation/thesis. Ultimately, all of these strategies provide fluency and consistency for the text.

Although the scoring rubric was initially proposed for postgraduate programs in education research, we are convinced that this checklist is a valuable tool for all academic areas. It enables the monitoring of students’ learning curves and a concentrated effort on any criteria that are not yet achieved. For institutions, the checklist is a guide to support supervisors’ feedback, improve students’ writing skills, and highlight the learning goals of each program. These criteria do not form a linear sequence, but ideally, all twelve achievements should be perceived in the LR.

CONCLUSIONS

A single correct method to classify, evaluate and guide the elaboration of an LR has not been established. In this essay, we have suggested directions for planning, structuring and critically evaluating an LR. The planning of the scope of an LR and approaches to complete it is a valuable effort, and the five steps represent a rational starting point. An institutional environment devoted to active learning will support students in continuously reflecting on LRs, which will form a dialogue between the writer and the current literature in a particular field ( 13 ).

The completion of an LR is a challenging and necessary process for understanding one’s own field of expertise. Knowledge is always transitory, but our responsibility as scholars is to provide a critical contribution to our field, allowing others to think through our work. Good researchers are grounded in sophisticated LRs, which reveal a writer’s training and long-lasting academic skills. We recommend using the LR checklist as a tool for strengthening the skills necessary for critical academic writing.

AUTHOR CONTRIBUTIONS

Leite DFB has initially conceived the idea and has written the first draft of this review. Padilha MAS and Cecatti JG have supervised data interpretation and critically reviewed the manuscript. All authors have read the draft and agreed with this submission. Authors are responsible for all aspects of this academic piece.

ACKNOWLEDGMENTS

We are grateful to all of the professors of the ‘Getting Started with Graduate Research and Generic Skills’ module at University College Cork, Cork, Ireland, for suggesting and supporting this article. Funding: DFBL has granted scholarship from Brazilian Federal Agency for Support and Evaluation of Graduate Education (CAPES) to take part of her Ph.D. studies in Ireland (process number 88881.134512/2016-01). There is no participation from sponsors on authors’ decision to write or to submit this manuscript.

No potential conflict of interest was reported.

1 The questions posed in systematic reviews usually follow the ‘PICOS’ acronym: Population, Intervention, Comparison, Outcomes, Study design.

2 In 1988, Cooper proposed a taxonomy that aims to facilitate students’ and institutions’ understanding of literature reviews. Six characteristics with specific categories are briefly described: Focus: research outcomes, research methodologies, theories, or practices and applications; Goals: integration (generalization, conflict resolution, and linguistic bridge-building), criticism, or identification of central issues; Perspective: neutral representation or espousal of a position; Coverage: exhaustive, exhaustive with selective citations, representative, central or pivotal; Organization: historical, conceptual, or methodological; and Audience: specialized scholars, general scholars, practitioners or policymakers, or the general public.

• Architecture and Design
• Asian and Pacific Studies
• Business and Economics
• Classical and Ancient Near Eastern Studies
• Computer Sciences
• Cultural Studies
• Engineering
• General Interest
• Geosciences
• Industrial Chemistry
• Islamic and Middle Eastern Studies
• Jewish Studies
• Library and Information Science, Book Studies
• Life Sciences
• Linguistics and Semiotics
• Literary Studies
• Materials Sciences
• Mathematics
• Social Sciences
• Sports and Recreation
• Theology and Religion
• Publish your article
• The role of authors
• Promoting your article
• Abstracting & indexing
• Publishing Ethics
• Why publish with De Gruyter
• How to publish with De Gruyter
• Our book series
• Our subject areas
• Your digital product at De Gruyter
• Contribute to our reference works
• Product information
• Tools & resources
• Product Information
• Promotional Materials
• Orders and Inquiries
• FAQ for Library Suppliers and Book Sellers
• Repository Policy
• Free access policy
• Open Access agreements
• Database portals
• For Authors
• Customer service
• People + Culture
• Journal Management
• How to join us
• Working at De Gruyter
• Mission & Vision
• De Gruyter Foundation
• De Gruyter Ebound
• Our Responsibility
• Partner publishers

Your purchase has been completed. Your documents are now available to view.

State-of-the-Art of STEAM Education in Science Classrooms: A Systematic Literature Review

STEAM education is designed to prepare students for the twenty-first-century life skills and has been extensively investigated in the past 10 years. Given the rapid evolution of educational practices and the diverse ways in which STEAM is implemented in science classrooms, there is an urgent need for an analysis of the current outlook of STEAM education within the science classrooms context. In this article, 22 empirical studies were included to highlight the methodology, STEAM characteristics, and pedagogical approaches implemented in STEAM-based learning. Findings suggest that STEAM education plays an important role in fostering students’ twenty-first-century skills and should be manifested by student-centered learning approaches. This study holds significant implications in providing direction for future studies regarding STEAM education in science classroom settings.

1 Introduction

Globalization and the current interconnected landscape have altered the face of education throughout the world. Trilling and Fadel ( 2009 ) asserted that a more comprehensive education must highlight the twenty-first-century skills, such as communication, collaboration, critical thinking, and creativity. Beyond content knowledge, a holistic education should prepare students to master twenty-first-century life skills (Teo, 2019 ). The twenty-first-century skills are crucial for students to succeed in the future (Stork, 2020 ) for sustainable development (Brandt, Barth, Merritt, & Hale, 2021 ). Taking creativity in solving problems as an example, it refers to the skill that tackles non-routine problems in the workspace. Organizations count on individuals who can identify the core problems and propose creative solutions. Therefore, students must be trained to enhance their thinking skills to evaluate, create, and apply relevant and essential knowledge to function in the twenty-first-century workplace (Beers, 2011 ).

Even though the twenty-first century life skills have long been argued to be important, the modern interconnected world has sparked renewed interest in these skills. The reconceptualization of life skills in the twenty-first century is a result of a paradigm shift from the twentieth to the twenty-first century. It is highlighted that throughout the twentieth century, education revolved around the three R’s: reading, writing, and arithmetic. The new three R’s, namely “relevance,” “rigor,” and “real world skills,” are fundamental to educational objectives in the twenty-first century (Smith & Hu, 2013 ; Teo, 2019 ). This shift undoubtedly entails a serious consequence in the twenty-first-century education setting. Today’s evolving society and work–life require the young generation to have good collaboration skills (Scardamalia, Bransford, Kozma, & Quellmalz, 2012 ). Collaboration is now taking place in a diverse cross-cultural environment. Communicating, collaborating, and working with people from different backgrounds, disciplines, and expertise using cutting-edge technological devices are inevitable in the twenty-first century (Graesser, Kuo, & Liao, 2017 ; Scardamalia et al., 2012 ). Collaboration, critical thinking, problem-solving, and creativity have all become important aspects of global education. The traditional method of teaching students’ basic numeracy and literacy skills is no longer relevant for today’s education (Teo, 2019 ). Thus, the teaching approaches must now be shifted from teacher-centered to student-centered learning.

As far as student-centered learning is concerned, many researchers have investigated and confirmed that STEAM education can be an effective student-centered learning approach to drive students’ twenty-first-century skills (Herro & Quigley, 2017 ; Jesionkowska, Wild, & Deval, 2020 ; Liao, 2016 ; Perignat & Katz-Buonincontro, 2019 ). It provides learning that is real-world-driven and problem-based in nature (Mehta, Keenan, Henriksen, & Mishra, 2019 ). The real-world problem-solving learning scenarios are claimed to significantly develop students’ twenty-first-century skills (Kozlov & Shemshurina, 2018 ). Henriksen, Mehta, and Mehta ( 2019 ) suggested that the essence of STEAM education honors the arts and sciences by involving the empirical and intuitive, the rational, and the aesthetic equally. Furthermore, Liao ( 2016 ) argued that STEAM education opens a transdisciplinary space, where it can be effectively promoted in science education classrooms (Bao & Koenig, 2019 ; Hanson, 2020 ). Therefore, one of the essential benefits of implementing STEAM education is to use and integrate science, technology, engineering, arts, and mathematics to develop students’ twenty-first-century skills through a student-centered way of learning. From previous studies, STEAM has been proven to minimize misconceptions (Ozkan & Topsakal, 2020) and reduce cognitive load (Chen & Huang, 2020 ). In addition, there are plenty of opportunities to teach the fundamental and applied knowledge of science in the real-world context using STEAM education. For example, some topics in science education entangle ethical and societal implications, which in turn require transdisciplinary concerns, such as climate change and global warming (Singh, 2020 ). The STEAM approach may give students real experiences to observe and explore abstract concepts in physics (Ozkan & Topsakal, 2020). Therefore, it is not surprising that Bao and Koening (2019) claimed that the promotion of integrated STEAM learning to meet the learning objectives of the twenty-first century could be effectively accomplished.

1.1 Existing Review on STEAM Education

Existing reviews have reported both general and specific aspects of STEAM education pedagogy and research without an explicit focus on science classroom settings. This includes the general analysis by Belbase et al. ( 2022 ) reporting the problems, processes, priorities, and prospects in STEAM education. Based on their review, Belbase et al. ( 2022 ) proposed a new alternative thinking by extending STEAM to the ecological and humanistic well-being, defined as STEAM-h, where “h” refers to humanity for future sustainable environmental balance. The work of Belbase et al. ( 2022 ) is an example that illustrates the effort to understand and define STEAM education. Such efforts create different views and varying definitions about STEAM education.

The inconsistencies in defining STEAM terminology, including the matter of defining “science,” “technology,” “engineering,” “arts,” and “mathematics” within the STEAM acronym, cause difficulties for STEAM pedagogy and research (Colucci-Gray et al., 2017 ; Perignat & Katz-Buonincontro, 2019 ). For example, very recently, Leavy, Dick, Meletiou‐Mavrotheris, Paparistodemou, and Stylianou ( 2023 ) engaged in a systematic literature review identifying the use and prevalence of up-and-coming technologies within the landscape of STEAM education. They found that although various kinds of emerging technologies, e.g., extended realities, maker spaces, and robotics, have been used in the design of STEAM learning, there are discrepancies in the way how they are defined and implemented (Leavy et al., 2023 ). In Leavy et al.’s ( 2023 ) review, the research methodologies and study design characteristics of STEAM education were not properly captured because they were not available in many of the extracted studies. This is probably because Leavy et al. ( 2023 ) also included grey literature in their review, which leads to selective publication bias. Perignat and Katz-Buonincontro ( 2019 ) reviewed how reported studies define the “arts” within the context of STEAM education. Perignat and Katz-Buonincontro ( 2019 ) discovered that, from 44 reviewed articles, not only did researchers fail to be consistent in defining “arts,” but also many of the studies did not support the claim that STEAM education is to enhance problem-solving, creativity, and other twenty-first-century skills. Nearly 55% of the reviewed articles mentioned such learning outcomes in their introductions with lack of explanation. This indicates that many practitioners are struggling to integrate “arts” in STEAM education and measuring the relevant learning outcomes (Herro & Quigley 2017 ; LaJevic 2013 ).

From the bibliographic analysis results reported by Marín-Marín, Moreno-Guerrero, Dúo-Terrón, and López-Belmonte ( 2021 ), one of their highlights was a trend in relating STEAM education and computational thinking. In response to their findings, Zhang, Ng, and Leung ( 2023 ) reported a descriptive review on investigating computational thinking and STEAM education in the context of early childhood education. In the other specific context, Li and Wong ( 2023 ) conducted a comprehensive review of STEAM education for personalized learning. They categorized personalization in STEAM education as applied in different modes of education, subject disciplines, and across various countries (Li & Wong, 2023 ). They found that the trend of research in this particular topic has been increasing over the last 10 years, yet many of the included studies only focused on one STEAM discipline (Li & Wong, 2023 ). Li and Wong ( 2023 ) therefore suggested that more integrative STEAM practices should be implemented in the context of personalized learning. The limitation of Li and Wong ( 2023 ) work relies on the use of only one database, namely Scopus. A single database searching may generate a biased representation in a literature review (Kugley et al., 2017 ).

Despite the growing body of research on STEAM education, there is a notable gap in reviews focusing specifically on its implementation within science classroom settings. General reviews of STEAM education (e.g., Belbase et al., 2022 ; Perignat & Katz-Buonincontro, 2019 ) and specific contexts, such as computational thinking (Marín-Marín et al., 2021 ) and personalized learning (Li & Wong, 2023 ), have been published. However, a comprehensive review of STEAM integration in science classrooms remains absent. It is crucial to review the implementation of STEAM education in science classrooms not only because science education is fundamental for global educational (Martins & Veiga, 2001 ), but also for addressing global challenges and fostering sustainable development (Li, Sjöström, Ding, & Eilks, 2022 ; Won et al., 2021 ). The attempt to develop effective approaches for science learning based on STEAM education can be enhanced by evaluating its current state-of-the-art. In addition, our search in Web of Science (WOS) and Scopus databases, using keywords “STEAM” AND “Education” within the years 2014–2023, reveals that there are no reported reviews on the STEAM education in the context of the science classroom. Therefore, this article is the first to capture the integration of STEAM learning in science classroom settings. The article provides insights into how STEAM learning enhances science education.

Since resolving real-world problems requires scientifically informed decisions, science education is therefore critical to prepare students to understand and generate creative solutions to real-world problems (Won et al., 2021 ). In line with that STEAM learning promotes a scientific understanding of real-world phenomena and various possible approaches to solving problems at individual and social levels (Quigley, Herro, Shekell, Cian, & Jacques, 2020a ; Wilson et al., 2021 ). Implementing STEAM learning in science classrooms has become a substantial response toward education for sustainable development (Sachs et al., 2019 ). However, studies have suggested that teachers had difficulty in implementing STEAM education, particularly in science classes. This is reported that teachers who want to adopt STEAM in their classrooms have difficulties in selecting integrated topics and having a tendency to reduce the science content (Son & Jung, 2019 ). Therefore, the article holds significant insight into adopting STEAM learning in science classroom settings.

1.2 Research Questions

What education levels, subject areas, and research methodologies dominate in science classroom STEAM learning studies?

What are the definitions and integration levels of STEAM adopted in science classroom research?

What are the pedagogical approaches applied in science classroom STEAM learning?

In this study, a systematic literature review with well-defined criteria for inclusion is presented. The selection procedure was conducted following the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) 2020 statement (Page et al., 2021 ) based on a transparently recorded and properly framed search procedure (Alexander, 2020 ). Taking state-of-the-art of STEAM education research in science classrooms as the focus of review, 22 articles published in international journals from WOS and Scopus databases are included in this study. Figure 1 shows the systematic literature search following the PRISMA 2020 (Page et al., 2021 ).

Procedure of article selection adapted from PRISMA 2020 flow diagram.

2.1 Minimizing Bias

A systematic literature review is done because researchers’ concern about the efforts of minimizing bias to obtain reproducible, objective, and thorough search results (Kugley et al., 2017 ). In this article, three common biases were taken into consideration, i.e., representation, judgment, and publication. First, Kugley et al. ( 2017 ) highlighted that the results may not represent all the studies if the search is only done in one database. Rice et al. ( 2016 ) recommended considering multi-database searches, moreover when the topic of interest is cross-disciplinary, like STEAM education. To minimize the result bias, WOS and Scopus databases were chosen because they are one of the biggest bibliographic databases. Second, Lasserson, Thomas, and Higgins ( 2019 ) addressed that too many judgments may arise during systematic reviews. Therefore, a good review protocol must be developed prior to the document search, wherein PRISMA 2020 protocol is applied to minimize judgment bias. Lastly, publication bias can be a big concern for literature review. Although Hopewell, Loudon, Clarke, Oxman, and Dickersin ( 2009 ) discovered that unpublished works were not necessarily of poorer quality than published studies, it is argued that the published articles have generally undergone a review process to ensure their quality. As highlighted by Bakker and Traniello ( 2019 ), peer-reviewed publications ensure that the published works have been scientifically validated by experts and therefore can be trusted. Hence, to minimize publication bias and ensure reproducibility, only research papers published in peer-reviewed international journals were included in this systematic literature review.

2.2 Search Strategy

In this systematic review, search terms “STEAM education” AND “science,” “STEAM” AND “science education,” “STEAM education” AND “physics,” “STEAM education” AND “biology,” and “STEAM education” AND “chemistry” were used. The searches were executed in WOS and Scopus databases in the timespan from 1 January 2012 to 31 May 2023. The strings used in the literature review search are provided in Table 1 . The initial search from WOS and Scopus resulted in 773 and 465 articles, respectively. However, documents from article reviews, conference papers, conference reviews, books, book chapters, editorial materials, meeting abstracts, short surveys, and notes were automatically excluded from the initial search. Therefore, 289 and 350 documents were respectively recorded from WOS and Scopus for further screening and inclusion stages.

Strings used in the literature review search

2.3 Selection Criteria

These records were identified and screened based on the specific selection criteria to justify the eligibility for inclusion. In this study, PRISMA 2020 (Page et al., 2021 ) was applied. The PRISMA 2020 statement is the updated version of the earlier version, PRISMA 2009 (Moher, Liberati, Tetzlaff, Altman, & PRISMA Group, 2009 ). The PRISMA 2020 consists of 27 checklist items, the expanded version of the previous version, to ensure a transparent and accurate systematic literature review. It also includes a three-phase flow diagram, i.e., identification, screening, and inclusion, as the systematic procedure for review ( Figure 1 ). This systematic process ensures the accountability and consistency of systematic literature analysis documentation. Herein, the search focuses on mapping existing empirical studies on the implementation of STEAM learning in science classrooms context.

First, the documents underwent general selection criteria, as depicted in Table 2 . Only English-written original research articles published in the ranges of January 2012 and May 2023 in the scopes of education, social sciences, and humanities were included. Next, more specific criteria for inclusion, as given in Table 3 , were applied to the identified records (350 and 289 documents, respectively, from Scopus and WOS).

General selection criteria

Specific criteria for inclusion

2.4 Quality Assessment

The specific criteria for inclusion ( Table 3 ) allow us to maintain the good quality of the review. The identification, screening, and inclusion process are shown in Figure 1 . This protocol was conducted by two independent reviewers (authors). In cases of discrepancies between the reviewers during this process, a consensus was reached through discussion between the reviewers. The 639 total identified records were carefully checked for specific inclusion criteria. The filtration of duplicate documents resulted in the exclusion of 97 records. Further exclusion of documents that did not meet criteria 1 (document type) and 2 (language) yielded 231 saved articles for further screening. The title, abstract, and keywords screening were then executed to test the sample on criteria 3 (keyword appearance) and 4 (original research article identification). This resulted in the rejection of 119 records that were not relevant to the implementation of STEAM education in classroom settings. During this screening process, one article’s abstract was not found, and another article was retracted. Therefore, 110 articles were further assessed for eligibility, i.e., tested on criteria 5 (focus on science-related subjects) and 6 (published in peer-reviewed international journals). This eligibility test led to the final inclusion of 22 published studies that were further analyzed to answer the research questions.

3.1 Education Levels, Subject Areas, and Research Methodologies that Dominate in Science Classroom STEAM Learning Studies

Table 4 shows a list of the articles included in this study, highlighting the country, subject area, and educational level. Studies on STEAM education in a science classroom context have been conducted in many countries. The international scope of the studies is evident, with contributions from countries such as Greece (Conradty & Bogner, 2019 ), Spain (Bassachs et al., 2020 ), Thailand (Khamhaengpol, Sriprom, & Chuamchaitrakool, 2021 ), China (Jia, Zhou, & Zheng, 2021 ), Indonesia (Rahmawati, Taylor, Taylor, Ridwan, & Mardiah, 2022 ), and Saudi Arabia (Alkhabra, Ibrahem, & Alkhabra, 2023 ). This geographical diversity represents the global significance of effective STEAM education practices in the science classroom context.

List of articles included in the study

Out of the 22 studies, 12 focused on middle school education, such as those conducted by Quigley and Herro ( 2016 ) in the USA and Park and Park ( 2020 ) in Korea. It emphasizes the importance of tailoring interventions to address the unique challenges and learning needs of students in this crucial stage of academic development. Additionally, investigations at the elementary school level were conducted by researchers like Bassachs et al. ( 2020 ), Cheng et al. ( 2022 ), and Wilson et al. ( 2021 ). Only two studies were conducted at high school levels (Conradty & Bogner, 2019 ; Khamhaengpol et al., 2021 ). The distribution of articles based on the school level is provided in Figure 2 .

Distribution of articles based on school level.

The 22 articles included in this review spanned various science subjects, ranging from Geoscience to Nanotechnology. Out of the 22 articles, 13 focused on Science in general, 5 emphasized Physics, 3 focused on chemistry, and the remaining articles addressed Geoscience and Nanotechnology. The distribution of articles based on subject area is shown in Figure 3 . Furthermore, research methodologies can be classified into three different types, i.e., quantitative research, qualitative research, and mixed methods (Creswell & Creswell, 2017 ; Snyder, 2019 ). In this review, 14 studies employed a quantitative approach, 7 studies were based on qualitative data, and the remaining two studies used mixed methods. The distribution of articles based on research methodology is shown in Figure 4 . Meanwhile, the details of the data collection techniques are shown in Table 5 .

Distribution of articles based on subject area.

Distribution of articles based on research methodology.

Article distribution based on the methodology

Test, which is designed to measure specific variables or constructs in a systematic and standardized manner. It includes pre- and post-tests (Alkhabra et al., 2023 ; Arpaci et al., 2023 ; Cheng et al., 2022 ; Conradty & Bogner, 2019 ; Hsiao et al., 2022 ; Hughes et al., 2022 ; Jia et al., 2021 ; Ozkan & Umdu Topsakal, 2021b ; Tran et al., 2021a , b ), worksheets (Khamhaengpol et al., 2021 ), and questions (Bassachs et al., 2020 ).

Survey, which involves collecting data from a sample of individuals via the administration of a set of standardized questions. It includes pre- and post-survey (Gates, 2017 ), digital survey (Jiang et al., 2020 ), and survey open-ended questions (Wilson et al., 2021 ).

Case study, which involves an in-depth, detailed examination of a specific instance or case. The articles that employed the case study were authored by Park and Park ( 2020 ).

Observation, which involves carefully and systematically watching and recording behaviors, events, or interactions. It can be done using observational rubrics (Quigley et al., 2020a , b ; Rahmawati et al., 2022 ).

Interview, which aims to explore participants’ perspectives, experiences, attitudes, and opinions on a particular topic. Interviews were conducted by Jiang et al. ( 2020 ), Ozkan and Topsakal ( 2020 ), Quigley et al. ( 2020a ), and Rahmawati et al. ( 2022 ).

Based on the findings, it is discovered that STEAM education in science classrooms spans across multiple countries, including Greece, Spain, Thailand, China, Indonesia, and Saudi Arabia. It implied that there is a global significance of effective STEAM practices. Furthermore, many of the included studies focused on middle school education, covering diverse science subjects. Research methodologies in STEAM education research varied from quantitative approaches to mixed methods. Mixed methods combined quantitative and qualitative techniques to provide comprehensive findings into STEAM learning practices in science classrooms.

3.2 Definitions and Integration Levels of STEAM Adopted in Science Classroom Research

Table 6 shows how the researchers define STEAM education in their research in the context of science classrooms. The table depicts the STEAM definition and the corresponding level of integration. In terms of integration level, as can be seen from Figure 5 , STEAM education can be classified in four levels, i.e., disciplinary, multidisciplinary, interdisciplinary, and transdisciplinary integration (Vasquez, Sneider, & Comer, 2013 ).

Definitions of STEAM adopted in the literature

Levels for integrated learning (adapted from Vasquez et al., 2013 ).

The disciplinary STEAM approach is defined by the separation of concepts and skills associated with each STEAM component. The term “multidisciplinary” refers to the study of multiple disciplines within a common theme. Interdisciplinary STEAM requires a strong connection between two or more STEAM disciplines to develop deeper knowledge and abilities. Transdisciplinary STEAM is considered the highest level of integration. The transdisciplinary approach to problem-solving requires the application of knowledge and abilities from two or more subjects. In transdisciplinary STEAM, students engage in problem-solving activities that demand the synthesis of concepts, methodologies, and techniques from different disciplines (Aguilera & Ortiz-Revilla, 2021 ). This approach encourages collaboration, communication, and the development of holistic thinking skills. Transdisciplinary STEAM projects often tackle complex, real-world problems that do not fit neatly within the confines of a single subject (Vasquez et al., 2013 ).

The distribution of articles based on level of integration is given in Figure 6 . The analysis of the provided data on STEAM integration levels suggests a predominant emphasis on Interdisciplinary approaches, cited by 11 of the authors. This indicates a shared perspective among scholars that STEAM education thrives on collaborative and cross-disciplinary interactions. Multidisciplinary approaches, mentioned by six authors, underscore the importance of enriching STEM subjects with Arts or combining specific disciplines to achieve learning objectives. The less frequent but notable presence of Transdisciplinary approaches was mentioned by five authors. It suggests a holistic problem-solving orientation, where collective expertise from various disciplines is harnessed to address real-world problems.

Distribution of articles based on level of integration.

From the definitions of STEAM adopted in the literature depicted in Table 6 , the characteristics of STEAM integration levels and their relevancies for education are inferred in Table 7 . Researchers have delineated distinct perspectives on the definitions and integration levels of STEAM education in the science classroom context. Transdisciplinary STEAM, as expounded by Quigley and Herro ( 2016 ), emphasizes a holistic nature within the formal education context in addressing real-world issues. This approach involves the integration of various disciplines to address problems and focuses on the issue itself rather than a specific discipline (Quigley et al., 2020a ; Wilson et al., 2021 ). Thus, transdisciplinary STEAM fosters a comprehensive real-world problem-solving framework. Meanwhile, multidisciplinary STEAM enriches STEM subjects with the inclusion of arts (Conradty & Bogner, 2019 ; Hughes et al., 2022 ). This approach aims to facilitate students’ disciplinary identity development (Jiang et al., 2020 ) and combines arts with STEM subjects (Cheng et al., 2022 ; Khamhaengpol et al., 2021 ; Rahmawati et al., 2022 ). It offers a balanced integration of disciplines that make it suitable for contexts where the Arts incorporation into specific STEM disciplines becomes the primary goal. Although transdisciplinary STEAM is less frequently advocated by the researchers, it promotes a holistic approach to real-world problem-solving through the integration of multiple disciplines. Alternatively, interdisciplinary STEAM emphasizes the integration of science, technology, engineering, arts, and mathematics (STEM), which highlights core competencies and emphasizes on integration of various disciplines (Bassachs et al., 2020 ; Gates, 2017 ; Park & Park 2020 ). This perspective signifies an interdisciplinary education strategy that encourages experiential learning and shifts from traditional teaching paradigms (Arpaci et al., 2023 ; Hsiao et al., 2022 ; Jia et al., 2021 ; López-Banet et al., 2021 ; Ozkan & Topsakal, 2021; Tran et al., 2021a , b ).

Comparative analysis of integration levels in STEAM education

3.3 Pedagogical Approaches Applied in Science Classroom STEAM Learning

Table 8 summarizes the learning approaches for STEAM education in the context of science classrooms. Various pedagogical approaches have been applied in science classroom STEAM learning. Project-based learning emerges as the most frequently cited approach, appearing six times in the data. Arts/Technology-based learning follows closely with four instances. Inquiry-based learning is referenced three times. Contextual learning was cited in two articles. Problem-based learning, engineering design-based learning, role-taking-based learning, explicit learning, and reflective learning each appear once in the reviewed articles. The distribution of articles based on the pedagogical approach is shown in Figure 7 .

Categories of strategies used in integrating STEAM in science classrooms

Distribution of articles based on a pedagogical approach.

Arts/technology-based learning, applied by Alkhabra et al. ( 2023 ), Gates ( 2017 ), Jia et al. ( 2021 ), and Ozkan and Topsakal ( 2021 ), recognizes the value of combining creativity, artistic expression, and technological skills to enhance the overall educational experience.

Project-based learning is an instructional approach where students actively engage in complex and real-world problems, resulting in a presentation or product for them to acquire knowledge and skills (Hypolite & Rogers, 2023 ). It was subscribed by Cheng et al. ( 2022 ), Hasio et al. (2022), Rahmawati et al. ( 2022 ), Tran et al. ( 2021a , b ), and Wilson et al. ( 2021 ).

Problem-based learning, adopted by adopted by Quigley et al. ( 2020a ), allows students to apply knowledge and skills to generate solutions to a defined problem (Savery, 2015 ).

Reflective learning, adopted by Bassachs et al. ( 2020 ), involves the activation of experiential knowledge through ongoing inquiry and self-assessment and ensures adaptability and responsiveness to the dynamic challenges of the environment (Sachs et al., 2019 ).

Explicit learning, adopted by Park and Park ( 2020 ), is defined as carefully planned lessons presented to students with the aim of assisting them in gaining a comprehensive understanding of a particular subject (Park, 2008 ).

Role-taking-based learning, followed by Jiang et al. ( 2020 ), incorporates activities or methods where students engage in taking on different roles or perspectives to develop science identities (Jiang et al., 2020 ).

Engineering design-based learning, applied by Khamhaengpol et al. ( 2021 ), emphasizes active engagement, problem-solving, and the application of engineering principles in a practical context.

Inquiry-based learning, adopted by Conradty and Bogner ( 2019 ), Hughes et al. ( 2022 ), López-Banet et al. ( 2021 ), refers to actively involved students by establishing connections to the real world through exploration and use of advanced questioning techniques (NRC, 2013 ).

Contextual learning, adopted by Arpaci et al. ( 2023 ) and Ozkan and Topsakal ( 2020 ), emphasizes the importance of placing learning within a meaningful context or setting (Hwang, Hariyanti, Chen, & Purba, 2023 ).

In the exploration of pedagogical approaches within the context of STEAM education in science classroom settings, project-based learning emerges as the predominant method. Many of studies showed its effectiveness in engaging students with real-world problems and fostering creativity. Since the “Arts” distinguishes STEAM from STEM, Arts/technology-based learning also plays a significant role. This learning approach emphasizes the integration of artistic and technological skills to enhance educational experiences. However, no studies investigate students’ creativity when they apply arts/technology-based learning. Other student-centered learning strategies, for example, inquiry-based learning, contextual learning, and role-taking-based learning, have been studied for their roles in implementing STEAM education in science classrooms. Each approach brings unique strengths to STEAM education, collectively offering a diverse range of strategies to enhance science learning outcomes.

4 Discussion and Implication

The first research question deals with education levels, subject areas, and research methodologies that dominate in science classroom STEAM learning studies. Overall, the diverse application of STEAM learning in various education levels and scientific subjects, coupled with the variety of applied research methodologies, suggests a multifaceted exploration of STEAM education in the science classrooms context. The review implied that 2, 9, and 12 studies reported the implementation of STEAM learning in elementary, middle, and high schools, respectively. The results suggest a notable distribution of STEAM learning implementation across different education levels, with a predominant focus on middle schools. Additionally, the incorporation of STEAM learning in various science-based subjects, for example, physics, chemistry, and nanotechnology, highlights its versatility and applicability across different scientific domains. The prevalence of quantitative methods in 14 studies suggests a quantitative approach to measuring the effectiveness and outcomes of STEAM interventions. Conversely, the seven studies utilizing qualitative methods likely delved into the experiential and nuanced aspects of STEAM implementation, providing a richer understanding of the learning process. The two studies employing mixed methods indicate a comprehensive investigation that combines both quantitative and qualitative data. Mixed-methods studies offer a more holistic perspective on STEAM learning (Ozkan & Umdu Topsakal, 2021a ; Wilson et al., 2021). Since only two studies have reported the use of mixed-methods for STEAM education in science classrooms, there is a need for more mixed-methods studies in this context. Mixed-methods research can be used to leverage the strengths of both quantitative and qualitative approaches while mitigating their limitations (Creswell & Creswell, 2017 ). By combining quantitative and qualitative data, researchers can gain a more comprehensive and nuanced understanding of a phenomenon, as well as corroborate or refute findings from one method with another (Creswell & Creswell, 2017 ).

The second research question focuses on definitions and integration levels of STEAM adopted in science classroom research. Authors of the reviewed articles tended to define STEAM education in the same way when the integration level was multidisciplinary. At the multidisciplinary level, STEAM is defined as Arts + STEM (Cheng et al., 2022 ; Conradty & Bogner, 2019 ; Hughes et al., 2022 ; Khamhaengpol et al., 2021 ; Rahmawati et al., 2022 ). Jiang et al. ( 2020 ) highlighted that the combination of Arts and STEM is to facilitate middle school students’ disciplinary identity development. At the multidisciplinary level, STEAM is defined as the acronyms that combine science, technology, engineering, arts, and mathematics (Arpaci et al., 2023 ; Gates, 2017 ; Hsiao et al., 2022 ). Other researchers explicitly use the word “interdisciplinary” to define STEAM education (López-Banet et al., 2021 ; Tran et al., 2021a , b ). Ozkan and Topsakal ( 2020 , 2021 ) define STEAM means the shift from “what to be taught” to “what I shall experience,” and it requires two or more disciplines to be combined into a specific concept to achieve the learning objectives. More than just the combination of Arts and STEM, STEAM must rely heavily on integrating the arts into STEM to reshape education in the sciences and humanities (Bassachs et al., 2020 ).

Furthermore, more diverse definitions of STEAM were found in the highest level of integration, specifically in transdisciplinary STEAM learning. Solving real-world problems becomes the keyword in transdisciplinary STEAM learning, more than just emphasizing multidiscipline. Well-defined transdisciplinary STEAM learning was adopted by some researchers. For example, Quigley and Herro ( 2016 ) stated that STEAM emphasizes the very nature of holistic in the context of formal education and emerges from disciplinary practices, and Quigley et al. ( 2020b ) highlighted that transdisciplinary STEAM approaches use the collective expertise from various disciplines to address and resolve problems, with a focus on emphasizing the problem itself rather than emphasizing a specific discipline.

The findings on definitions and integration levels of STEAM in science classrooms reveal a noteworthy consistency in how authors define STEAM education when the integration level is multidisciplinary. Moreover, diverse terminologies such as acronyms combining science, technology, engineering, arts, and mathematics, as well as the explicit use of the term interdisciplinary, highlight some flexibility in conceptualizing and communicating the multidisciplinary nature of STEAM. As the integration level progresses to transdisciplinary STEAM learning, the definitions become more varied, with a heightened emphasis on problem-solving as a central component. Thus, these implications provide a nuanced understanding of STEAM education, which encompasses diverse perspectives that range from disciplinary identity development to transdisciplinary problem-solving.

For researchers and educators, the choice between these STEAM integration levels is contingent upon the specific goals and context of the study or educational implementation. Transdisciplinary STEAM is apt for addressing complex and real-world problems in a holistic manner. Multidisciplinary STEAM offers a balanced integration among the disciplines, while interdisciplinary STEAM provides a flexible approach that encourages diverse perspectives. Ultimately, the decision should align with the objectives of the study or educational initiative that ensure a tailored and effective integration of STEAM principles.

The last research question concerns about pedagogical approaches applied in science classroom STEAM learning. It was captured that project-based learning, arts/technology-based learning, inquiry-based learning, and contextual-based learning dominate in the implementation of STEAM education in science classroom settings. The reviewed articles show that STEAM learning was implemented to facilitate students to master science knowledge and twenty-first-century skills, for example, creativity (Cheng et al., 2022 , Conradty & Bogner, 2019 ; Hasio et al., 2022; Wilson et al., 2021), scientific creativity (Tran et al., 2021a , b ), critical thinking (Alkhabra et al., 2023 ; Rahmawati et al., 2022 ; Wilson et al., 2021), and collaborative decision-making (Rahmawati et al., 2022 ). The findings regarding pedagogical approaches in science classroom STEAM learning carry significant implications for educational practices. The research underscores the prevalence of diverse instructional methods, with student-centered active learning approaches dominating in implementing STEAM education within science classrooms. The reviewed articles highlight that the implementation of STEAM learning in science classrooms is geared toward fostering essential twenty-first t-century skills. As such, these findings advocate for the continued exploration and incorporation of innovative pedagogical approaches in science classrooms to effectively integrate STEAM education and prepare students for the challenges and opportunities of the twenty-first century.

5 Conclusion and Recommendations

This article attempts to analyze the published works on the implementation of STEAM education in science classroom context. STEAM education was dominantly implemented in the middle schools for various science subjects. Furthermore, quantitative data collections were widely applied by a majority of the researchers to investigate the effect of STEAM learning toward students’ knowledge and twenty-first-century skills. Researchers agreed on the same STEAM definition at the multidisciplinary level. Half of the reviewed articles incorporated interdisciplinary STEAM learning where a strong connection between two or more STEAM disciplines to develop deeper knowledge and abilities is required. At the transdisciplinary level, more variations of the STEAM definition were adopted by different authors. However, all were in line to justify that in transdisciplinary STEAM, students must be able to solve real-world problems. In science classroom settings, STEAM learning is mainly promoted by project-based learning, arts/technology-based learning, inquiry-based learning, and contextual-based learning. The literature consistently shows that regardless of the integration level applied, the pedagogical approach for STEAM education should be based on student-centered and active learning strategies. The findings suggest that there are diverse investigations on STEAM learning within the science classroom contexts.

While the existing body of research provides valuable insights into STEAM education within science classrooms context, the lack of a standardized definition, particularly in transdisciplinary STEAM learning, poses a challenge for educators. To address this issue, urgent attention should be directed toward further studies that establish a unified STEAM definition and framework. This is important to facilitate the seamless adoption of transdisciplinary STEAM by science teachers. The transdisciplinary STEAM should emphasize the integration of multiple disciplines to solve real-world problems in such a way that students are prepared with the twenty-first-century skills. With that, a decisive shift toward student-centered and active learning strategies is recommended. In this sense, project-based learning can be applied as the best approach for transdisciplinary STEAM as it contributes to a more comprehensive and engaging STEAM education within science classrooms. Furthermore, the next studies should also focus on developing standardized metrics to assess the impact of STEAM learning in science classrooms, especially on the students’ twenty-first-century skills. Mixed-methods studies should be more extensively adopted to allow a flexible and adaptable research design that can accommodate different research questions, settings, and populations. More importantly, future research should attempt to establish a cohesive framework for STEAM education to facilitate consistency across studies.

Furthermore, it is important to acknowledge certain limitations that may affect the generalizability of the findings in this article. The scope of this article is limited to published works available in Scopus and WOS databases in the timespan from 1 January 2012 to 31 May 2023, which may not encompass all relevant studies on STEAM education from other recourses. Additionally, while the review highlights the prevalence of certain pedagogical approaches, it does not examine deeply the contextual factors, for example, cultural contexts, educational settings, and resource availability, that may influence the effectiveness of these approaches. Therefore, future studies should explore the impact of those contextual variables on the effectiveness of STEAM pedagogies.

Funding information: The research was supported by Faculty of Mathematics and Natural Sciences, Universitas Negeri Malang, Indonesia, and the Ministry of Higher Education Malaysia under the Fundamental Research Grant Scheme (FRGS/1/2023/SSI07/UTM/01/3) managed by Universiti Teknologi Malaysia (R.J130000.7853.5F609).

Author contributions: Erni Yulianti: investigation, methodology, writing – original draft, writing – review and editing, visualization. Hadi Suwono: conceptualization, formal analysis, writing – review and editing. Nor Farahwahidah Abd Rahman: resources, data curation, visualization. Fatin Aliah Phang: resources, methodology, formal analysis, writing – review and editing.

Conflict of interest: The authors state no conflict of interest.

Data availability statement: The data supporting the conclusion of this article are included in the article. Any queries regarding these data may be directed to the corresponding authors.

Aguilera, D., & Ortiz-Revilla, J. (2021). STEM vs STEAM education and student creativity: A systematic literature review. Education Sciences, 11(7), 331. doi: 10.3390/educsci11070331 . Search in Google Scholar

Alexander, P. A. (2020). Methodological guidance paper: The art and science of quality systematic reviews. Review of Educational Research, 90(1), 6–23. doi: 10.3102/0034654319854352 . Search in Google Scholar

Alkhabra, Y. A., Ibrahem, U. M., & Alkhabra, S. A. (2023). Augmented reality technology in enhancing learning retention and critical thinking according to STEAM program. Humanities and Social Sciences Communications, 10(1), 1–10. doi: 10.1057/s41599-023-01650-w . Search in Google Scholar

Arpaci, I., Dogru, M. S., Kanj, H., Ali, N., & Bahari, M. (2023). An experimental study on the implementation of a STEAM-based learning module in science education. Sustainability, 15(8), 6807. doi: 10.3390/su15086807 . Search in Google Scholar

Bakker, T. C., & Traniello, J. F. (2019). Peer-review reciprocity and commitment to manuscript evaluation. Behavioral Ecology and Sociobiology, 73, 1–3. doi: 10.1007/s00265-019-2647-2 . Search in Google Scholar

Bao, L., & Koenig, K. (2019). Physics education research for 21st century learning. Disciplinary and Interdisciplinary Science Education Research, 1(1), 2. doi: 10.1186/s43031-019-0007-8 . Search in Google Scholar

Bassachs, M., Cañabate, D., Nogué, L., Serra, T., Bubnys, R., & Colomer, J. (2020). Fostering critical reflection in primary education through STEAM approaches. Education sciences, 10(12), 384. doi: 10.3390/educsci10120384 . Search in Google Scholar

Beers, S. (2011). Teaching 21st century skills: An ASCD action tool. Virginia, USA: ASCD. Search in Google Scholar

Belbase, S., Mainali, B. R., Kasemsukpipat, W., Tairab, H., Gochoo, M., & Jarrah, A. (2022). At the dawn of science, technology, engineering, arts, and mathematics (STEAM) education: Prospects, priorities, processes, and problems. International Journal of Mathematical Education in Science and Technology, 53(11), 2919–2955. doi: 10.1080/0020739X.2021.1922943 . Search in Google Scholar

Brandt, J.-O., Barth, M., Merritt, E., & Hale, A. (2021). A matter of connection: The 4 Cs of learning in pre-service teacher education for sustainability. Journal of Cleaner Production, 279, 123749. doi: 10.1016/j.jclepro.2020.123749 . Search in Google Scholar

Chen, C. C., & Huang, P. H. (2020). The effects of STEAM-based mobile learning on learning achievement and cognitive load. Interactive Learning Environments, 3(1),1–17. doi: 10.1080/10494820.2020.1761838 . Search in Google Scholar

Cheng, L., Wang, M., Chen, Y., Niu, W., Hong, M., & Zhu, Y. (2022). Design my music instrument: A project-based science, technology, engineering, arts, and mathematics program on the development of creativity. Frontiers in Psychology, 12, 763948. doi: 10.3389/fpsyg.2021.763948 . Search in Google Scholar

Colucci-Gray, L., Trowsdale, J., Cooke, C. F., Davies, R., Burnard, P., & Gray, D. S. (2017). Reviewing the potential and challenges of developing STEAM education through creative pedagogies for 21st learning: How can school curricula be broadened towards a more responsive, dynamic, and inclusive form of education? London, UK: British Educational Research Association. Search in Google Scholar

Conradty, C., & Bogner, F. X. (2019). From STEM to STEAM: Cracking the code? How creativity and motivation interacts with inquiry-based learning. Creativity Research Journal, 31(3), 284–295. doi: 10.1080/10400419.2019.1641678 . Search in Google Scholar

Creswell, J. W., & Creswell, J. D. (2017). Research design: Qualitative, quantitative, and mixed methods approaches. California, USA: Sage publications. Search in Google Scholar

Gates, A. E. (2017). Benefits of a STEAM collaboration in Newark, New Jersey: Volcano simulation through a glass-making experience. Journal of Geoscience Education, 65(1), 4–11. doi: 10.5408/16-188.1 . Search in Google Scholar

Graesser, A., Kuo, B., & Liao, C. (2017). Complex problem solving in assessments of collaborative problem solving. Journal of Intelligence, 5(2), 1–14. doi: 10.3390/jintelligence5020010 . Search in Google Scholar

Hanson, N. R. (2020). The contributions of other disciplines to nineteenth century physics. In N. R. Hanson & M. D. Lund (Eds.), What I do not believe, and other essays (pp. 147–156). Netherlands: Springer. doi: 10.1007/978-94-024-1739-5_7 . Search in Google Scholar

Henriksen, D., Mehta, R., & Mehta, S. (2019). Design thinking gives STEAM to teaching: A framework that breaks disciplinary boundaries. Switzerland: Springer. doi: 10.1007/978-3-030-04003-1_4 . Search in Google Scholar

Herro, D., & Quigley, C. (2017). Exploring teachers’ perceptions of STEAM teaching through professional development: Implications for teacher educators. Professional Development in Education, 43(3), 416–438. doi: 10.1080/19415257.2016.1205507 . Search in Google Scholar

Hopewell, S., Loudon, K., Clarke, M. J., Oxman, A. D., & Dickersin, K. (2009). Publication bias in clinical trials due to statistical significance or direction of trial results. Cochrane Database of Systematic Reviews, 1, Art. No.: MR000006. doi: 10.1002/14651858.MR000006.pub3 . Search in Google Scholar

Hsiao, H. S., Chen, J. C., Chen, J. H., Zeng, Y. T., & Chung, G. H. (2022). An assessment of junior high school students’ knowledge, creativity, and hands-on performance using PBL via cognitive–affective interaction model to achieve STEAM. Sustainability, 14(9), 5582. doi: 10.3390/su14095582 . Search in Google Scholar

Hughes, B. S., Corrigan, M. W., Grove, D., Andersen, S. B., & Wong, J. T. (2022). Integrating arts with STEM and leading with STEAM to increase science learning with equity for emerging bilingual learners in the United States. International Journal of STEM Education, 9(1), 1–19. doi: 10.1186/s40594-022-00375-7 . Search in Google Scholar

Hwang, W. Y., Hariyanti, U., Chen, N. S., & Purba, S. W. D. (2023). Developing and validating an authentic contextual learning framework: Promoting healthy learning through learning by applying. Interactive Learning Environments, 31(4), 2206–2218. doi: 10.1080/10494820.2021.1876737 . Search in Google Scholar

Hypolite, L. I., & Rogers Jr, K. D. (2023). Closing STEM opportunity gaps through critical approaches to teaching and learning for Black youth. Theory Into Practice, 62, 431–447. doi: 10.1080/00405841.2023.2278972 . Search in Google Scholar

Jesionkowska, J., Wild, F., & Deval, Y. (2020). Active learning augmented reality for STEAM education – A case study. Education Sciences, 10(8), 198. doi: 10.3390/educsci10080198 . Search in Google Scholar

Jia, Y., Zhou, B., & Zheng, X. (2021). A curriculum integrating STEAM and maker education promotes pupils’ learning motivation, self-efficacy, and interdisciplinary knowledge acquisition. Frontiers in Psychology, 12, 725525. doi: 10.3389/fpsyg.2021.725525 . Search in Google Scholar

Jiang, S., Shen, J., Smith, B. E., & Kibler, K. W. (2020). Science identity development: How multimodal composition mediates student role-taking as scientist in a media-rich learning environment. Educational Technology Research and Development, 68, 3187–3212. doi: 10.1007/s11423-020-09816-y . Search in Google Scholar

Khamhaengpol, A., Sriprom, M., & Chuamchaitrakool, P. (2021). Development of STEAM activity on nanotechnology to determine basic science process skills and engineering design process for high school students. Thinking Skills and Creativity, 39, 100796. doi: 10.1016/j.tsc.2021.100796 . Search in Google Scholar

Kozlov, A. V., & Shemshurina, S. A. (2018). Fostering creativity in engineering universities: Research activity and curriculum policy. International Journal of Instruction, 11(4), 93–106. doi: 10.12973/iji.2018.1147a . Search in Google Scholar

Kugley, S., Wade, A., Thomas, J., Mahood, Q., Klint Jørgensen, A. M., Hammerstrøm, K., & Sathe, N. (2017). Searching for studies: A guide to information retrieval for Campbell systematic reviews. Campbell Method Guides, 13(1), 1–73. doi: 10.4073/cmg.2016.1 . Search in Google Scholar

LaJevic, L. (2013). Arts integration: What is really happening in the elementary classroom?. Journal for Learning through the Arts, 9(1), n1. doi: 10.21977/D99112615 . Search in Google Scholar

Lasserson, T. J., Thomas, J., & Higgins, J. P. T. (2019). Starting a review. In H. JPT, J. Thomas, J. Chandler, M. Cumpston, T. Li, M. J. Page, & V. A. Welch (Eds.), Cochrane handbook for systematic reviews of interventions (2nd ed., pp. 3–10). Chichester, UK: John Wiley and Sons. Search in Google Scholar

Leavy, A., Dick, L., Meletiou‐Mavrotheris, M., Paparistodemou, E., & Stylianou, E. (2023). The prevalence and use of emerging technologies in STEAM education: A systematic review of the literature. Journal of Computer Assisted Learning, 39(4), 1–22. doi: 10.1111/jcal.12806 . Search in Google Scholar

Li, B., Sjöström, J., Ding, B., & Eilks, I. (2022). Education for sustainability meets Confucianism in science education. Science and Education, 32(4), 1–30. doi: 10.1007/s11191-022-00349-9 . Search in Google Scholar

Li, K. C., & Wong, B. T. M. (2023). Personalisation in STE (A) M education: A review of literature from 2011 to 2020. Journal of Computing in Higher Education, 35(1), 186–201. doi: 10.1007/s12528-022-09341-2 . Search in Google Scholar

Liao, C. (2016). From interdisciplinary to transdisciplinary: An arts-integrated approach to STEAM education. Art Education, 69(6), 44–49. doi: 10.1080/00043125.2016.1224873 . Search in Google Scholar

López-Banet, L., Perales, F. J., & Jimenez-Liso, M. R. (2021). STEAM views from a need: The case of the chewing gum and pH sensopill (Miradas STEAM desde la necesidad: El caso de la sensopíldora chicles y pH). Journal for the Study of Education and Development, 44(4), 909–941. doi: 10.1080/02103702.2021.1927505 . Search in Google Scholar

Marín-Marín, J. A., Moreno-Guerrero, A. J., Dúo-Terrón, P., & López-Belmonte, J. (2021). STEAM in education: A bibliometric analysis of performance and co-words in Web of Science. International Journal of STEM Education, 8(1), 41. doi: 10.1186/s40594-021-00296-x . Search in Google Scholar

Martins, I. P., & Veiga, L. (2001). Early science education: Exploring familiar contexts to improve the understanding of some basic scientific concepts. European Early Childhood Education Research Journal, 9(2), 69–82. doi: 10.1080/13502930185208771 . Search in Google Scholar

Mehta, R., Keenan, S., Henriksen, D., & Mishra, P. (2019). Developing a rhetoric of aesthetics: The (Often) forgotten link between art and STEM. In M. S. Khine & S. Areepattamannil (Eds.), STEAM education: Theory and practice (pp. 117–141). Switzerland: Springer International Publishing. doi: 10.1007/978-3-030-04003-1_7 . Search in Google Scholar

Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & PRISMA Group. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. Annals of Internal Medicine, 151(4), 264–269. doi: 10.7326/0003-4819-151-4-200908180-00135 . Search in Google Scholar

National Research Council. (2013). Next generation science standards: For states, by states. Washington DC, USA: The National Academies Press. Search in Google Scholar

Ozkan, G., & Topsakal, U. (2020). Investigating the effectiveness of STEAM education on students’ conceptual understanding of force and energy topics. Research in Science and Technological Education, 39(4), 441–460. 10.1080/02635143.2020.1769586 Search in Google Scholar

Ozkan, G., & Topsakal, U. (2021). Exploring the effectiveness of STEAM design processes on middle school students’ creativity. International Journal of Technology and Design Education, 31, 95–116. doi: 10.1007/s10798-019-09547-z . Search in Google Scholar

Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., … Moher, D. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ, 372, n71. doi: 10.1136/bmj.n71 . Search in Google Scholar

Park, Y. S., & Park, J. H. (2020). Exploring the explicit teaching strategies in STEAM program of climate change. Asia-Pacific Science Education, 6(1), 116–151. doi: 10.1163/23641177-bja00002 . Search in Google Scholar

Park, Y. S. (2008). Analyzing science teachers’ understandings about scientific argumentation in terms of scientific inquiry. Journal of the Korean Association for Science Education, 28(3), 211–226. Search in Google Scholar

Perignat, E., & Katz-Buonincontro, J. (2019). STEAM in practice and research: An integrative literature review. Thinking Skills and Creativity, 31, 31–43. doi: 10.1016/j.tsc.2018.10.002 . Search in Google Scholar

Quigley, C. F., & Herro, D. (2016). “Finding the joy in the unknown”: Implementation of STEAM teaching practices in middle school science and math classrooms. Journal of Science Education and Technology, 25, 410–426. doi: 10.1007/s10956-016-9602-z . Search in Google Scholar

Quigley, C. F., Herro, D., Shekell, C., Cian, H., & Jacques, L. (2020a). Connected learning in STEAM classrooms: Opportunities for engaging youth in science and math classrooms. International Journal of Science and Mathematics Education, 18, 1441–1463. doi: 10.1007/s10763-019-10034-z . Search in Google Scholar

Quigley, C. F., Herro, D., King, E., & Plank, H. (2020b). STEAM designed and enacted: Understanding the process of design and implementation of STEAM curriculum in an elementary school. Journal of Science Education and Technology, 29, 499–518. doi: 10.1007/s10956-020-09832-w . Search in Google Scholar

Rahmawati, Y., Taylor, E., Taylor, P. C., Ridwan, A., & Mardiah, A. (2022). Students’ engagement in education as sustainability: Implementing an ethical dilemma-STEAM teaching model in chemistry learning. Sustainability, 14(6), 3554. doi: 10.3390/su14063554 . Search in Google Scholar

Rice, D. B., Kloda, L. A., Levis, B., Qi, B., Kingsland, E., & Thombs, B. D. (20016). Are MEDLINE searches sufficient for systematic reviews and meta-analyses of the diagnostic accuracy of depression screening tools? A review of meta-analyses. Journal of Psychosomatic Research, 87, 7–13. 10.1016/j.jpsychores.2016.06.002 Search in Google Scholar

Sachs, J. D., Schmidt-Traub, G., Mazzucato, M., Messner, D., Nakicenovic, N., & Rockström, J. (2019). Six transformations to achieve the sustainable development goals. Nature sustainability, 2(9), 805–814. doi: 10.1038/s41893-019-0352-9 . Search in Google Scholar

Savery, J. R. (2015). Overview of problem-based learning: Definitions and distinctions. In A. Walker, H. Leary, C. Hmelo-Silver, & P. A. Ertmer (Eds.), Essential readings in problem-based learning: Exploring and extending the legacy of Howard S. Barrows (pp. 5–15). Indiana, USA: Perdue University Press. doi: 10.7771/1541-5015.1002 . Search in Google Scholar

Scardamalia, M., Bransford, J., Kozma, B., & Quellmalz, E. (2012). New assessments and environments for knowledge building. In P. Griffin, B. McGaw, & E. Care (Eds.), Assessment and teaching of 21st century skills (pp. 231–300). Switzerland: Springer. doi: 10.1007/978-94-007-2324-5_5 . Search in Google Scholar

Singh, V. (2020). Teaching climate change in a physics classroom: Towards a transdisciplinary approach. ArXiv:2008.00281 [Physics]. http://arxiv.org/abs/2008.00281. Search in Google Scholar

Smith, J., & Hu, R. (2013). Rethinking teacher education: Synchronizing eastern and western views of teaching and learning to promote 21st century skills and global perspectives. Education Research and Perspectives, 40(1), 86–108. Search in Google Scholar

Snyder, H. (2019). Literature review as a research methodology: An overview and guidelines. Journal of Business Research, 104, 333–339. doi: 10.1016/j.jbusres.2019.07.039 . Search in Google Scholar

Son, M. H., & Jung, D. H. (2019). Limits of STEAM education and its Improvement alternative: Based on the viewpoints of STEAM expert teachers. Journal of the Korean Association for Research in Science Education, 39(5), 573–584. doi: 10.14697/jkase.2019.39.5.573 . Search in Google Scholar

Stork, M. G. (2020). Supporting twenty-first century competencies using robots and digital storytelling. Journal of Formative Design in Learning, 4(1), 43–50. doi: 10.1007/s41686-019-00039-w . Search in Google Scholar

Teo, P. (2019). Teaching for the 21st century: A case for dialogic pedagogy. Learning, Culture and Social Interaction, 21, 170–178. doi: 10.1016/j.lcsi.2019.03.009 . Search in Google Scholar

Tran, N. H., Huang, C. F., & Hung, J. F. (2021a). Exploring the effectiveness of STEAM-based courses on junior high school students’ scientific creativity. In Frontiers in education (p. 472). Switzerland: Frontiers. doi: 10.3389/feduc.2021.666792 . Search in Google Scholar

Tran, N. H., Huang, C. F., Hsiao, K. H., Lin, K. L., & Hung, J. F. (2021b). Investigation on the influences of STEAM-based curriculum on scientific creativity of elementary school students. In Frontiers in education (Vol. 6, p. 694516). Switzerland: Frontiers Media SA. doi: 10.3389/feduc.2021.694516 . Search in Google Scholar

Trilling, B., & Fadel, C. (2009). 21st century skills: Learning for life in our times. New Jersey, USA: John Wiley & Sons. Search in Google Scholar

Vasquez, J., Sneider, C., & Comer, M. (2013). STEM lesson essentials-integrating science, technology, engineering, and mathematics, grades 3–8. Portsmouth, NH: Heinemann. Search in Google Scholar

Wilson, H. E., Song, H., Johnson, BJ., Presley, L., & Olson, K. (2021). Effects of transdisciplinary STEAM lessons on student critical and creative thinking. The Journal of Educational Research, 114(5), 445–457 10.1080/00220671.2021.1975090 Search in Google Scholar

Won, A. R., Choi, S. Y., Chu, H. E., Cha, H. J., Shin, H., & Kim, C. J. (2021). A teacher’s practical knowledge in an SSI-STEAM program dealing with climate change. Asia-Pacific Science Education, 7(1), 134–172. doi: 10.1163/23641177-bja10023 . Search in Google Scholar

Zhang, Y., Ng, O. L., & Leung, S. (2023). Researching computational thinking in early childhood STE (A) M education context: A descriptive review on the state of research and future directions. Journal for STEM Education Research, 6(3),1–29. doi: 10.1007/s41979-023-00097-7 . Search in Google Scholar

© 2024 the author(s), published by De Gruyter

This work is licensed under the Creative Commons Attribution 4.0 International License.

• X / Twitter

Supplementary Materials

Please login or register with De Gruyter to order this product.

Journal and Issue

Articles in the same issue.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• My Bibliography
• Collections
• Citation manager

Save citation to file

Email citation, add to collections.

• Create a new collection
• Add to an existing collection

Add to My Bibliography

Your saved search, create a file for external citation management software, your rss feed.

• Search in PubMed
• Search in NLM Catalog
• Add to Search

Impact of Robotic Surgery Objective Performance Indicators: A Systematic Review

Affiliations.

• 1 Department of Surgery, Section of Minimally Invasive Surgery, Washington University School of Medicine, St. Louis, MO.
• 2 Data and Analytics, Intuitive Surgical, Inc., Peachtree Corners, GA.
• PMID: 39268949
• DOI: 10.1097/XCS.0000000000001208

Objective intra-operative performance metrics are an new resource for the field of surgical data science. In this study, we conducted a comprehensive review to analyze the existing literature on robotic surgical objective performance indicators (OPIs). OPIs have been used to assess surgical skill and provide automated, objective feedback in multiple settings. Clinically, OPIs have been utilized to predict post-operative patient outcomes and measure intra-operative efficiency. OPIs have shown promise in preliminary surgical education and patient outcomes related studies, and further multi-center collaborative research will be imperative to validate the use of OPIs in the field of surgery.

Copyright © 2024 by the American College of Surgeons. Published by Wolters Kluwer Health, Inc. All rights reserved.

PubMed Disclaimer

• Citation Manager

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.