literature review mobile application development

Systematic literature review of mobile application development and testing effort estimation

New citation alert added.

This alert has been successfully added and will be sent to:

You will be notified whenever a record that you have chosen has been cited.

To manage your alert preferences, click on the button below.

New Citation Alert!

Please log in to your account

Information & Contributors

Bibliometrics & citations, view options.

• Hadi M Fard F (2023) Evaluating pre-trained models for user feedback analysis in software engineering: a study on classification of app-reviews Empirical Software Engineering 10.1007/s10664-023-10314-x 28 :4 Online publication date: 23-May-2023 https://dl.acm.org/doi/10.1007/s10664-023-10314-x

Index Terms

General and reference

Cross-computing tools and techniques

Social and professional topics

Professional topics

Management of computing and information systems

Software management

Software and its engineering

Software creation and management

Software development process management

Software verification and validation

Software defect analysis

Software testing and debugging

Recommendations

Using metrics in agile and lean software development - a systematic literature review of industrial studies.

ContextSoftware industry has widely adopted Agile software development methods. Agile literature proposes a few key metrics but little is known of the actual metrics use in Agile teams. ObjectiveThe objective of this paper is to increase knowledge of ...

Challenges in Combining Agile Development and CMMI: A Systematic Literature Review

Recently, Agile Development has emerged as an alternative approach in software engineering. The Agile Software Development (ASD) process provides the ability to cope with ever-changing requirements. On the other hand, the Capability Maturity Model ...

A COSMIC function points based test effort estimation model for mobile applications

With the substantial proliferation in demand for applications running on mobile devices, developers and testers are foreseen to release applications with high caliber, on time and inside budget. Mobile application testing is imperative ...

Information

Published in.

Elsevier Science Inc.

United States

Publication History

Author tags.

• Mobile Applications
• Test effort
• Systematic literature review
• Review-article

Contributors

Other metrics, bibliometrics, article metrics.

• 1 Total Citations View Citations
• 0 Total Downloads
• Downloads (Last 12 months) 0
• Downloads (Last 6 weeks) 0

View options

Login options.

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Share this publication link.

Copying failed.

Share on social media

Affiliations, export citations.

• Please download or close your previous search result export first before starting a new bulk export. Preview is not available. By clicking download, a status dialog will open to start the export process. The process may take a few minutes but once it finishes a file will be downloadable from your browser. You may continue to browse the DL while the export process is in progress. Download
• Download citation
• Copy citation

We are preparing your search results for download ...

We will inform you here when the file is ready.

Your file of search results citations is now ready.

Your search export query has expired. Please try again.

• Open access
• Published: 07 May 2013

Usability of mobile applications: literature review and rationale for a new usability model

• Rachel Harrison 1 ,
• Derek Flood 1 &
• David Duce 1  

Journal of Interaction Science volume  1 , Article number:  1 ( 2013 ) Cite this article

191k Accesses

347 Citations

3 Altmetric

Metrics details

The usefulness of mobile devices has increased greatly in recent years allowing users to perform more tasks in a mobile context. This increase in usefulness has come at the expense of the usability of these devices in some contexts. We conducted a small review of mobile usability models and found that usability is usually measured in terms of three attributes; effectiveness, efficiency and satisfaction. Other attributes, such as cognitive load, tend to be overlooked in the usability models that are most prominent despite their likely impact on the success or failure of an application. To remedy this we introduces the PACMAD (People At the Centre of Mobile Application Development) usability model which was designed to address the limitations of existing usability models when applied to mobile devices. PACMAD brings together significant attributes from different usability models in order to create a more comprehensive model. None of the attributes that it includes are new, but the existing prominent usability models ignore one or more of them. This could lead to an incomplete usability evaluation. We performed a literature search to compile a collection of studies that evaluate mobile applications and then evaluated the studies using our model.

Introduction

Advances in mobile technology have enabled a wide range of applications to be developed that can be used by people on the move. Developers sometimes overlook the fact that users will want to interact with such devices while on the move. Small screen sizes, limited connectivity, high power consumption rates and limited input modalities are just some of the issues that arise when designing for small, portable devices. One of the biggest issues is the context in which they are used. As these devices are designed to enable users to use them while mobile, the impact that the use of these devices has on the mobility of the user is a critical factor to the success or failure of the application.

Current research has demonstrated that cognitive overload can be an important aspect of usability [ 1 , 2 ]. It seems likely that mobile devices may be particularly sensitive to the effects of cognitive overload, due to their likely deployment in multiple task settings and limitations of size. This aspect of usability is often overlooked in existing usability models, which are outlined in the next section, as these models are designed for applications which are seldom used in a mobile context. Our PACMAD usability model for mobile applications, which we then introduce, incorporates cognitive load as this attribute directly impacts and may be impacted by the usability of an application.

A literature review, outlined in the following section, was conducted as validation of the PACMAD model. This literature review examined which attributes of usability, as defined in the PACMAD usability model, were used during the evaluation of mobile applications presented in a range of papers published between 2008 and 2010. Previous work by Kjeldskov & Graham [ 3 ] has looked at the research methods used in mobile HCI, but did not examine the particular attributes of usability incorporated in the PACMAD model. We also present the results of the literature review.

The impact of this work on future usability studies and what lessons other researchers should consider when performing usability evaluations on mobile applications are also discussed.

Background and literature review

Existing models of usability.

Nielsen [ 4 ] identified five attributes of usability:

  Efficiency : Resources expended in relation to the accuracy and completeness with which users achieve goals;

  Satisfaction : Freedom from discomfort, and positive attitudes towards the use of the product.

  Learnability : The system should be easy to learn so that the user can rapidly start getting work done with the system;

  Memorability : The system should be easy to remember so that the casual user is able to return to the system after some period of not having used it without having to learn everything all over again;

  Errors : The system should have a low error rate, so that users make few errors during the use of the system and that if they do make errors they can easily recover from them. Further, catastrophic errors must not occur.

In addition to this Nielsen defines Utility as the ability of a system to meet the needs of the user. He does not consider this to be part of usability but a separate attribute of a system. If a product fails to provide utility then it does not offer the features and functions required; the usability of the product becomes superfluous as it will not allow the user to achieve their goals. Likewise, the International Organization for Standardization (ISO) defined usability as the “Extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use” [ 5 ]. This definition identifies 3 factors that should be considered when evaluating usability.

  User : Person who interacts with the product;

  Goal : Intended outcome;

  Context of use : Users, tasks, equipment (hardware, software and materials), and the physical and social environments in which a product is used.

Each of the above factors may have an impact on the overall design of the product and in particular will affect how the user will interact with the system. In order to measure how usable a system is, the ISO standard outlines three measurable attributes:

  Effectiveness : Accuracy and completeness with which users achieve specified goals;

Unlike Nielsen’s model of usability, the ISO standard does not consider Learnability, Memorability and Errors to be attributes of a product’s usability although it could be argued that they are included implicitly within the definitions of Effectiveness, Efficiency and Satisfaction. For example, error rates can be argued to have a direct effect on efficiency.

Limitations for mobile applications

The models presented above were largely derived from traditional desktop applications. For example, Nielsen’s work was largely based on the design of telecoms systems, rather than computer software. The advent of mobile devices has presented new usability challenges that are difficult to model using traditional models of usability. Zhang and Adipat [ 6 ] highlighted a number of issues that have been introduced by the advent of mobile devices:

  Mobile Context : When using mobile applications the user is not tied to a single location. They may also be interacting with nearby people, objects and environmental elements which may distract their attention.

  Connectivity : Connectivity is often slow and unreliable on mobile devices. This will impact the performance of mobile applications that utilize these features.

  Small Screen Size : In order to provide portability mobile devices contain very limited screen size and so the amount of information that can be displayed is limited.

  Different Display Resolution : The resolution of mobile devices is reduced from that of desktop computers resulting in lower quality images.

  Limited Processing Capability and Power : In order to provide portability, mobile devices often contain less processing capability and power. This will limit the type of applications that are suitable for mobile devices.

  Data Entry Methods : The input methods available for mobile devices are different from those for desktop computers and require a certain level of proficiency. This problem increases the likelihood of erroneous input and decreases the rate of data entry.

From our review it is apparent that many existing models for usability do not consider mobility and its consequences, such as additional cognitive load. This complicates the job of the usability practitioner, who must consequently define their task model to explicitly include mobility. One might argue that the lack of reference to a particular context could be a strength of a usability model provided that the usability practitioner has the initiative and knows how to modify the model for a particular context.

The PACMAD usability model aims to address some of the shortcomings of existing usability models when applied to mobile applications. This model builds on existing theories of usability but is tailored specifically for applications that can be used on mobile devices. The PACMAD usability model is depicted in Figure  1 side by side with Nielsen’s and the ISO’s definition of usability. The PACMAD usability model incorporates the attributes of both the ISO standard and Nielsen’s model and also introduces the attribute of cognitive load which is of particular importance to mobile applications. The following section introduces the PACMAD usability model and describes in detail each of the attributes of usability mentioned below as well as the three usability factors that are part of this model: user, task and context.

Comparison of usability models.

The PACMAD usability model for mobile applications identifies three factors (User, Task and Context of use) that should be considered when designing mobile applications that are usable. Each of these factors will impact the final design of the interface for the mobile application. In addition to this the model also identifies seven attributes that can be used to define metrics to measure the usability of an application. The following section outlines each of these factors and attributes in more detail.

Factors of usability

The PACMAD usability model identifies three factors which can affect the overall usability of a mobile application: User , Task and Context of use . Existing usability models such as those proposed by the ISO [ 5 ] and Nielsen [ 4 ] also recognise these factors as being critical to the successful usability of an application. For mobile applications Context of use plays a critical role as an application may be used in multiple, very different contexts.

User It is important to consider the end user of an application during the development process. As mobile applications are usually designed to be small, the traditional input methods, such as a keyboard and mouse, are no longer practical. It is therefore necessary for application designers to look at alternative input methods. Some users may find it difficult to use some of these methods due to physical limitations. For example it has been shown [ 7 ] that some Tetraplegic users who have limited mobility in their upper extremities tend to have high error rates when using touch screens and this may cause unacceptable difficulties with certain (usually small) size targets.

Another factor that should be considered is the user’s previous experience. If a user is an expert at the chosen task then they are likely to favour shortcut keys to accomplish this task. On the other hand novice users may prefer an interface that is intuitive and easy to navigate and which allows them to discover what they need. This trade-off must be considered during the design of the application.

Task The word task refers here to the goal the user is trying to accomplish with the mobile application. During the development of applications, additional features can be added to an application in order to allow the user to accomplish more with the software. This extra functionality comes at the expense of usability as these additional features increase the complexity of the software and therefore the user’s original goal can become difficult to accomplish.

For example, consider a digital camera. If a user wants to take a photograph, they must first select between different modes (e.g. video, stills, action, playback, etc.) and then begin to line up the shot. This problem is further compounded if the user needs to take a photograph at night and needs to search through a number of menu items to locate and turn on a flashlight.

Context of use The word context refers here to the environment in which the user will use the application. We want to be able to view context separately from both the user and the task. Context not only refers to a physical location but also includes other features such as the user’s interaction with other people or objects (e.g. a motor vehicle) and other tasks the user may be trying to accomplish. Research has shown that using mobile applications while walking can slow down the walker’s average walking speed [ 8 ]. As mobile applications can be used while performing other tasks it is important to consider the impact of using the mobile application in the appropriate context.

Attributes of usability

The PACMAD usability model identifies 7 attributes which reflect the usability of an application: Effectiveness , Efficiency , Satisfaction , Learnability , Memorability , Errors and Cognitive load . Each of these attributes has an impact on the overall usability of the application and as such can be used to help assess the usability of the application.

Effectiveness Effectiveness is the ability of a user to complete a task in a specified context. Typically effectiveness is measured by evaluating whether or not participants can complete a set of specified tasks.

Efficiency Efficiency is the ability of the user to complete their task with speed and accuracy. This attribute reflects the productivity of a user while using the application. Efficiency can be measured in a number of ways, such as the time to complete a given task, or the number of keystrokes required to complete a given task.

Satisfaction Satisfaction is the perceived level of comfort and pleasantness afforded to the user through the use of the software. This is reflected in the attitudes of the user towards the software. This is usually measured subjectively and varies between individual users. Questionnaires and other qualitative techniques are typically used to measure a user’s attitudes towards a software application.

Learnability A recent survey of mobile application users [ 9 ] found that users will spend on average 5 minutes or less learning to use a mobile application. There are a large number of applications available on mobile platforms and so if users are unable to use an application they may simply select a different one. For this reason the PACMAD model includes the attribute Learnability as suggested by Nielsen.

Learnability is the ease with which a user can gain proficiency with an application. It typically reflects how long it takes a person to be able to use the application effectively. In order to measure Learnability, researchers may look at the performance of participants during a series of tasks, and measure how long it takes these participants to reach a pre-specified level of proficiency.

Memorability The survey also found that mobile applications are used on an infrequent basis and that participants used almost 50% of the applications only once a month [ 9 ]. Thus there may be a large period of inactivity between uses and so participants may not easily recall how to use the application. Consequently the PACMAD usability model includes the attribute of Memorability as also suggested by Nielsen.

Memorability is the ability of a user to retain how to use an application effectively. Software might not be used on a regular basis and sometimes may only be used sporadically. It is therefore necessary for users to remember how to use the software without the need to relearn it after a period of inactivity. Memorability can be measured by asking participants to perform a series of tasks after having become proficient with the use of the software and then asking them to perform similar tasks after a period of inactivity. A comparison can then be made between the two sets of results to determine how memorable the application was.

Errors The PACMAD usability model extends the description of Errors, first proposed by Nielsen, to include an evaluation of the errors that are made by participants while using mobile apps. This allows developers to identify the most troublesome areas for users and to improve these areas in subsequent iterations of development. This attribute is used to reflect how well the user can complete the desired tasks without errors. Nielsen [ 4 ] states that users should make few errors during the use of a system and that if they do make errors they should be able to easily recover from them. The error rate of users may be used to infer the simplicity of a system. The PACMAD usability model considers the nature of errors as well as the frequency with which they occur. By understanding the nature of these errors it is possible to prevent these errors from occurring in future versions of the application.

Cognitive load The main contribution of the PACMAD model is its inclusion of Cognitive Load as an attribute of usability. Unlike traditional desktop applications, users of mobile applications may be performing additional tasks, such as walking, while using the mobile device. For this reason it is important to consider the impact that using the mobile device will have on the performance of the user of these additional tasks. For example a user may wish to send a text message while walking. In this case the user’s walking speed will be reduced as they are concentrating on sending the message which is distracting them from walking.

Cognitive load refers to the amount of cognitive processing required by the user to use the application. In traditional usability studies a common assumption is that the user is performing only a single task and can therefore concentrate completely on that task. In a mobile context users will often be performing a second action in addition to using the mobile application [ 8 , 10 ]. For example a user may be using a stereo while simultaneously driving a car. In this scenario it is important that the cognitive load required by the mobile application, in this case the stereo, does not adversely impact the primary task.

While the user is using the application in a mobile context it will impact both the user’s ability to move and to operate the mobile application. Therefore it is important to consider both dimensions when studying the usability of mobile applications. One way this can be measured is through the NASA Task Load Index (TLX) [ 11 ]. This is a subjective workload assessment tool for measuring the cognitive workload placed on a user by the use of a system. In this paper we adopt a relatively simple view of cognitive load. For a more accurate assessment it may be preferable to adopt a more powerful multi-factorial approach [ 1 , 12 ] but this is beyond the scope of this paper.

Literature review

In order to evaluate the appropriateness and timeliness of the PACMAD usability model for mobile applications, a literature review was conducted to review current approaches and to determine the need for a comprehensive model that includes cognitive load. We focused on papers published between 2008 and 2010 which included an evaluation of the usability of a mobile application.

Performing the literature review

The first step in the literature review was to collect all of the publications from the identified sources. These sources were identified by searching the ACM digital library, IEEE digital library and Google Scholar. The search strings used during these searches were “ Mobile Application Evaluations ”, “ Usability of mobile applications ” and “ Mobile application usability evaluations ”. The following conferences and journals were identified as being the most relevant sources: the Mobile HCI conference (MobileHCI), the International Journal of Mobile Human Computer Interaction (IJMHCI), the ACM Transactions on Computer-Human Interaction (TOCHI), the International Journal of Human Computer Studies (IJHCS), the Personal and Ubiquitous Computing journal (PUC), and the International Journal of Human-Computer Interaction (IJHCI). We also considered the ACM Conference on Human Factors in Computing Systems (CHI) and the IEEE Transactions on Mobile Computing (IEEE TOMC). These sources were later discarded as very few papers (less than 5% of the total) were relevant.

The literature review was limited to the publications between the years 2008 and 2010 due to the emergence of smart phones during this time. Table  1 shows the number of publications that were examined from each source.

The sources presented above included a number of different types of publications (Full papers, short papers, doctoral consortium, editorials, etc.). We focused the study only on full or short research papers from peer reviewed sources. This approach was also adopted by Budgen et al. [ 13 ]. Table  2 shows the number of remaining publications by source.

The abstract of each of the remaining papers was examined to determine if the paper:

Conducted an evaluation of a mobile application/device;

Contained some software component with which the users interact;

Conducted an evaluation which was focused on the interaction with the application or device;

Publications which did not meet the above criteria were removed.

The following exclusion criteria were u sed to exclude papers:

Focused only on application development methodologies and techniques;

Contained only physical interaction without a software component;

Examined only social aspects of using mobile applications;

Did not consider mobile applications.

Each abstract was reviewed by the first two authors to determine if it should be included within the literature review. When a disagreement arose between the reviewers it was discussed until mutual agreement was reached. A small number of relevant publications were unavailable to the authors. Table  3 shows the number of papers included within the literature review by source.

Each of the remaining papers was examined by one reviewer (either the first or second author of this paper). The reviewer examined each paper in detail and identified for each one:

 The attribute of usability that could be measured through the collected metrics;

 The focus of the research presented.

 The type of study conducted;

To ensure the quality of the data extraction performed the first and second author independently reviewed a 10% sample and compared these results. When a disagreement arose it was discussed until an agreement was reached.

Twenty papers that were identified as being relevant did not contain any formal evaluations of the proposed technologies. The results presented below exclude these 20 papers. In addition to this some papers presented multiple studies. In these cases each study was considered independently and so the results based on the number of studies within the evaluated papers rather than the number of papers.

Limitations

This literature review is limited for a number of reasons. Firstly a small number of papers were unavailable to the researchers (8 out of 139 papers considered relevant). This unavailability of less than 6% of the papers probably does not have a large impact on the results presented. By omitting certain sources from the study a bias may have been introduced. We felt that the range of sources considered was a fair representation of the field of usability of mobile applications although some outlying studies may have been omitted due to limited resources. Our reviews of these sources led us to believe that the omitted papers were of borderline significance. Ethical approval for this research was given by Oxford Brookes University Research Ethics Committee.

Research questions

To evaluate the PACMAD usability model three Research Questions (RQ1 to RQ3) were established to determine how important each of the factors and attributes of usability are in the context of mobile applications.

RQ1: What attributes are used when considering the usability of mobile applications?

This research question was established to discover what attributes are typically used to analyse mobile applications and which metrics are associated with them. The answers to this question provide evidence and data for the PACMAD usability model.

RQ2: To what extent are the factors of usability considered in existing research?

In order to determine how research in mobile applications is evolving, RQ2 was established to examine the current research trends into mobile applications, with a particular focus on the factors that affect usability.

In addition to this we wanted to establish which research methods are most commonly used when evaluating mobile applications. For this reason, a third research question was established.

RQ3: What research methodologies are used to evaluate the usability of mobile applications?

There are many ways in which mobile applications can be evaluated including controlled studies, field studies, ethnography, experiments, case-studies, surveys, etc. This research question aims to identify the most common research methodologies used to evaluate mobile apps. The answers to this question will throw light on the maturity of the mobile app engineering field.

The above research questions were answered by examining the literature on mobile applications. The range of literature on the topic of mobile applications is so broad it was important to limit the literature review to the most relevant and recent publications and to limit the publication interval to papers published between 2008 and 2010.

Table  4 shows the percentage of studies that include metrics, such as time to complete a given task, which either directly or indirectly assesses the attributes of usability included within the PACMAD usability model. In some cases the studies evaluated multiple attributes of usability and therefore the results above present both the percentage and the number of studies in which each attribute was considered. These studies often do not explicitly cite usability or any usability related criteria, and so the metrics used for the papers’ analyses were used to discover the usability attributes considered. This lack of precision is probably due to a lack of agreement as to what constitutes usability and the fact that the attributes are not orthogonal. The three most common attributes, Effectiveness, Efficiency and Satisfaction, correspond to the attributes identified by the ISO’s standard for usability.

One of the reasons these attributes are so widely considered is their direct relationship to the technical capabilities of the system. Both Effectiveness and Efficiency are related to the design and implementation of the system and so are usually tested thoroughly. These attributes are also relatively easy to measure. In most cases the Effectiveness of the system is evaluated by monitoring whether a user can accomplish a pre-specified task. Efficiency can be measured by finding the time taken by the participant to complete this task. Questionnaires and structured interviews can be used to determine the Satisfaction of users towards the system. Approximately 22% of the papers reviewed evaluated all three of these attributes.

The focus on these attributes of usability implies that Learnability, Memorability, Errors, and Cognitive load, are considered to be of less importance than Effectiveness, Efficiency and Satisfaction. Learnability, Memorability, Errors, and Cognitive load are not easy to evaluate and this may be why their assessment is often overlooked. As technology matures designers have begun to consider usability earlier in the design process. This is reflected to a certain extent by technological changes away from command line towards GUI based interfaces.

The aspects of usability that were considered least often in the papers reviewed are Learnability and Memorability. There are numerous reasons for this. The nature of these attributes demands that they are evaluated over periods of time. To effectively measure Learnability, users’ progress needs to be checked at regular intervals or tracked over many completions of a task. In the papers reviewed, Learnability was usually measured indirectly by the changes in effectiveness or efficiency over many completions of a specified task.

Memorability was only measured subjectively in the papers reviewed. One way to objectively measure Memorability is to examine participants’ use of the system after a period of inactivity with the system. The practical problem of recruiting participants who are willing to return multiple times to participate in an evaluation is probably one of the reasons why this attribute is not often measured objectively.

What differentiates mobile applications from more traditional applications is the ability of the user to use the application while moving. In this context, the users’ attention is divided between the act of moving and using the application. About 26% of the studies considered cognitive load. Some of these studies used the change in performance of the user performing the primary task (which was usually walking or driving) as an indication of the cognitive load. Other studies used the NASA TLX [ 11 ] to subjectively measure cognitive load.

Table  5 shows the current research trends within mobile application research. It can be seen that the majority of work is focused on a task approximately 47% of the papers reviewed focus on allowing users to complete a specific task. The range of tasks considered is too broad to provide a detailed description and so we present here only some of the most dominant trends seen within the literature review.

The integration of cameras into mobile devices has enabled the emergence of a new class of application for mobile devices known as augmented reality. For example Bruns and Bimber [ 14 ] have developed an augmented reality application which allows users to take a photograph of an exhibit at an art gallery which allows the system to find additional information about the work of art. Similar systems have also been developed for Points of Interest (POIs) for tourists [ 15 ].

While using maps is a traditional way of navigating to a destination, mobile devices incorporating GPS (Global Positioning Satellite) technology have enabled researchers to investigate new ways of helping users to navigate. A number of systems [ 16 , 17 ] have proposed the use of tactile feedback to help guide users. Through the use of different vibration techniques the system informs users whether they should turn left, right or keep going straight. Another alternative to this is the use of sound. By altering the spatial balance and volume of a user’s music, Jones et al. [ 18 ] have developed a system for helping guide users to their destination.

One of the biggest limitations to mobile devices is the limited input modalities. Developers of apps do not have a large amount of space for physical buttons and therefore researchers are investigating other methods of interaction. This type of research accounts for approximately 29% of the studies reviewed.

The small screen size found on mobile applications has meant that only a small fraction of a document can be seen in detail. When mobile devices are used navigating between locations, this restriction can cause difficulty for users. In an effort to address this issue Burigat et al. [ 19 ] have developed a Zoomable User Interface with Overview (ZUIO). This interface allows a user to zoom into small sections of a document, such as a map, while displaying a small scale overview of the entire document so that the user can see where on the overall document they are. This type of system can also be used with large documents, such as web pages and images.

Audio interfaces [ 20 ] are a type of interface that is being investigated to assist drivers to use in-car systems. Traditional interfaces present information to users by visual means, but for drivers this distraction has safety critical implications. To address this issue audio inputs are common for in-vehicle systems. The low quality of voice recognition technology can limit its effectiveness within this context. Weinberg et al. [ 21 ] have shown that multiple push-to-talk buttons can improve the performance of users of such systems. Other types of interaction paradigms in these papers include touch screens [ 22 ], pressure based input [ 23 ], spatial awareness [ 24 ] and gestures [ 25 ]. As well as using these new input modalities a number of researchers are also looking at alternative output modes such as sound [ 26 ] and tactile feedback [ 27 ].

In addition to considering the specific tasks and input modalities, a small number of researchers are investigating ways to assist specific types of users, such as those suffering from physical or psychological disabilities, to complete common tasks. This type of research accounts for approximately 9% of the evaluated papers. Approximately 8% of the papers evaluated have focused on the context in which mobile applications are being used. The remaining 6% of studies are concerned with new development and evaluation methodologies for mobile applications. These include rapid prototyping tools for in-car systems, the effectiveness of expert evaluations and the use of heuristics for evaluating mobile haptic interfaces.

RQ3 was posed to investigate how usability evaluations are currently conducted. The literature review revealed that 7 of the papers evaluated did not contain any usability evaluations. Some of the remaining papers included multiple studies to evaluate different aspects of a technology or were conducted at different times during the development process. Table  6 shows the percentage of studies that were conducted using each research methodology.

By far the most dominant research methodology used in the examined studies was controlled experiments, accounting for approximately 59% of the studies. In a controlled experiment, all variables are held constant except the independent variable, which is manipulated by the experimenter. The dependant variable is the metric which is measured by the experimenter. In this way a cause and effect relationship may be investigated between the dependant and independent variables. Causality can be inferred from the covariation of the independent and dependent variables, temporal precedence of the cause as the manipulation of the independent variable and the elimination of confounding factors though control and internal validity tests.

Although the most common approach is the use of controlled experiments, other research methodologies were also used. A number of studies evaluated the use of new technologies through field studies. Field studies are conducted in a real world context, enabling evaluators to determine how users would use a technology outside of a controlled setting. These studies often revealed issues that would not be seen in a controlled setting.

For example a system designed by Kristoffersen and Bratteberg [ 28 ] to help travellers get to and from an airport by train without the use of paper tickets was deployed. This system used a credit card as a form of ticket for a journey to or from the airport. During the field study a number of usability issues were experienced by travellers. One user wanted to use a card to buy a ticket for himself and a companion; the system did not include this functionality as the developers of the system had assumed each user would have their own credit card and therefore designed the system to issue each ticket on a different credit card.

The evaluation also revealed issues relating to how the developers had implemented the different journey types, i.e. to and from the airport. When travelling to the airport users are required to swipe their credit card at the beginning and end of each journey, whereas when returning from the airport the user only needs to swipe their card when leaving the airport. One user found this out after he had swiped his card to terminate a journey from the airport, but was instead charged for a second ticket to the airport.

Although controlled experiments and field studies account for almost 90% of the studies, other strategies are also used. Surveys were used to better understand how the public reacted to mobile systems. Some of these studies were specific to a new technology or paradigm, [ 29 ] while others considered uses such as working while on the move [ 30 ]. In two cases (1% of the studies) archival research was used to investigate a particular phenomena relating to mobile technologies. A study conducted by Fehnert and Kosagowsky [ 31 ] used archival research to investigate the relationship between expert evaluations of user experience quality of mobile phones and subsequent usage figures. Lacroix et al. [ 32 ] used archival research to investigate the relationship between goal difficulty and performance within the context of an on-going activity intervention program.

In some cases it was found that no formal evaluation was conducted but instead the new technology presented in the paper was evaluated informally with colleagues of the developers. These evaluations typically contained a small number of participants and provide anecdotal evidence of a system’s usability.

The results obtained during the literature review reinforced the importance of cognitive load as an attribute of usability. It was found that almost 23% of the studies measured the cognitive load of the application under evaluation. These results show that current researchers in the area of mobile applications are beginning to recognise the importance of cognitive load in this domain and as such there is sufficient evidence for including it within the PACMAD model of usability.

The results also show that Memorability is not considered an important aspect of usability by many researchers. Only 2% of the studies evaluated Memorability. If an application is easy to learn then users may be willing to relearn how to use the application and therefore Memorability may indeed not be significant. On the other hand, some applications have a high learning curve and as such require a significant amount of time to learn. For these applications Memorability is an important attribute.

The trade-off between Learnability and Memorability is a consideration for application developers. Factors such as the task to be accomplished and the characteristics of the user should be considered when making this decision. The PACMAD model recommends that both factors should be considered although it also recognises that it may be adequate to evaluate only one of these factors depending on the application under evaluation. The literature review has also shown that the remaining attributes of usability are considered extensively by current research. Effectiveness, Efficiency and Satisfaction were included in over 50% of the studies. It was also found the Errors were evaluated in over 30% of these studies.

When considering the factors that can affect usability, it was found that the task is the most dominant factor being researched. Over 45% of the papers examined focused primarily on allowing a user to accomplish a task. When the interaction with an application is itself considered as a task this figure rises to approximately 75%. Context of use and the User were considered in less than 10% of the papers. Context of use can vary enormously and so should be considered an important factor of usability [ 5 , 33 ]. Our results indicate that context is not extensively researched and this suggests a gap in the literature.

It was revealing that some components of the PACMAD model occur only infrequently in the literature. As mentioned above Learnability and Memorability are rarely investigated, perhaps suggesting that researchers expected users to be able to learn to use apps without much difficulty., This finding could also be due to the difficulty of finding suitable subjects willing to undergo experiments on these attributes or the lack of standard research methods for these attributes. Effectiveness, Efficiency, Satisfaction and Errors were investigated more frequently, possibly because these attributes are widely recognised as important, and also possibly because research methods for investigating these attributes are well understood and documented. Almost a quarter of the studies investigated discussed Cognitive Load. It is surprising that this figure is not higher although this could again be due to the lack of a well-defined research methodology for investigating this attribute.

Conclusions

The range and availability of mobile applications is expanding rapidly. With the increased processing power available on portable devices, developers are increasing the range of services that they provide. The small size of mobile devices has limited the ways in which users can interact with them. Issues such as the small screen size, poor connectivity and limited input modalities have an effect on the usability of mobile applications.

The prominent models of usability do not adequately capture the complexities of interacting with applications on a mobile platform. For this reason, this paper presents our PACMAD usability model which augments existing usability models within the context of mobile applications.

To prove the concept of this model a literature review has been conducted. This review has highlighted the extent to which the attributes of the PACMAD model are considered within the mobile application domain. It was found that each attribute was considered in at least 20% of studies, with the exception of Memorability. It is believed one reason for this may be the difficulty associated with evaluating Memorability.

The literature review has also revealed a number of novel interaction methods that are being researched at present, such as spatial awareness and pressure based input. These techniques are in their infancy but with time and more research they may eventually be adopted.

Appendix A: Papers used in the literature review

Apitz, G., F. Guimbretière, and S. Zhai, Foundations for designing and evaluating user interfaces based on the crossing paradigm. ACM Trans. Comput.-Hum. Interact., 2008. 17(2): p. 1–42.

Arning, K. and M. Ziefle, Ask and You Will Receive: Training Novice Adults to use a PDA in an Active Learning Environment. International Journal of Mobile Human Computer Interaction (IJMHCI), 2010. 2(1): p. 21–47.

Arvanitis, T.N., et al., Human factors and qualitative pedagogical evaluation of a mobile augmented reality system for science education used by learners with physical disabilities. Personal Ubiquitous Comput., 2009. 13(3): p. 243–250.

Axtell, C., D. Hislop, and S. Whittaker, Mobile technologies in mobile spaces: Findings from the context of train travel. Int. J. Hum.-Comput. Stud., 2008. 66(12): p. 902–915.

Baber, C., et al., Mobile technology for crime scene examination. Int. J. Hum.-Comput. Stud., 2009. 67(5): p. 464–474.

Bardram, J.E., Activity-based computing for medical work in hospitals. ACM Trans. Comput.-Hum. Interact., 2009. 16(2): p. 1–36.

Bergman, J., J. Kauko, and J. Keränen, Hands on music: physical approach to interaction with digital music, in Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services. 2009, ACM: Bonn, Germany.

Bergman, J. and J. Vainio, Interacting with the flow, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Bertini, E., et al., Appropriating Heuristic Evaluation for Mobile Computing International Journal of Mobile Human Computer Interaction (IJMHCI), 2009. 1(1): p. 20–41.

Böhmer, M. and G. Bauer, Exploiting the icon arrangement on mobile devices as information source for context-awareness, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Bostr, F., et al., Capricorn - an intelligent user interface for mobile widgets, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Brewster, S.A. and M. Hughes, Pressure-based text entry for mobile devices, in Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services. 2009, ACM: Bonn, Germany.

Bruns, E. and O. Bimber, Adaptive training of video sets for image recognition on mobile phones. Personal Ubiquitous Comput., 2009. 13(2): p. 165–178.

Brush, A.J.B., et al., User experiences with activity-based navigation on mobile devices, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Burigat, S., L. Chittaro, and S. Gabrielli, Navigation techniques for small-screen devices: An evaluation on maps and web pages. Int. J. Hum.-Comput. Stud., 2008. 66(2): p. 78–97.

Büring, T., J. Gerken, and H. Reiterer, Zoom interaction design for pen-operated portable devices. Int. J. Hum.-Comput. Stud., 2008. 66(8): p. 605–627.

Buttussi, F., et al., Using mobile devices to support communication between emergency medical responders and deaf people, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Chen, N.Y., F. Guimbretière, and C.E. Löckenhoff, Relative role of merging and two-handed operation on command selection speed. Int. J. Hum.-Comput. Stud., 2008. 66(10): p. 729–740.

Chen, T., Y. Yesilada, and S. Harper, What input errors do you experience? Typing and pointing errors of mobile Web users. Int. J. Hum.-Comput. Stud., 2010. 68(3): p. 138–157.

Cherubini, M., et al., Text versus speech: a comparison of tagging input modalities for camera phones, in Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services. 2009, ACM: Bonn, Germany.

Chittaro, L. and A. Marassi, Supporting blind users in selecting from very long lists of items on mobile phones, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Chittaro, L. and D. Nadalutti, Presenting evacuation instructions on mobile devices by means of location-aware 3D virtual environments, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Clawson, J., et al., Mobiphos: a collocated-synchronous mobile photo sharing application, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Cockburn, A. and C. Gutwin, A model of novice and expert navigation performance in constrained-input interfaces. ACM Trans. Comput.-Hum. Interact., 2010. 17(3): p. 1–38.

Cox, A.L., et al., Tlk or txt? Using voice input for SMS composition. Personal Ubiquitous Comput., 2008. 12(8): p. 567–588.

Crossan, A., et al., Instrumented Usability Analysis for Mobile Devices International Journal of Mobile Human Computer Interaction (IJMHCI), 2009. 1(1): p. 1–19.

Cui, Y., et al., Linked internet UI: a mobile user interface optimized for social networking, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Cummings, M.L., et al., Supporting intelligent and trustworthy maritime path planning decisions. Int. J. Hum.-Comput. Stud., 2010. 68(10): p. 616–626.

Dahl, Y. and D. Svan, A comparison of location and token-based interaction techniques for point-of-care access to medical information. Personal Ubiquitous Comput., 2008. 12(6): p. 459–478.

Dai, L., A. Sears, and R. Goldman, Shifting the focus from accuracy to recallability: A study of informal note-taking on mobile information technologies. ACM Trans. Comput.-Hum. Interact., 2009. 16(1): p. 1–46.

Decle, F. and M. Hachet, A study of direct versus planned 3D camera manipulation on touch-based mobile phones, in Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services. 2009, ACM: Bonn, Germany.

Duh, H.B.-L., V.H.H. Chen, and C.B. Tan, Playing different games on different phones: an empirical study on mobile gaming, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Dunlop, M.D. and M.M. Masters, Investigating five key predictive text entry with combined distance and keystroke modelling. Personal Ubiquitous Comput., 2008. 12(8): p. 589–598.

Ecker, R., et al., pieTouch: a direct touch gesture interface for interacting with in-vehicle information systems, in Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services. 2009, ACM: Bonn, Germany.

Eslambolchilar, P. and R. Murray-Smith, Control centric approach in designing scrolling and zooming user interfaces. Int. J. Hum.-Comput. Stud., 2008. 66(12): p. 838–856.

Fehnert, B. and A. Kosagowsky, Measuring user experience: complementing qualitative and quantitative assessment, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Fickas, S., M. Sohlberg, and P.-F. Hung, Route-following assistance for travelers with cognitive impairments: A comparison of four prompt modes. Int. J. Hum.-Comput. Stud., 2008. 66(12): p. 876–888.

Froehlich, P., et al., Exploring the design space of Smart Horizons, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Gellersen, H., et al., Supporting device discovery and spontaneous interaction with spatial references. Personal Ubiquitous Comput., 2009. 13(4): p. 255–264.

Ghiani, G., B. Leporini, and F. Patern, Vibrotactile feedback as an orientation aid for blind users of mobile guides, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Gostner, R., E. Rukzio, and H. Gellersen, Usage of spatial information for selection of co-located devices, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Goussevskaia, O., M. Kuhn, and R. Wattenhofer, Exploring music collections on mobile devices, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Greaves, A. and E. Rukzio, Evaluation of picture browsing using a projector phone, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Hachet, M., et al., Navidget for 3D interaction: Camera positioning and further uses. Int. J. Hum.-Comput. Stud., 2009. 67(3): p. 225–236.

Hall, M., E. Hoggan, and S. Brewster, T-Bars: towards tactile user interfaces for mobile touchscreens, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Hang, A., E. Rukzio, and A. Greaves, Projector phone: a study of using mobile phones with integrated projector for interaction with maps, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Hardy, R., et al., Mobile interaction with static and dynamic NFC-based displays, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Heikkinen, J., T. Olsson, and K. Väänänen-Vainio-Mattila, Expectations for user experience in haptic communication with mobile devices, in Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services. 2009, ACM: Bonn, Germany.

Henze, N. and S. Boll, Evaluation of an off-screen visualization for magic lens and dynamic peephole interfaces, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Herbst, I., et al., TimeWarp: interactive time travel with a mobile mixed reality game, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Hinze, A.M., C. Chang, and D.M. Nichols, Contextual queries express mobile information needs, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Hutter, H.-P., T. Müggler, and U. Jung, Augmented mobile tagging, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Jones, M., et al., ONTRACK: Dynamically adapting music playback to support navigation. Personal Ubiquitous Comput., 2008. 12(7): p. 513–525.

Joshi, A., et al., Rangoli: a visual phonebook for low-literate users, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Jumisko-Pyykk, S. and M.M. Hannuksela, Does context matter in quality evaluation of mobile television?, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Kaasinen, E., User Acceptance of Mobile Services. International Journal of Mobile Human Computer Interaction (IJMHCI), 2009. 1(1): p. 79–97 pp.

Kaasinen, E., et al., User Experience of Mobile Internet: Analysis and Recommendations. International Journal of Mobile Human Computer Interaction (IJMHCI), 2009. 1(4): p. 4–23.

Kane, S.K., J.O. Wobbrock, and I.E. Smith, Getting off the treadmill: evaluating walking user interfaces for mobile devices in public spaces, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Kang, N.E. and W.C. Yoon, Age- and experience-related user behavior differences in the use of complicated electronic devices. Int. J. Hum.-Comput. Stud., 2008. 66(6): p. 425–437.

Kanjo, E., et al., MobGeoSen: facilitating personal geosensor data collection and visualization using mobile phones. Personal Ubiquitous Comput., 2008. 12(8): p. 599–607.

Kawsar, F., E. Rukzio, and G. Kortuem, An explorative comparison of magic lens and personal projection for interacting with smart objects, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Keijzers, J., E.d. Ouden, and Y. Lu, Usability benchmark study of commercially available smart phones: cell phone type platform, PDA type platform and PC type platform, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Kenteris, M., D. Gavalas, and D. Economou, An innovative mobile electronic tourist guide application. Personal Ubiquitous Comput., 2009. 13(2): p. 103–118.

Komninos, A. and M.D. Dunlop, A calendar based Internet content pre-caching agent for small computing devices. Personal Ubiquitous Comput., 2008. 12(7): p. 495–512.

Kratz, S., I. Brodien, and M. Rohs, Semi-automatic zooming for mobile map navigation, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Kray, C., et al., Bridging the gap between the Kodak and the Flickr generations: A novel interaction technique for collocated photo sharing. Int. J. Hum.-Comput. Stud., 2009. 67(12): p. 1060–1072.

Kristoffersen, S. and I. Bratteberg, Design ideas for IT in public spaces. Personal Ubiquitous Comput., 2010. 14(3): p. 271–286.

Lacroix, J., P. Saini, and R. Holmes, The relationship between goal difficulty and performance in the context of a physical activity intervention program, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Lavie, T. and J. Meyer, Benefits and costs of adaptive user interfaces. Int. J. Hum.-Comput. Stud., 2010. 68(8): p. 508–524.

Lee, J., J. Forlizzi, and S.E. Hudson, Iterative design of MOVE: A situationally appropriate vehicle navigation system. Int. J. Hum.-Comput. Stud., 2008. 66(3): p. 198–215.

Liao, C., et al., Papiercraft: A gesture-based command system for interactive paper. ACM Trans. Comput.-Hum. Interact., 2008. 14(4): p. 1–27.

Lin, P.-C. and L.-W. Chien, The effects of gender differences on operational performance and satisfaction with car navigation systems. Int. J. Hum.-Comput. Stud., 2010. 68(10): p. 777–787.

Lindley, S.E., et al., Fixed in time and “time in motion”: mobility of vision through a SenseCam lens, in Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services. 2009, ACM: Bonn, Germany.

Liu, K. and R.A. Reimer, Social playlist: enabling touch points and enriching ongoing relationships through collaborative mobile music listening, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Liu, N., Y. Liu, and X. Wang, Data logging plus e-diary: towards an online evaluation approach of mobile service field trial, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Liu, Y. and K.-J. Räihä, RotaTxt: Chinese pinyin input with a rotator, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Lucero, A., J. Keränen, and K. Hannu, Collaborative use of mobile phones for brainstorming, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Luff, P., et al., Swiping paper: the second hand, mundane artifacts, gesture and collaboration. Personal Ubiquitous Comput., 2010. 14(3): p. 287–299.

Mallat, N., et al., An empirical investigation of mobile ticketing service adoption in public transportation. Personal Ubiquitous Comput., 2008. 12(1): p. 57–65.

McAdam, C., C. Pinkerton, and S.A. Brewster, Novel interfaces for digital cameras and camera phones, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

McDonald, D.W., et al., Proactive displays: Supporting awareness in fluid social environments. ACM Trans. Comput.-Hum. Interact., 2008. 14(4): p. 1–31.

McKnight, L. and B. Cassidy, Children’s Interaction with Mobile Touch-Screen Devices: Experiences and Guidelines for Design. International Journal of Mobile Human Computer Interaction (IJMHCI), 2010. 2(2): p. 1–18.

Melto, A., et al., Evaluation of predictive text and speech inputs in a multimodal mobile route guidance application, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Miyaki, T. and J. Rekimoto, GraspZoom: zooming and scrolling control model for single-handed mobile interaction, in Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services. 2009, ACM: Bonn, Germany.

Moustakas, K., et al., 3D content-based search using sketches. Personal Ubiquitous Comput., 2009. 13(1): p. 59–67.

Oakley, I. and J. Park, Motion marking menus: An eyes-free approach to motion input for handheld devices. Int. J. Hum.-Comput. Stud., 2009. 67(6): p. 515–532.

Oulasvirta, A., Designing mobile awareness cues, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Oulasvirta, A., S. Estlander, and A. Nurminen, Embodied interaction with a 3D versus 2D mobile map. Personal Ubiquitous Comput., 2009. 13(4): p. 303–320.

Ozok, A.A., et al., A Comparative Study Between Tablet and Laptop PCs: User Satisfaction and Preferences. International Journal of Human-Computer Interaction, 2008. 24(3): p. 329–352.

Park, Y.S., et al., Touch key design for target selection on a mobile phone, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Peevers, G., G. Douglas, and M.A. Jack, A usability comparison of three alternative message formats for an SMS banking service. Int. J. Hum.-Comput. Stud., 2008. 66(2): p. 113–123.

Preuveneers, D. and Y. Berbers, Mobile phones assisting with health self-care: a diabetes case study, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Puikkonen, A., et al., Practices in creating videos with mobile phones, in Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services. 2009, ACM: Bonn, Germany.

Reischach, F.v., et al., An evaluation of product review modalities for mobile phones, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Reitmaier, T., N.J. Bidwell, and G. Marsden, Field testing mobile digital storytelling software in rural Kenya, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Robinson, S., P. Eslambolchilar, and M. Jones, Exploring casual point-and-tilt interactions for mobile geo-blogging. Personal and Ubiquitous Computing, 2010. 14(4): p. 363–379.

Rogers, Y., et al., Enhancing learning: a study of how mobile devices can facilitate sensemaking. Personal Ubiquitous Comput., 2010. 14(2): p. 111–124.

Rohs, M., et al., Impact of item density on the utility of visual context in magic lens interactions. Personal Ubiquitous Comput., 2009. 13(8): p. 633–646.

Sá, M.d. and L. Carriço, Lessons from early stages design of mobile applications, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Sadeh, N., et al., Understanding and capturing people’s privacy policies in a mobile social networking application. Personal Ubiquitous Comput., 2009. 13(6): p. 401–412.

Salvucci, D.D., Rapid prototyping and evaluation of in-vehicle interfaces. ACM Trans. Comput.-Hum. Interact., 2009. 16(2): p. 1–33.

Salzmann, C., D. Gillet, and P. Mullhaupt, End-to-end adaptation scheme for ubiquitous remote experimentation. Personal Ubiquitous Comput., 2009. 13(3): p. 181–196.

Schildbach, B. and E. Rukzio, Investigating selection and reading performance on a mobile phone while walking, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Schmid, F., et al., Situated local and global orientation in mobile you-are-here maps, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Schröder, S. and M. Ziefle, Making a completely icon-based menu in mobile devices to become true: a user-centered design approach for its development, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Scott, J., et al., RearType: text entry using keys on the back of a device, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Seongil, L., Mobile Internet Services from Consumers’ Perspectives. International Journal of Human-Computer Interaction, 2009. 25(5): p. 390–413.

Sharlin, E., et al., A tangible user interface for assessing cognitive mapping ability. Int. J. Hum.-Comput. Stud., 2009. 67(3): p. 269–278.

Sintoris, C., et al., MuseumScrabble: Design of a Mobile Game for Children’s Interaction with a Digitally Augmented Cultural Space. International Journal of Mobile Human Computer Interaction (IJMHCI), 2010. 2(2): p. 53–71.

Smets, N.J.J.M., et al., Effects of mobile map orientation and tactile feedback on navigation speed and situation awareness, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Sodnik, J., et al., A user study of auditory versus visual interfaces for use while driving. Int. J. Hum.-Comput. Stud., 2008. 66(5): p. 318–332.

Sørensen, C. and A. Al-Taitoon, Organisational usability of mobile computing-Volatility and control in mobile foreign exchange trading. Int. J. Hum.-Comput. Stud., 2008. 66(12): p. 916–929.

Stapel, J.C., Y.A.W.d. Kort, and W.A. IJsselsteijn, Sharing places: testing psychological effects of location cueing frequency and explicit vs. inferred closeness, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Streefkerk, J.W., M.P.v. Esch-Bussemakers, and M.A. Neerincx, Field evaluation of a mobile location-based notification system for police officers, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Takayama, L. and C. Nass, Driver safety and information from afar: An experimental driving simulator study of wireless vs. in-car information services. Int. J. Hum.-Comput. Stud., 2008. 66(3): p. 173–184.

Takeuchi, Y. and M. Sugimoto, A user-adaptive city guide system with an unobtrusive navigation interface. Personal Ubiquitous Comput., 2009. 13(2): p. 119–132.

Tan, F.B. and J.P.C. Chou, The Relationship Between Mobile Service Quality, Perceived Technology Compatibility, and Users’ Perceived Playfulness in the Context of Mobile Information and Entertainment Services. International Journal of Human-Computer Interaction, 2008. 24(7): p. 649–671.

Taylor, C.A., N. Samuels, and J.A. Ramey, Always On: A Framework for Understanding Personal Mobile Web Motivations, Behaviors, and Contexts of Use. International Journal of Mobile Human Computer Interaction (IJMHCI), 2009. 1(4): p. 24–41.

Turunen, M., et al., User expectations and user experience with different modalities in a mobile phone controlled home entertainment system, in Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services. 2009, ACM: Bonn, Germany.

Vartiainen, E., Improving the User Experience of a Mobile Photo Gallery by Supporting Social Interaction International Journal of Mobile Human Computer Interaction (IJMHCI), 2009. 1(4): p. 42–57.

Vuolle, M., et al., Developing a questionnaire for measuring mobile business service experience, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Weinberg, G., et al., Contextual push-to-talk: shortening voice dialogs to improve driving performance, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Wilson, G., C. Stewart, and S.A. Brewster, Pressure-based menu selection for mobile devices, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Wobbrock, J.O., B.A. Myers, and H.H. Aung, The performance of hand postures in front- and back-of-device interaction for mobile computing. Int. J. Hum.-Comput. Stud., 2008. 66(12): p. 857–875.

Xiangshi, R. and Z. Xiaolei, The Optimal Size of Handwriting Character Input Boxes on PDAs. International Journal of Human-Computer Interaction, 2009. 25(8): p. 762–784.

Xu, S., et al., Development of a Dual-Modal Presentation of Texts for Small Screens. International Journal of Human-Computer Interaction, 2008. 24(8): p. 776–793.

Yong, G.J. and J.B. Suk, Development of the Conceptual Prototype for Haptic Interface on the Telematics System. International Journal of Human-Computer Interaction, 2010. 26(1): p. 22–52.

Yoo, J.-W., et al., Cocktail: Exploiting Bartenders’ Gestures for Mobile Interaction. International Journal of Mobile Human Computer Interaction (IJMHCI), 2010. 2(3): p. 44–57.

Yoon, Y., et al., Context-aware photo selection for promoting photo consumption on a mobile phone, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

You, Y., et al., Deploying and evaluating a mixed reality mobile treasure hunt: Snap2Play, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Yu, K., F. Tian, and K. Wang, Coupa: operation with pen linking on mobile devices, in Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services. 2009, ACM: Bonn, Germany.

Authors’ note

This research is supported by Oxford Brookes University through the central research fund and in part by Lero - the Irish Software Engineering Research Centre ( http://www.lero.ie ) grant 10/CE/I1855.

Adams R: Decision and stress: cognition and e-accessibility in the information workplace. Springer Universal Access in the Information Society 2007, 5 (4):363–379. 10.1007/s10209-006-0061-9

Article   Google Scholar  

Adams R: Applying advanced concepts of cognitive overload and augmentation in practice; the future of overload. In Foundations of augmented cognition . 2nd edition. Edited by: Schmorrow D, Stanney KM, Reeves LM. Arlington, VA: Springer Berlin Heidelberg; 2006:223–229.

Google Scholar  

Kjeldskov J, Graham C: A review of mobile HCI research methods . Udine, Italy: 5th International Symposium, Mobile HCI 2003; 2003. September 8–11, 2003, Proceedings

Book   Google Scholar  

Nielsen J: Usability engineering. Morgan Kaufmann Pub 1994.

ISO 9241: Ergonomics Requirements for Office Work with Visual Display Terminals (VDTs) International Standards Organisation, Geneva 1997.

Zhang D, Adipat B: Challenges, methodologies, and issues in the usability testing of mobile applications. International Journal of Human-Computer Interaction 2005, 18 (3):293–308. 10.1207/s15327590ijhc1803_3

Guerreiro TJV, Nicolau H, Jorge J, Gonçalves D Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. In Assessing mobile touch interfaces for tetraplegics . Lisbon, Portugal: ACM; 2010. 2010

Chapter   Google Scholar  

Schildbach B, Rukzio E Proceedings of the 12th international conference on human computer interaction with mobile devices and services. In Investigating selection and reading performance on a mobile phone while walking . Lisbon, Portugal: ACM; 2010. 2010

Flood D, Harrison R, Duce D, Iacob C: Evaluating Mobile Applications: A Spreadsheet Case Study. International Journal of Mobile Human Computer Interaction (IJMHCI) 2013, 4 (4):37–65. 10.4018/jmhci.2012100103

Salvucci DD: Predicting the effects of in-car interface use on driver performance: an integrated model approach. International Journal of Human-Computer Studies 2001, 55 (1):85–107. 10.1006/ijhc.2001.0472

Hart SG, Staveland LE: Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. Human mental workload 1988, 1 (3):139–183.

Flood D, Germanakos P, Harrison R, Mc Caffery F: Estimating cognitive overload in mobile applications for decision support within the medical domain . Wroclaw, Poland: 14th International conference on Enterprise Information Systems (ICEIS 2012); 2012.

Budgen D, Burn AJ, Brereton OP, Kitchenham BA, Pretorius R: (2010) Empirical evidence about the UML: a systematic literature review . Software: Practice and Experience; 2010.

Bruns E, Bimber O: Adaptive training of video sets for image recognition on mobile phones. Personal Ubiquitous Comput 2009, 13 (2):165–178. 10.1007/s00779-008-0194-3

Schinke T, Henze N, Boll S Proceedings of the 12th international conference on human computer interaction with mobile devices and services, September 07–10, 2010. In Visualization of off-screen objects in mobile augmented reality . Portugal: Lisbon; 2010.

Smets NJJM, Brake GM, Neerincx MA, Lindenberg J Proceedings of the 10th international conference on human computer interaction with mobile devices and services. In Effects of mobile map orientation and tactile feedback on navigation speed and situation awareness . Amsterdam, The Netherlands: ACM; 2008.

Ghiani G, Leporini B, Patern F Proceedings of the 10th international conference on human computer interaction with mobile devices and services. In Vibrotactile feedback as an orientation aid for blind users of mobile guides . Amsterdam, The Netherlands: ACM; 2008.

Jones M, Jones S, Bradley G, Warren N, Bainbridge D, Holmes G: ONTRACK: Dynamically adapting music playback to support navigation. Personal Ubiquitous Computing 2008, 12 (7):513–525. 10.1007/s00779-007-0155-2

Burigat, S, Chittaro, L, Parlato, E, ACM In proceedings of the 10th international conference on Human computer interaction with mobile devices and services (pp. 147–156). Map, diagram, and web page navigation on mobile devices: the effectiveness of zoomable user interfaces with overviews 2008. September

Sodnik J, Dicke C, Tomaic S, Billinghurst M: A user study of auditory versus visual interfaces for use while driving. Int. J. Hum.-Comput. Stud 2008, 66 (5):318–332. 10.1016/j.ijhcs.2007.11.001

Weinberg G, Harsham B, Forlines C, Medenica Z Proceedings of the 12th international conference on human computer interaction with mobile devices and services. In Contextual push-to-talk: shortening voice dialogs to improve driving performance . Lisbon, Portugal: ACM; 2010. 2010

Park YS, Han SH, Park J, Cho Y Proceedings of the 10th international conference on human computer interaction with mobile devices and services. In Touch key design for target selection on a mobile phone . Amsterdam, The Netherlands: ACM; 2008.

Brewster SA, Hughes M Proceedings of the 11th international conference on human-computer interaction with mobile devices and services. In Pressure-based text entry for mobile devices . Bonn, Germany: ACM; 2009.

Oakley I, Park J: Motion marking menus: an eyes-free approach to motion input for handheld devices. Int J Hum.-Comput. Stud 2009, 67 (6):515–532. 10.1016/j.ijhcs.2009.02.002

Hall M, Hoggan E, Brewster S Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. In T-Bars: towards tactile user interfaces for mobile touchscreens . Amsterdam, The Netherlands: ACM; 2008. 2008

McAdam C, Pinkerton C, Brewster SA Proceedings of the 12th international conference on human computer interaction with mobile devices and services. In Novel interfaces for digital cameras and camera phones . Lisbon, Portugal: ACM; 2010. 2010

Heikkinen J, Olsson T, Väänänen-Vainio-Mattila K Proceedings of the 11th international conference on human-computer interaction with mobile devices and services. In Expectations for user experience in haptic communication with mobile devices . Bonn, Germany: ACM; 2009.

Kristoffersen S, Bratteberg I: Design ideas for IT in public spaces. Personal Ubiquitous Comput 2010, 14 (3):271–286. 10.1007/s00779-009-0255-2

Mallat N, Rossi M, Tuunainen VK, Oörni A: An empirical investigation of mobile ticketing service adoption in public transportation. Personal Ubiquitous Comput 2008, 12 (1):57–65.

Axtell C, Hislop D, Whittaker S: Mobile technologies in mobile spaces: findings from the context of train travel. International Journal of Human Computer Studies 2008, 66 (12):902–915. 10.1016/j.ijhcs.2008.07.001

Fehnert B, Kosagowsky A Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. In Measuring user experience: complementing qualitative and quantitative assessment . Amsterdam, The Netherlands: ACM; 2008.

Lacroix J, Saini P, Holmes R Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. In The relationship between goal difficulty and performance in the context of a physical activity intervention program, . Amsterdam, The Netherlands: ACM; 2008.

Maguire M: Context of use within usability activities. International Journal of Human-Computer Studies 2001, 55 (4):453–483. 2001 10.1006/ijhc.2001.0486

Download references

Author information

Authors and affiliations.

Oxford Brookes University, Oxford, UK

Rachel Harrison, Derek Flood & David Duce

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Rachel Harrison .

Additional information

Competing interests.

The authors declare that they have no competing interests.

Authors’ contributions

DF performed the literature review, helped to propose the PACMAD model and drafted the manuscript. RH assisted the literature review, proposed the PACMAD model and drafted the limitations section. DAD helped to refine the conceptual framework and direct the research. All authors read and approved the final manuscript.

Authors’ original submitted files for images

Below are the links to the authors’ original submitted files for images.

Authors’ original file for figure 1

Authors’ original file for figure 2, rights and permissions.

Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License ( https://creativecommons.org/licenses/by/2.0 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article.

Harrison, R., Flood, D. & Duce, D. Usability of mobile applications: literature review and rationale for a new usability model. J Interact Sci 1 , 1 (2013). https://doi.org/10.1186/2194-0827-1-1

Download citation

Received : 10 March 2013

Accepted : 10 March 2013

Published : 07 May 2013

DOI : https://doi.org/10.1186/2194-0827-1-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Mobile Phone
• Mobile Device
• Cognitive Load
• Augmented Reality
• Usability Model

Mobile Application Development: A comprehensive and systematic literature review

Hanif khan , jagadeesan s , vinayak a. drave & priyanka chand bhatt, publisher: ieom society international, track: technology management.

In the present age of e-commerce, mobile penetration has risen to a significant level and can be seen as it grows exponentially in the past decade. The mobile application has opened up a new sense of customer service. With an increase of these applications, researchers have inclined their interest towards development and identify various issues linked to it. This study focuses on accumulating a vast academic literature in a time frame of the last ten years. A comprehensive and systematic literature review was done in the emerging area of mobile application development. After the process of reviewing the extensive literature from the biggest database ‘Scopus’, a total of 26 relevant journal papers were considered for the review after multiple steps of filtrations. This paper gives the scope and the direction in the field of mobile development and helps the organization to have a vast knowledge regarding the same. The findings of this study provide valuable insights and future recommendations in this developing field.

Published in : 3rd European International Conference on Industrial Engineering and Operations Management, Pilsen, Czech Republic

Publisher : IEOM Society International Date of Conference : July 23-26, 2019

ISBN : 978-1-5323-5949-1 ISSN/E-ISSN : 2169-8767

Related Research

THE PHENOMENON OF E-WALLET USAGE IN INDONESIA BASED ON THE THEORY OF TECHNOLOGY ACCEPTANCE MODEL

5th European International Conference on Industrial Engineering and Operations Management

Published: 2022

Impact of Technology Adoption in Forward Logistics: A Quantitative Approach

12th Annual International Conference on Industrial Engineering and Operations Management

“The Influence of Various E-Learning techniques upon Technology Acceptance and student engagement in differing Classroom Environments” - A Research Review

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• PeerJ Comput Sci

A systematic literature review on the usability of mobile applications for visually impaired users

Muna al-razgan.

1 King Saud University, Riyadh, Saudi Arabia

Sarah Almoaiqel

Nuha alrajhi.

2 Imam Muhammad Ibn Saud University, Riyadh, Saudi Arabia

Alyah Alhumegani

Abeer alshehri, bashayr alnefaie, raghad alkhamiss, shahad rushdi, associated data.

The following information was supplied regarding data availability:

This is a systematic literature review; there is no raw data.

Interacting with mobile applications can often be challenging for people with visual impairments due to the poor usability of some mobile applications. The goal of this paper is to provide an overview of the developments on usability of mobile applications for people with visual impairments based on recent advances in research and application development. This overview is important to guide decision-making for researchers and provide a synthesis of available evidence and indicate in which direction it is worthwhile to prompt further research. We performed a systematic literature review on the usability of mobile applications for people with visual impairments. A deep analysis following the Preferred Reporting Items for SLRs and Meta-Analyses (PRISMA) guidelines was performed to produce a set of relevant papers in the field. We first identified 932 papers published within the last six years. After screening the papers and employing a snowballing technique, we identified 60 studies that were then classified into seven themes: accessibility, daily activities, assistive devices, navigation, screen division layout, and audio guidance. The studies were then analyzed to answer the proposed research questions in order to illustrate the different trends, themes, and evaluation results of various mobile applications developed in the last six years. Using this overview as a foundation, future directions for research in the field of usability for the visually impaired (UVI) are highlighted.

Introduction

The era of mobile devices and applications has begun. With the widespread use of mobile applications, designers and developers need to consider all types of users and develop applications for their different needs. One notable group of users is people with visual impairments. According to the World Health Organization, there are approximately 285 million people with visual impairments worldwide ( World Health Organization, 2020 ). This is a huge number to keep in mind while developing new mobile applications.

People with visual impairments have urged more attention from the tech community to provide them with the assistive technologies they need ( Khan & Khusro, 2021 ). Small tasks that we do daily, such as picking out outfits or even moving from one room to another, could be challenging for such individuals. Thus, leveraging technology to assist with such tasks can be life changing. Besides, increasing the usability of applications and developing dedicated ones tailored to their needs is essential. The usability of an application refers to its efficiency in terms of the time and effort required to perform a task, its effectiveness in performing said tasks, and its users’ satisfaction ( Ferreira et al., 2020 ). Researchers have been studying this field intensively and proposing different solutions to improve the usability of applications for people with visual impairments.

This paper provides a systematic literature review (SLR) on the usability of mobile applications for people with visual impairments. The study aims to find discussions of usability issues related to people with visual impairments in recent studies and how they were solved using mobile applications. By reviewing published works from the last six years, this SLR aims to update readers on the newest trends, limitations of current research, and future directions in the research field of usability for the visually impaired (UVI).

This SLR can be of great benefit to researchers aiming to become involved in UVI research and could provide the basis for new work to be developed, consequently improving the quality of life for the visually impaired. This review differs from previous review studies ( i.e.,   Khan & Khusro, 2021 ) because we classified the studies into themes in order to better evaluate and synthesize the studies and provide clear directions for future work. The following themes were chosen based on the issues addressed in the reviewed papers: “Assistive Devices,” “Navigation,” “Accessibility,” “Daily Activities,” “Audio Guidance,” and “Gestures.” Figure 1 illustrates the percentage of papers classified in each theme.

The remainder of this paper is organized as follows: the next section specifies the methodology, following this, the results section illustrates the results of the data collection, the discussion section consists of the research questions with their answers and the limitations and potential directions for future work, and the final section summarizes this paper’s main findings and contribution.

Survey Methodology

This systematic literature review used the Meta-Analyses (PRISMA, 2009) guidelines to produce a set of relevant papers in the field. This SLR was undertaken to address the research questions described below. A deep analysis was performed based on a group of studies; the most relevant studies were documented, and the research questions were addressed.

A. Research questions

The research questions addressed by this study are presented in Table 1 with descriptions and the motivations behind them.

B. Search strategy

This review analysed and synthesised studies on usability for the visually impaired from a user perspective following a systematic approach. As proposed by Tanfield, Denyer & Smart (2003) , the study followed a three-stage approach to ensure that the findings were both reliable and valid. These stages were planning the review, conducting the review by analysing papers, and reporting emerging themes and recommendations. These stages will be discussed further in the following section.

1. Planning stage

The planning stage of this review included defining data sources and the search string protocol as well as inclusion and exclusion criteria.

Data sources.

We aimed to use two types of data sources: digital libraries and search engines. The search process was manually conducted by searching through databases. The selected databases and digital libraries are as follows:

• • ACM Library
• • IEEE Xplore
• • ScienceDirect
• • SpringerLink
• • ISI Web of Knowledge
• • Scopus.

The selected search engines were as follows:

• • DBLP (Computer Science Bibliography Website)
• • Google Scholar
• • Microsoft Academic

Search string.

The above databases were initially searched using the following keyword protocol: (“Usability” AND (”visual impaired” OR ”visually impaired” OR “blind” OR “impairment”) AND “mobile”). However, in order to generate a more powerful search string, the Network Analysis Interface for Literature Studies (NAILS) project was used. NAILS is an automated tool for literature analysis. Its main function is to perform statistical and social network analysis (SNA) on citation data ( Knutas et al., 2015 ). In this study, it was used to check the most important work in the relevant fields as shown in Fig. 2 .

NAILS produced a report displaying the most important authors, publications, and keywords and listed the references cited most often in the analysed papers ( Knutas et al., 2015 ) . The new search string was generated after using the NAILS project as follows: (“Usability” OR “usability model” OR “usability dimension” OR “Usability evaluation model” OR “Usability evaluation dimension”) AND (“mobile” OR “Smartphone”) AND (“Visually impaired” OR “Visual impairment” OR “Blind” OR “Low vision” OR “Blindness”).

Inclusion and exclusion criteria.

To be included in this systematic review, each study had to meet the following screening criteria:

• • The study must have been published between 2015 and 2020.
• • The study must be relevant to the main topic (Usability of Mobile Applications for Visually Impaired Users).
• • The study must be a full-length paper.
• • The study must be written in English because any to consider any other languages, the research team will need to use the keywords of this language in this topic and deal with search engines using that language to extract all studies related to our topic to form an SLR with a comprehensive view of the selected languages. Therefore, the research team preferred to focus on studies in English to narrow the scope of this SLR.

A research study was excluded if it did not meet one or more items of the criteria.

2. Conducting stage

The conducting stage of the review involved a systematic search based on relevant search terms. This consisted of three substages: exporting citations, importing citations into Mendeley, and importing citations into Rayyan.

Exporting citations.

First, in exporting the citations and conducting the search through the mentioned databases, a total of 932 studies were found. The numbers are illustrated in Fig. 3 below. The highest number of papers was found in Google Scholar, followed by Scopus, ISI Web of Knowledge, ScienceDirect, IEEE Xplore, Microsoft Academic, and DBLP and ACM Library with two studies each. Finally, SpringerLink did not have any studies that met the inclusion criteria.

The chance of encountering duplicate studies was determined to be high. Therefore, importing citations into Mendeley was necessary in order to eliminate the duplicates.

Importing citations into mendeley.

Mendeley is an open-source reference and citation manager. It can highlight paragraphs and sentences, and it can also list automatic references on the end page. Introducing the use of Mendeley is also expected to avoid duplicates in academic writing, especially for systematic literature reviews ( Basri & Patak, 2015 ). Hence, in the next step, the 932 studies were imported into Mendeley, and each study’s title and abstract were screened independently for eligibility. A total of 187 duplicate studies were excluded. 745 total studies remained after the first elimination process. The search stages are shown in Fig. 4 below.

Importing citations into rayyan.

Rayyan QCRI is a free web and mobile application that helps expedite the initial screening of both abstracts and titles through a semi-automated process while incorporating a high level of usability. Its main benefit is to speed up the most tedious part of the systematic literature review process: selecting studies for inclusion in the review ( Ouzzani et al., 2016 ). Therefore, for the last step, another import was done using Rayyan to check for duplications a final time. Using Rayyan, a total of 124 duplicate studies were found, resulting in a total of 621 studies. Using Rayyan, a two-step filtration was conducted to guarantee that the papers have met the inclusion criteria of this SLR. After filtering based on the abstracts, 564 papers did not meet the inclusion criteria. At this stage, 57 studies remained. The second step of filtration eliminated 11 more studies by reading the full papers; two studies were not written in the English language, and nine were inaccessible.

Snowballing.

Snowballing is an emerging technique used to conduct systematic literature reviews that are considered both efficient and reliable using simple procedures. The procedure for snowballing consisted of three phases in each cycle. The first phase is refining the start set, the second phase is backward snowballing, and the third is forward snowballing. The first step, forming the start set, is basically identifying relevant papers that can have a high potential of satisfying the criteria and research question. Backward snowballing was conducted using the reference list to identify new papers to include. It shall start by going through the reference list and excluding papers that do not fulfill the basic criteria; the rest that fulfil criteria shall be added to the SLR. Forward snowballing refers to identifying new papers based on those papers that cited the paper being examined ( Juneja & Kaur, 2019 ). Hence, in order to be sure that we concluded all related studies after we got the 46 papers, a snowballing step was essential. Forward and backward snowballing were conducted. Each of the 46 studies was examined by checking their references to take a look at any possible addition of sources and examining all papers that cited this study. The snowballing activity added some 38 studies, but after full reading, it became 33 that matched the inclusion criteria. A total of 79 studies were identified through this process.

Quality assessment.

A systematic literature review’s quality is determined by the content of the papers included in the review. As a result, it is important to evaluate the papers carefully ( Zhou et al., 2015 ). Many influential scales exist in the software engineering field for evaluating the validity of individual primary studies and grading the overall intensity of the body of proof. Hence, we adapted the comprehensive guidelines specified by Kitchenhand and Charters ( Keele, 2007 ), and the quasi-gold standard (QGS) ( Keele, 2007 ) was used to establish the quest technique, where a robust search strategy for enhancing the validity and reliability of a SLR’s search process is devised using the QGS. By applying this technique, our quality assessment questions were focused and aligned with the research questions mentioned earlier.

In our last step, we had to verify the papers’ eligibility; we conducted a quality check for each of the 79 studies. For quality assessment, we considered whether the paper answered the following questions:

QA1: Is the research aim clearly stated in the research?

QA2: Does the research contain a usability dimension or techniques for mobile applications for people with visual impairments?

QA3: Is there an existing issue with mobile applications for people with visual impairments that the author is trying to solve?

QA4: Is the research focused on mobile application solutions?

After discussing the quality assessment questions and attempting to find an answer in each paper, we agreed to score each study per question. If the study answers a question, it will be given 2 points; if it only partially answers a question, it will be given 1 point; and if there is no answer for a given question in the study, it will have 0 points.

The next step was to calculate the weight of each study. If the total weight was higher or equal to four points, the paper was accepted in the SLR; if not, the paper was discarded since it did not reach the desired quality level. Figure 5 below illustrates the quality assessment process. After applying the quality assessment, 39 papers were rejected since they received less than four points, which resulted in a final tally of 60 papers.

To summarize, this review was conducted according to the Preferred Reporting Items for SLRs and Meta-Analyses (PRISMA) ( Liberati et al., 2009 ). The PRISMA diagram shown in Fig. 6 illustrates all systematic literature processes used in this study.

3. Analysing stage

All researchers involved in this SLR collected the data. The papers were distributed equally between them, and each researcher read each paper completely to determine its topic, extract the paper’s limitations and future work, write a quick summary about it, and record this information in an Excel spreadsheet.

All researchers worked intensively on this systematic literature review. After completing the previously mentioned steps, the papers were divided among all the researchers. Then, each researcher read their assigned papers completely and then classified them into themes according to the topic they covered. The researchers held several meetings to discuss and specify those themes. The themes were identified by the researchers based on the issues addressed in the reviewed papers. In the end, the researchers resulted in seven themes, as shown in Fig. 7 below. The references selected for each theme can be found in the Table A1 . Afterwards, each researcher was assigned one theme to summarize its studies and report the results. In this section, we review the results.

A. Accessibility

Of a total of 60 studies, 10 focused on issues of accessibility. Accessibility is concerned with whether all users are able to have equivalent user experiences, regardless of abilities. Six studies, Darvishy, Hutter & Frei (2019) , Morris et al. (2016) , Qureshi & Hooi-Ten Wong (2020) , Khan, Khusro & Alam (2018) , Paiva et al. (2020) , and Pereda, Murillo & Paz (2020) , gave suggestions for increasing accessibility, ( Darvishy, Hutter & Frei, 2019 ; Morris et al., 2016 ), gave some suggestions for making mobile map applications and Twitter accessible to visually impaired users, and ( Qureshi & Hooi-Ten Wong, 2020 ; Khan, Khusro & Alam, 2018 ) focused on user interfaces and provided accessibility suggestions suitable for blind people. Paiva et al. (2020) and Pereda, Murillo & Paz (2020) proposed a set of heuristics to evaluate the accessibility of mobile applications. Two studies, Khowaja et al. (2019) and Carvalho et al. (2018) , focused on evaluating usability and accessibility issues on some mobile applications, comparing them, and identifying the number and types of problems that visually impaired users faced. Aqle, Khowaja & Al-Thani (2020) proposed a new web search interface designed for visually impaired users. One study, McKay (2017) , focused on accessibility challenges by applying usability tests on a hybrid mobile app with some visually impaired university students.

B. Assistive devices

People with visual impairments have an essential need for assistive technology since they face many challenges when performing activities in daily life. Out of the 60 studies reviewed, 13 were related to assistive technology. The studies Smaradottir, Martinez & Håland (2017) , Skulimowski et al. (2019) , Barbosa, Hayes & Wang, (2016) , Rosner & Perlman (2018) , Csapó et al. (2015) , Khan & Khusro (2020) , Sonth & Kallimani (2017) , Kim et al. (2016) , Vashistha et al. (2015) ; Kameswaran et al. (2020) , Griffin-Shirley et al. (2017) , and Rahman, Anam & Yeasin (2017) were related to screen readers (voiceovers). On the other hand, Bharatia, Ambawane & Rane (2019) , Lewis et al. (2016) were related to proposing an assistant device for the visually impaired. Of the studies related to screening readers, Sonth & Kallimani, (2017) , Vashistha et al. (2015) , Khan & Khusro (2020) Lewis et al. (2016) cited challenges faced by visually impaired users. Barbosa, Hayes & Wang (2016) , Kim et al. (2016) , Rahman, Anam & Yeasin (2017) suggested new applications, while Smaradottir, Martinez & Håland (2017) , Rosner & Perlman (2018) , Csapó et al. (2015) and Griffin-Shirley et al. (2017) evaluated current existing work. The studies Bharatia, Ambawane & Rane (2019) , Lewis et al. (2016) proposed using wearable devices to improve the quality of life for people with visual impairments.

C. Daily activities

In recent years, people with visual impairments have used mobile applications to increase their independence in their daily activities and learning, especially those based on the braille method. We divide the daily activity section into braille-based applications and applications designed to enhance the independence of the visually impaired. Four studies, Nahar, Sulaiman & Jaafar (2020) , Nahar, Jaafar & Sulaiman (2019) , Araújo et al. (2016) and Gokhale et al. (2017) , implemented and evaluated the usability of mobile phone applications that use braille to help visually impaired people in their daily lives. Seven studies, Vitiello et al. (2018) , Kunaratana-Angkul, Wu & Shin-Renn (2020) , Ghidini et al. (2016) , Madrigal-Cadavid et al. (2019) , Marques, Carriço & Guerreiro (2015) , Oliveira et al. (2018) and Rodrigues et al. (2015) , focused on building applications that enhance the independence and autonomy of people with visual impairments in their daily life activities.

D. Screen division layout

People with visual impairments encounter various challenges in identifying and locating non-visual items on touch screen interfaces like phones and tablets. Incidents of accidentally touching a screen element and frequently following an incorrect pattern in attempting to access objects and screen artifacts hinder blind people from performing typical activities on smartphones ( Khusro et al., 2019 ). In this review, 9 out of 60 studies discuss screen division layout: ( Khusro et al., 2019 ; Khan & Khusro, 2019 ; Grussenmeyer & Folmer, 2017 ; Palani et al., 2018 ; Leporini & Palmucci, 2018 ) discuss touch screen (smartwatch tablets, mobile phones, and tablet) usability among people with visual impairments, while ( Cho & Kim, 2017 ; Alnfiai & Sampalli, 2016 ; Niazi et al., 2016 ; Alnfiai & Sampalli, 2019 ) concern text entry methods that increase the usability of apps among visually impaired people. Khusro et al. (2019) provides a novel contribution to the literature regarding considerations that can be used as guidelines for designing a user-friendly and semantically enriched user interface for blind people. An experiment in Cho & Kim (2017) was conducted comparing the two-button mobile interface usability with the one-finger method and voiceover. Leporini & Palmucci (2018) gathered information on the interaction challenges faced by visually impaired people when answering questions on a mobile touch-screen device, investigated possible solutions to overcome the accessibility and usability challenges.

E. Gestures

In total, 3 of 60 studies discuss gestures in usability. Alnfiai & Sampalli (2017) compared the performance of BrailleEnter, a gesture based input method to the Swift Braille keyboard, a method that requires finding the location of six buttons representing braille dot, while Buzzi et al. (2017) and Smaradottir, Martinez & Haland (2017) provide an analysis of gesture performance on touch screens among visually impaired people.

F. Audio guidance

People with visual impairment primarily depend on audio guidance forms in their daily lives; accordingly, audio feedback helps guide them in their interaction with mobile applications.

Four studies discussed the use of audio guidance in different contexts: one in navigation ( Gintner et al., 2017 ), one in games ( Ara’ujo et al., 2017 ), one in reading ( Sabab & Ashmafee, 2016 ), and one in videos ( Façanha et al., 2016 ). These studies were developed and evaluated based on usability and accessibility of the audio guidance for people with visual impairments and aimed to utilize mobile applications to increase the enjoyment and independence of such individuals.

G. Navigation

Navigation is a common issue that visually impaired people face. Indoor navigation is widely discussed in the literature. Nair et al. (2020) , Al-Khalifa & Al-Razgan (2016) and De Borba Campos et al. (2015) discuss how we can develop indoor navigation applications for visually impaired people. Outdoor navigation is also common in the literature, as seen in Darvishy et al. (2020) , Hossain, Qaiduzzaman & Rahman (2020) , Long et al. (2016) , Prerana et al. (2019) and Bandukda et al. (2020) . For example, in Darvishy et al. (2020) , Touch Explorer, an accessible digital map application, was presented to alleviate many of the problems faced by people with visual impairments while using highly visually oriented digital maps. Primarily, it focused on using non-visual output modalities like voice output, everyday sound, and vibration feedback. Issues with navigation applications were also presented in Maly et al. (2015) . Kameswaran et al. (2020) discussed commonly used technologies in navigation applications for blind people and highlighted the importance of using complementary technologies to convey information through different modalities to enhance the navigation experience. Interactive sonification of images for navigation has also been shown in Skulimowski et al. (2019) .

In this section, the research questions are addressed in detail to clearly achieve the research objective. Also, a detailed overview of each theme will be mentioned below.

Answers to the research questions

This section will answer the research question proposed:

RQ1: What existing UVI issues did authors try to solve with mobile devices?

Mobile applications can help people with visual impairments in their daily activities, such as navigation and writing. Additionally, mobile devices may be used for entertainment purposes. However, people with visual impairments face various difficulties while performing text entry operations, text selection, and text manipulation on mobile applications ( Niazi et al., 2016 ). Thus, the authors of the studies tried to increase touch screens’ usability by producing prototypes or simple systems and doing usability testing to understand the UX of people with visual impairments.

RQ2: What is the role of mobile devices in solving those issues?

Mobile phones are widely used in modern society, especially among users with visual impairments; they are considered the most helpful tool for blind users to communicate with people worldwide ( Smaradottir, Martinez & Håland, 2017 ). In addition, the technology of touch screen assistive technology enables speech interaction between blind people and mobile devices and permits the use of gestures to interact with a touch user interface. Assistive technology is vital in helping people living with disabilities perform actions or interact with systems ( Niazi et al., 2016 ).

RQ3: What are the publication trends on the usability of mobile applications among the visually impaired?

As shown in Fig. 8 below, research into mobile applications’ usability for the visually impaired has increased in the last five years, with a slight dip in 2018. Looking at the most frequent themes, we find that “Assistive Devices” peaked in 2017, while “Navigation” and “Accessibility” increased significantly in 2020. On the other hand, we see that the prevalence of “Daily Activities” stayed stable throughout the research years. The term “Audio Guidance” appeared in 2016 and 2017 and has not appeared in the last three years. “Gestures” also appeared only in 2017. “Screen Layout Division” was present in the literature in the last five years and increased in 2019 but did not appear in 2020.

RQ4: What are the current research limitations and future research directions regarding usability among the visually impaired?

We divide the answer to this question into two sections: first, we will discuss limitations; then, we will discuss future work for each proposed theme.

A. Limitations

Studies on the usability of mobile applications for visually impaired users in the literature have various limitations, and most of them were common among the studies. These limitations were divided into two groups. The first group concerns proposed applications; for example, Rahman, Anam & Yeasin (2017) , Oliveira et al. (2018) and Madrigal-Cadavid et al. (2019) faced issues regarding camera applications in mobile devices due to the considerable effort needed for its usage and being heavily dependent on the availability of the internet. The other group of studies, Rodrigues et al. (2015) , Leporini & Palmucci (2018) , Alnfiai & Sampalli (2016) , and Ara’ujo et al. (2017) , have shown limitations in visually impaired users’ inability to comprehend a graphical user interface. Alnfiai & Sampalli (2017) and Alnfiai & Sampalli (2019) evaluated new braille input methods and found that the traditional braille keyboard, where knowing the exact position of letters QWERTY is required, is limited in terms of usability compared to the new input methods. Most studies faced difficulties regarding the sample size and the fact that many of the participants were not actually blind or visually impaired but only blindfolded. This likely led to less accurate results, as blind or visually impaired people can provide more useful feedback as they experience different issues on a daily basis and are more ideal for this type of study. So, the need for a good sample of participants who actually have this disability is clear to allow for better evaluation results and more feedback and recommendations for future research.

B. Future work

A commonly discussed future work in the chosen literature is to increase the sample sizes of people with visual impairment and focus on various ages and geographical areas to generalize the studies. Table 2 summarizes suggestions for future work according to each theme. Those future directions could inspire new research in the field.

RQ5: What is the focus of research on usability for visually impaired people, and what are the research outcomes in the studies reviewed?

There are a total of 60 outcomes in this research. Of these, 40 involve suggestions to improve usability of mobile applications; four of them address problems that are faced by visually impaired people that reduce usability. Additionally, 16 of the outcomes are assessments of the usability of the prototype or model. Two of the results are recommendations to improve usability. Finally, the last two outcomes are hardware solutions that may help the visually impaired perform their daily activities. Figure 9 illustrates these numbers.

Overview of the reviewed studies

In the following subsections, we summarize all the selected studies based on the classified theme: accessibility, assistive devices, daily activities, screen division layout, gestures, audio guidance, and navigation. The essence of the studies will be determined, and their significance in the field will be explored.

For designers dealing with mobile applications, it is critical to determine and fix accessibility issues in the application before it is delivered to the users ( Khowaja et al., 2019 ). Accessibility refers to giving the users the same user experience regardless of ability. In Khowaja et al. (2019) and Carvalho et al. (2018) , the researchers focused on comparing the levels of accessibility and usability in different applications. They had a group of visually impaired users and a group of sighted users test out the applications to compare the number and type of problems they faced and determine which applications contained the most violations. Because people with visual impairments cannot be ignored in the development of mobile applications, many researchers have sought solutions for guaranteeing accessibility. For example, in Qureshi & Hooi-Ten Wong (2020) , the study contributed to producing a new, effective design for mobile applications based on the suggestions of people with visual impairments and with the help of two expert mobile application developers. In Khan, Khusro & Alam (2018) , an adaptive user interface model for visually impaired people was proposed and evaluated in an empirical study with 63 visually impaired people. In Aqle, Khowaja & Al-Thani (2020) , the researchers proposed a new web search interface for users with visual impairments that is based on discovering concepts through formal concept analysis (FCA). Users interact with the interface to collect concepts, which are then used as keywords to narrow the search results and target the web pages containing the desired information with minimal effort and time. The usability of the proposed search interface (InteractSE) was evaluated by experts in the field of HCI and accessibility, with a set of heuristics by Nielsen and a set of WCAG 2.0 guidelines.

In Darvishy, Hutter & Frei (2019) , the researchers proposed a solution for making mobile map applications accessible for people with blindness or visual impairment. They suggested replacing forests in the map with green color and birds’ sound, replacing water with blue color and water sounds, replacing streets with grey color and vibration, and replacing buildings with yellow color and pronouncing the name of the building. The prototype showed that it was possible to explore a simple map through vibrations, sounds, and speech.

In Morris et al. (2016) the researchers utilized a multi-faceted technique to investigate how and why visually impaired individuals use Twitter and the difficulties they face in doing so. They noted that Twitter had become more image-heavy over time and that picture-based tweets are largely inaccessible to people with visual impairments. The researchers then made several suggestions for how Twitter could be amended to continue to be usable for people with visual impairments.

The researchers in Paiva et al. (2020) focused on how to evaluate proposed methods for ensuring the accessibility and usability of mobile applications. Their checklist, Acc-MobileCheck, contains 47 items that correspond to issues related to comprehension (C), operation (O), perception (P), and adaptation (A) in mobile interface interaction. To validate Acc-MobileCheck, it was reviewed by five experts and three developers and determined to be effective. In Pereda, Murillo & Paz (2020) , the authors also suggest a set of heuristics to evaluate the accessibility of mobile e-commerce applications for visually impaired people. Finally, McKay (2017) conducted an accessibility test for hybrid mobile apps and found that students with blindness faced many barriers to access based on how they used hybrid mobile applications. While hybrid apps can allow for increased time for marketing, this comes at the cost of app accessibility for people with disabilities.

A significant number of people with visual impairments use state-of-the-art software to perform tasks in their daily lives. These technologies are made up of electronic devices equipped with sensors and processors that can make intelligent decisions.

One of the most important and challenging tasks in developing such technologies is to create a user interface that is appropriate for the sensorimotor capabilities of users with blindness ( Csapó et al., 2015 ). Several new hardware tools have proposed to improve the quality of life for people with visual impairments. Three tools were presented in this SLR: a smart stick that can notify the user of any obstacle, helping them to perform tasks easily and efficiently ( Bharatia, Ambawane & Rane, 2019 ), and an eye that can allow users to detect colors (medical evaluation is still required) ( Lewis et al., 2016 ).

The purpose of the study in Griffin-Shirley et al. (2017) was to understand how people with blindness use smartphone applications as assistive technology and how they perceive them in terms of accessibility and usability. An online survey with 259 participants was conducted, and most of the participants rated the applications as useful and accessible and were satisfied with them.

The researchers in Rahman, Anam & Yeasin (2017) designed and implemented EmoAssist, which is a smartphone application that assists with natural dyadic conversations and aims to promote user satisfaction by providing options for accessing non-verbal communication that predicts behavioural expressions and contains interactive dimensions to provide valid feedback. The usability of this application was evaluated in a study with ten people with blindness where several tools were applied in the application. The study participants found that the usability of EmoAssist was good, and it was an effective assistive solution.

This theme contains two main categories: braille-based application studies and applications to enhance the independence of VIU. Both are summarized below.

1- Braille-based applications

Braille is still the most popular method for assisting people with visual impairments in reading and studying, and most educational mobile phone applications are limited to sighted people. Recently, however, some researchers have developed assistive education applications for students with visual impairments, especially those in developing countries. For example, in India, the number of children with visual impairments is around 15 million, and only 5% receive an education ( Gokhale et al., 2017 ). Three of the braille studies focused on education: ( Nahar, Sulaiman & Jaafar, 2020 ; Nahar, Jaafar & Sulaiman, 2019 , and Araújo et al., 2016 ). These studies all used smartphone touchscreens and action gestures to gain input from the student, and then output was provided in the form of audio feedback. In Nahar, Sulaiman & Jaafar (2020) , vibrational feedback was added to guide the users. The participants in Nahar, Sulaiman & Jaafar (2020) ; Nahar, Jaafar & Sulaiman (2019) , and Araújo et al. (2016) included students with blindness of visual impairment and their teachers. The authors in Nahar, Sulaiman & Jaafar (2020) , Nahar, Jaafar & Sulaiman (2019) evaluated the usability of their applications following the same criteria (efficiency, learnability, memorability, errors, and satisfaction). The results showed that in Nahar, Sulaiman & Jaafar (2020) , Nahar, Jaafar & Sulaiman (2019) , and Araújo et al. (2016) , the applications met the required usability criteria. The authors in Gokhale et al. (2017) presented a braille-based solution to help people with visual impairments call and save contacts. A braille keypad on the smartphone touchscreen was used to gain input from the user, which was then converted into haptic and auditory feedback to let the user know what action was taken. The usability of this application was considered before it was designed. The participants’ responses were positive because this kind of user-centric design simplifies navigation and learning processes.

2- Applications to enhance the independence of people with visual impairments

The authors in the studies explored in this section focused on building applications that enhance independence and autonomy in daily life activities for users with visual impairments.

In Vitiello et al. (2018) , the authors presented their mobile application, an assistive solution for visually impaired users called “Crania”, which uses machine learning techniques to help users with visual impairments get dressed by recognizing the colour and texture of their clothing and suggesting suitable combinations. The system provides feedback through voice synthesis. The participants in the study were adults and elderly people, some of whom were completely blind and the rest of whom had partial sight. After testing for usability, all the participants with blindness agreed that using the application was better than their original method, and half of the participants with partial sight said the same thing. At the end of the study, the application was determined to be accessible and easy to use.

In Kunaratana-Angkul, Wu & Shin-Renn (2020) , an application which allows elderly people to measure low vision status at home through their smartphones instead of visiting hospitals was tested, and most of the participants considered it to be untrustworthy because the medical information was insufficient. Even when participants were able to learn how to use the application, most of them were still confused while using it and needed further instruction.

In Ghidini et al. (2016) , the authors studied the habits of people with visual impairments when using their smartphones in order to develop an electronic calendar with different interaction formats, such as voice commands, touch, and vibration interaction. The authors presented the lessons learned and categorized them based on usability heuristics such as feedback, design, user freedom and control, and recognition instead of remembering.

In Madrigal-Cadavid et al. (2019) , the authors developed a drug information application for people with visual impairments to help them access the labels of medications. The application was developed based on a user-centered design process. By conducting a usability test, the authors recognized some usability issues for people with visual impairments, such as difficulty in locating the bar code. Given this, a new version will include a search function that is based on pictures. The application is searched by capturing the bar code or text or giving voice commands that allow the user to access medication information. The participants were people with visual impairments, and most of them required assistance using medications before using the application. This application will enhance independence for people with visual impairments in terms of using medications.

In Marques, Carriço & Guerreiro (2015) , an authentication method is proposed for users with visual impairments that allows them to protect their passwords. It is not secure when blind or visually impaired users spell out their passwords or enter the numbers in front of others, and the proposed solution allows the users to enter their password with one hand by tapping the screen. The blind participants in this study demonstrated that this authentication method is usable and supports their security needs.

In Oliveira et al. (2018) , the author noted that people with visual impairments face challenges in reading, thus he proposed an application called LeR otulos. This application was developed and evaluated for the Android operating system and recognizes text from photos taken by the mobile camera and converts them into an audio description. The prototype was designed to follow the guidelines and recommendations of usability and accessibility. The requirements of the application are defined based on the following usability goals: the steps are easy for the user to remember; the application is efficient, safe, useful, and accessible; and user satisfaction is achieved.

Interacting with talkback audio devices is still difficult for people with blindness, and it is unclear how much benefit they provide to people with visual impairments in their daily activities. The author in Rodrigues et al. (2015) investigates the smartphone adoption process of blind users by conducting experiments, observations, and weekly interviews. An eight-week study was conducted with five visually impaired participants using Samsung and an enabled talkback 2 screen reader. Focusing on understanding the experiences of people with visual impairments when using touchscreen smartphones revealed accessibility and usability issues. The results showed that the participants have difficulties using smartphones because they fear that they cannot use them properly, and that impacts their ability to communicate with family. However, they appreciate the benefits of using smartphones in their daily activities, and they have the ability to use them.

People with visual impairments encounter various challenges identifying and locating non-visual items on touch screen interfaces, such as phones and tablets. Various specifications for developing a user interface for people with visual impairments must be met, such as having touch screen division to enable people with blindness to easily and comfortably locate objects and items that are non-visual on the screen ( Khusro et al., 2019 ). Article ( Khusro et al., 2019 ) highlighted the importance of aspects of the usability analysis, such as screen partitioning, to meet specific usability requirements, including orientation, consistency, operation, time consumption, and navigation complexity when users want to locate objects on their touchscreen. The authors of Khan & Khusro (2019) describe the improvements that people with blindness have experienced in using the smartphone while performing their daily tasks. This information was determined through an empirical study with 41 people with blindness who explained their user and interaction experiences operating a smartphone.

The authors in Palani et al. (2018) provide design guidelines governing the accurate display of haptically perceived graphical materials. Determining the usability parameters and the various cognitive abilities required for optimum and accurate use of device interfaces is crucial. Also the authors of Grussenmeyer & Folmer (2017) highlight the importance of usability and accessibility of smartphones and touch screens for people with visual impairments. The primary focus in Leporini & Palmucci (2018) is on interactive tasks used to finish exercises and to answer questionnaires or quizzes. These tools are used for evaluation tests or in games. When using gestures and screen readers to interact on a mobile device, difficulties may arise ( Leporini & Palmucci, 2018 ), The study has various objectives, including gathering information on the difficulties encountered by people with blindness during interactions with mobile touch screen devices to answer questions and investigating practicable solutions to solve the detected accessibility and usability issues. A mobile app with an educational game was used to apply the proposed approach. Moreover, in Alnfiai & Sampalli (2016) and Niazi et al. (2016) , an analysis of the single-tap braille keyboard created to help people with no or low vision while using touch screen smartphones was conducted. The technology used in Alnfiai & Sampalli (2016) was the talkback service, which provides the user with verbal feedback from the application, allowing users with blindness to key in characters according to braille patterns. To evaluate single tap braille, it was compared to the commonly used QWERTY keyboard. In Niazi et al. (2016) , it was found that participants adapted quickly to single-tap Braille and were able to type on the touch screen within 15 to 20 min of being introduced to this system. The main advantage of single tap braille is that it allows users with blindness to enter letters based on braille coding, which they are already familiar with. The average error rate is lower using single-tap Braille than it is on the QWERTY keyboard. The authors of Niazi et al. (2016) found that minimal typing errors were made using the proposed keypad, which made it an easier option for people with blindness ( Niazi et al., 2016 ). In Cho & Kim (2017) , the authors describe new text entry methods for the braille system including a left touch and a double touch scheme that form a two-button interface for braille input so that people with visual impairments are able to type textual characters without having to move their fingers to locate the target buttons.

One of the main problems affecting the visually impaired is limited mobility for some gestures. We need to know what gestures are usable by people with visual impairments. Moreover, the technology of assistive touchscreen-enabled speech interaction between blind people and mobile devices permits the use of gestures to interact with a touch user interface. Assistive technology is vital in helping people living with disabilities to perform actions or interact with systems. Smaradottir, Martinez & Haland (2017) analyses a voiceover screen reader used in Apple Inc.’s products. An assessment of this assistive technology was conducted with six visually impaired test participants. The main objectives were to pinpoint the difficulties related to the performance of gestures applicable in screen interactions and to analyze the system’s response to the gestures. In this study, a user evaluation was completed in three phases. The first phase entailed training users regarding different hand gestures, the second phase was carried out in a usability laboratory where participants were familiarized with technological devices, and the third phase required participants to solve different tasks. In Knutas et al. (2015) , the vital feature of the system is that it enables the user to interactively select a 3D scene region for sonification by merely touching the phone screen. It uses three different modes to increase usability. Alnfiai & Sampalli (2017) explained a study done to compare the use of two data input methods to evaluate their efficiency with completely blind participants who had prior knowledge of braille. The comparison was made between the braille enter input method that uses gestures and the swift braille keyboard, which necessitates finding six buttons representing braille dots. Blind people typically prefer rounded shapes to angular ones when performing complex gestures, as they experience difficulties performing straight gestures with right angles. Participants highlighted that they experienced difficulties particularly with gestures that have steep or right angles. In Buzzi et al. (2017) , 36 visually impaired participants were selected and split into two groups of low-vision and blind people. They examined their touch-based gesture preferences in terms of the number of strokes, multitouch, and shape angles. For this reason, a wireless system was created to record sample gestures from various participants simultaneously while monitoring the capture process.

People with visual impairment typically cannot travel without guidance due to the inaccuracy of current navigation systems in describing roads and especially sidewalks. Thus, the author of Gintner et al. (2017) aims to design a system to guide people with visual impairments based on geographical features and addresses them through a user interface that converts text to audio using a built-in voiceover engine (Apple iOS). The system was evaluated positively in terms of accessibility and usability as tested in a qualitative study involving six participants with visual impairment.

Based on challenges faced by visually impaired game developers, Ara’ujo et al. (2017) provides guidance for developers to provide accessibility in digital games by using audio guidance for players with visual impairments. The interactions of the player can be conveyed through audio and other basic mobile device components with criteria focused on the game level and speed adjustments, high contrast interfaces, accessible menus, and friendly design. Without braille, people with visual impairments cannot read, but braille is expensive and takes effort, and so it is important to propose technology to facilitate reading for them. In Sabab & Ashmafee (2016) , the author proposed developing a mobile application called “Blind Reader” that reads an audio document and allows the user to interact with the application to gain knowledge. This application was evaluated with 11 participants, and the participants were satisfied with the application. Videos are an important form of digital media, and unfortunately people with visual impairment cannot access these videos. Therefore, Façanha et al. (2016) aims to discover sound synthesis techniques to maximize and accelerate the production of audio descriptions with low-cost phonetic description tools. This tool has been evaluated based on usability with eight people and resulted in a high acceptance rate among users.

1- Indoor navigation

Visually impaired people face critical problems when navigating from one place to another. Whether indoors or outdoors, they tend to stay in one place to avoid the risk of injury or seek the help of a sighted person before moving ( Al-Khalifa & Al-Razgan, 2016 ). Thus, aid in navigation is essential for those individuals. In Nair et al. (2020) , Nair developed an application called ASSIST, which leverages Bluetooth low energy (BLE) beacons and augmented reality (AR) to help visually impaired people move around cluttered indoor places ( e.g. , subways) and provide the needed safe guidance, just like having a sighted person lead the way. In the subway example, the beacons will be distributed across the halls of the subway and the application will detect them. Sensors and cameras attached to the individual will detect their exact location and send the data to the application. The application will then give a sequence of audio feedback explaining how to move around the place to reach a specific point ( e.g. , “in 50 ft turn right”, “now turn left”, “you will reach the destination in 20 steps”). The application also has an interface for sighted and low-vision users that shows the next steps and instructions. A usability study was conducted to test different aspects of the proposed solution. The majority of the participants agreed that they could easily reach a specified location using the application without the help of a sighted person. A survey conducted to give suggestions from the participants for future improvements showed that most participants wanted to attach their phones to their bodies and for the application to consider the different walking speeds of users. They were happy with the audio and vibration feedback that was given before each step or turn they had to take.

In Al-Khalifa & Al-Razgan (2016) , the main purpose of the study was to provide an Arabic-language application for guidance inside buildings using Google Glass and an associated mobile application. First, the building plan must be set by a sighted person who configures the different locations needed. Ebsar will ask the map builder to mark each interesting location with a QR code and generate a room number, and the required steps and turns are tracked using the mobile device’s built-in compass and accelerometer features. All of these are recorded in the application for the use of a visually impaired individual, and at the end, a full map is generated for the building. After setting the building map, a user can navigate inside the building with the help of Ebsar, paired with Google Glass, for input and output purposes. The efficiency, effectiveness, and levels of user satisfaction with this solution were evaluated. The results showed that the errors made were few, indicating that Ebsar is highly effective. The time consumed in performing tasks ranged from medium to low depending on the task; this can be improved later. Interviews with participants indicated the application’s ease of use. De Borba Campos et al. (2015) shows an application simulating a museum map for people with visual impairments. It discusses whether mental maps and interactive games can be used by people with visual impairments to recognize the space around them. After multiple usability evaluation sessions, the mobile application showed high efficiency among participants in understanding the museum’s map without repeating the visitation. The authors make a few suggestions based on feedback from the participants regarding enhancing usability, including using audio cues, adding contextual help to realise the activities carried around in a space, and focusing on audio feedback instead of graphics.

2- Outdoor navigation

Outdoor navigation is also commonly discussed in the literature. In Darvishy et al. (2020) , Touch Explorer was presented to alleviate many of the problems faced by visually impaired people in navigation by developing a non-visual mobile digital map. The application relies on three major methods of communication with the user: voice output, vibration feedback, and everyday sounds. The prototype was developed using simple abstract visuals and mostly relies on voice for explanation of the content. Usability tests show the great impact the prototype had on the understanding of the elements of the map. Few suggestions were given by the participants to increase usability, including GPS localization to locate the user on the map, a scale element for measuring the distance between two map elements, and an address search function.

In Hossain, Qaiduzzaman & Rahman (2020) , a navigation application called Sightless Helper was developed to provide a safe navigation method for people with visual impairments. It relies on footstep counting and GPS location to provide the needed guidance. It can also ensure safe navigation by detect objects and unsafe areas and can detect unusual shaking of the user and alert an emergency contact about the problem. The user interaction categories are voice recognition, touchpad, buttons, and shaking sensors. After multiple evaluations, the application was found to be useful in different scenarios and was considered usable by people with visual impairments. The authors in Long et al. (2016) propose an application that uses both updates from users and information about the real world to help visually impaired people navigate outdoor settings. After interviews with participants, some design goals were set, including the ability to tag an obstacle on the map, check the weather, and provide an emergency service. The application was evaluated and was found to be of great benefit; users made few errors and found it easy to use. In Prerana et al. (2019) , a mobile application called STAVI was presented to help visually impaired people navigate from a source to a destination safely and avoid issues of re-routing. The application depends on voice commands and voice output. The application also has additional features, such as calling, messages, and emergency help. The authors in Bandukda et al. (2020) helped people with visual impairments explore parks and natural spaces using a framework called PLACES. Different interviews and surveys were conducted to identify the issues visually impaired people face when they want to do any leisure activity. These were considered in the development of the framework, and some design directions were presented, such as the use of audio to share an experience.

3- General issues

The authors in Maly et al. (2015) discuss implementing an evaluation model to assess the usability of a navigation application and to understand the issues of communication with mobile applications that people with visual impairments face. The evaluation tool was designed using a client–server architecture and was applied to test the usability of an existing navigation application. The tool was successful in capturing many issues related to navigation and user behavior, especially the issue of different timing between the actual voice instruction and the position of the user. The authors in Kameswaran et al. (2020) conducted a study to find out which navigation technologies blind people can use and to understand the complementarity between navigation technologies and their impact on navigation for visually impaired users. The results of the study show that visually impaired people use both assistive technologies and those designed for non-visually impaired users. Improving voice agents in navigation applications was discussed as a design implication for the visually impaired. In Skulimowski et al. (2019) , the authors show how interactive sonification can be used in simple travel aids for the blind. It uses depth images and a histogram called U-depth, which is simple auditory representations for blind users. The vital feature of this system is that it enables the user to interactively select a 3D scene region for sonification by touching the phone screen. This sonic representation of 3D scenes allows users to identify the environment’s general appearance and determine objects’ distance. The prototype structure was tested by three blind individuals who successfully performed the indoor task. Among the test scenes used included walking along an empty corridor, walking along a corridor with obstacles, and locating an opening between obstacles. However, the results showed that it took a long time for the testers to locate narrow spaces between obstacles.

RQ6: What evaluation methods were used in the studies on usability for visually impaired people that were reviewed?

The most prevalent methods to evaluate the usability of applications were surveys and interviews. These were used to determine the usability of the proposed solutions and obtain feedback and suggestions regarding additional features needed to enhance the usability from the participants’ points of view. Focus groups were also used extensively in the literature. Many of the participants selected were blindfolded and were not actually blind or visually impaired. Moreover, the samples selected for the evaluation methods mentioned above considered the age factor depending on the study’s needs.

Limitation and future work

The limitations of this paper are mainly related to the methodology followed. Focusing on just eight online databases and restricting the search with the previously specified keywords and string may have limited the number of search results. Additionally, a large number of papers were excluded because they were written in other languages. Access limitations were also faced due to some libraries asking for fees to access the papers. Therefore, for future works, a study to expand on the SLR results and reveal the current usability models of mobile applications for the visually impaired to verify the SLR results is needed so that this work contributes positively to assessing difficulties and expanding the field of usability of mobile applications for users with visual impairments.

Conclusions

In recent years, the number of applications focused on people with visual impairments has grown, which has led to positive enhancements in those people’s lives, especially if they do not have people around to assist them. In this paper, the research papers focusing on usability for visually impaired users were analyzed and classified into seven themes: accessibility, daily activities, assistive devices, gestures, navigation, screen division layout, and audio guidance. We found that various research studies focus on accessibility of mobile applications to ensure that the same user experience is available to all users, regardless of their abilities. We found many studies that focus on how the design of the applications can assist in performing daily life activities like braille-based application studies and applications to enhance the independence of VI users. We also found papers that discuss the role of assistive devices like screen readers and wearable devices in solving challenges faced by VI users and thus improving their quality of life. We also found that some research papers discuss limited mobility of some gestures for VI users and investigated ways in which we can know what gestures are usable by people with visual impairments. We found many research papers that focus on improving navigation for VI users by incorporating different output modalities like sound and vibration. We also found various studies focusing on screen division layout. By dividing the screen and focusing on visual impairment-related issues while developing user interfaces, visually impaired users can easily locate the objects and items on the screens. Finally, we found papers that focus on audio guidance to improve usability. The proposed applications use voice-over and speech interactions to guide visually impaired users in performing different activities through their mobiles. Most of the researchers focused on usability in different applications and evaluated the usability issues of these applications with visually impaired participants. Some of the studies included sighted participants to compare the number and type of problems they faced. The usability evaluation was generally based on the following criteria: accessibility, efficiency, learnability, memorability, errors, safety, and satisfaction. Many of the studied applications show a good indication of these applications’ usability and follow the participants’ comments to ensure additional enhancements in usability. This paper aims to provide an overview of the developments on usability of mobile applications for people with visual impairments and use this overview to highlight potential future directions.

References selected for each theme.

Funding Statement

This research project was supported by a grant from the Research Center of the Female Scientific and Medical Colleges, Deanship of Scientific Research, King Saud University. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Additional Information and Declarations

The authors declare there are no competing interests.

Muna Al-Razgan , Sarah Almoaiqel , Nuha Alrajhi , Alyah Alhumegani , Abeer Alshehri , Bashayr Alnefaie , Raghad AlKhamiss and Shahad Rushdi conceived and designed the experiments, performed the experiments, analyzed the data, performed the computation work, prepared figures and/or tables, authored or reviewed drafts of the paper, and approved the final draft.

IEEE Account

• Change Username/Password
• Update Address

Purchase Details

• Payment Options
• Order History
• View Purchased Documents

Profile Information

• Communications Preferences
• Profession and Education
• Technical Interests
• US & Canada: +1 800 678 4333
• Worldwide: +1 732 981 0060
• Contact & Support
• About IEEE Xplore
• Accessibility
• Terms of Use
• Nondiscrimination Policy
• Privacy & Opting Out of Cookies

A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity. © Copyright 2024 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions.

Usability of mobile applications: Literature review and rationale for a new usability model

• January 2013
• Journal of Interaction Science 1(1)

• Oxford Brookes University

• Dundalk Institute of Technology

Discover the world's research

• 25+ million members
• 160+ million publication pages
• 2.3+ billion citations
• Edgar Schero

• Irene Martorelli
• Aram Pooryousefi
• Haike van Thiel

• Marc Gonzalez-Capdevila

• Seonyeong Mun
• Myeong-Kyeong Shin

• Sven Heitmann
• Alexia Pagotto
• Christian Kray

• Sarah Binte Mahbub
• Nafis Islam
• Mehedi Hasan Surem
• Ashraful Islam
• Linda Theresia
• Mega Bagus Herlambang
• Ni Made Sudri
• Yenny Widianty

• Steve Benford
• Mark Paxton
• Adrain Woolard

• L.E. Staveland

• SOFTWARE PRACT EXPER
• David Budgen
• Andy J. Burn
• O. Pearl Brereton
• Rialette Pretorius

• Recruit researchers
• Join for free
• Login Email Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google Welcome back! Please log in. Email · Hint Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google No account? Sign up

• Corpus ID: 189808581

Literature Review : Starting Mobile Application Development for E-Sports Portal

• Hayoung Noh
• Published 2014
• Computer Science

Tables from this paper

4 Citations

Mobile application development: an agile approach, enhancing user-engagement in the design process through augmented reality applications conducting an experiment to measure effectiveness of mobile augmented reality user interfaces, aeon: a smart medicine delivery and inventory system for cebu city government’s long life medical assistance program, augmented reality in the design process using visual effects ( vfx ) motion tracking techniques to conduct quantification research on the performance of augmented reality, 12 references, mobile web applications: bringing mobile apps and web together, html5 – bridging the mobile platform gap: mobile technologies in scholarly communication, developing design guidelines for context-aware mobile applications, software engineering issues for mobile application development.

• Highly Influential

Evaluation of mobile app paradigms

Mobile application development, a cloud on the horizon: the challenge of developing applications for android and iphone, understanding mobile web and mobile search use in today's dynamic mobile landscape, smart smartphone development: ios versus android, android vs windows mobile vs java me: a comparative study of mobile development environments, related papers.

Showing 1 through 3 of 0 Related Papers

Academia.edu no longer supports Internet Explorer.

To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to  upgrade your browser .

Enter the email address you signed up with and we'll email you a reset link.

• We're Hiring!
• Help Center

Literature Review On Mobile Application Development Effort Estimation and Sizing Measurement

Mobile Application are new emerging technology dominating the software engineering platform, this new technology comes with new features, restrictions and new possibilities which does not exist before. Building capable software for this new environment comes with new requirement, constrains and characteristics that are not used in the previous traditional estimation method. In this paper we analyses the identified characteristics that directly interfere with mobile application development. We also analyzed the most common current estimation method that has been used to measure mobile application effort and size and we propose Objective for future estimation method that should be able to measure mobile application development effort and sizing measurement.

Related Papers

IOSR Journals

A Mobile phone has evolved from being a voice communication system to a medium for technology. Mobile applications are a kind of software that is installed on a mobile device with some important differences than traditional software application and web applications. Most of the organizations have switched their web application based software's to a mobile application based software's. With the growth of smart phones there is a great demand for smart applications. For software companies it is important to deliver application software's on time, within budget and with high accuracy. Effort estimation is the fundamental area that chooses budgetary constraints related to mobile application development which keeps company to maintain accurate estimates for mobile application development to maintain their reputation in the market. In this paper, different reviews are made clear to propose a way cosmic an appropriate method that can be used to size mobile application in a fast and accurate way.

Computer Science & Information Technology (CS & IT) Computer Science Conference Proceedings (CSCP)

The rise of the use of mobile technologies in the world, such as smartphones and tablets, connected to mobile networks is changing old habits and creating new ways for the society to access information and interact with computer systems. Thus, traditional information systems are undergoing a process of adaptation to this new computing context. However, it is important to note that the characteristics of this new context are different. There are new features and, thereafter, new possibilities, as well as restrictions that did not exist before. Finally, the systems developed for this environment have different requirements and characteristics than the traditional information systems. For this reason, there is the need to reassess the current knowledge about the processes of planning and building for the development of systems in this new environment. One area in particular that demands such adaptation is software estimation. The estimation processes, in general, are based on characteristics of the systems, trying to quantify the complexity of implementing them. Hence, the main objective of this paper is to present a proposal for an estimation model for mobile applications, as well as discuss the applicability of traditional estimation models for the purpose of developing systems in the context of mobile computing. Hence, the main objective of this paper is to present an effort estimation model for mobile applications.

Eric Githua

Software sizing is an activity in software engineering that is used to estimate the size of a software project in order to be able to apply other software project activities. Accurate software project estimation is determined by the degree to which the software managers have correctly estimated the size of the software. Accurate sizing estimation is an important measurement in the calculation of estimated project costs, effort, schedules and duration which provides important information for software project development. Estimation in software projects can be carried out by first measuring the size of the product to be developed. This paper analyses and provides a details overview of two most common software sizing metrics; Lines of codes (LOC) and Function Point Analysis (FPA). Their strengths and weaknesses are also examined; the ‘second generation’ software sizing methods are also discussed. The paper also presents remarks on the findings and future research of the software sizing methods.

Davide Taibi , Luigi Lavazza , Valentina Lenarduzzi

—Background. Function Point Analysis is the most used technique for sizing software functional specifications. Function Point measures are widely used to estimate the effort needed to develop software, hence the cost of software. However , Function Point Analysis adopts the point of view of the end user, and –consistently– considers a software application as a whole. This approach does not allow for assessing the role of reusable components in software development. In fact, reusing available components decreases the cost of software development, but standard Function Point measures are not able to account for the savings deriving from component reuse. Objective. We aim at modifying the definition of Function Point Analysis so that the role of components can be taken into account. More specifically, we redefine the measurement so that when no components are used the resulting measure is the same yielded by the standard measurement process, but in presence of components, our modified measure is less than the standard measure (the bigger the role of components, the smaller the measure). Method. Components partly support the realization of elementary processes. Therefore, we split elementary processes into sub-processes, such that each sub-process is either totally supported by a component or it is not supported at all by any component; the size of the elementary process is defined to be inversely proportional to the size of sub-processes supported by components. Results. The proposed approach was applied to a Web application , which was developed in two versions: one from scratch and one using available components. As expected, the 'component-aware' measures obtained are smaller than the standard measures. We also compared the reduction in size with the reduction in development effort. Conclusions. The proposed method proved effective in taking into account the usage of components in the development of the considered application. However, the observed decrease in size is smaller than the decrease of development effort. The latter result suggests that this initial proposal needs further experimentation to support accurate effort estimation.

Proceedings of the International Conferences on Software Process and Product Measurement

Olga Ormandjieva

The increased awareness of the non-functional requirements as a key to software project and product success makes explicit the need to include them in any software project effort estimation activity. However, the existing approaches to defining size-based effort relationships still pay insufficient attention to this need. This paper presents a flexible, yet systematic approach to the early requirements-based effort estimation, based on Non-Functional Requirements ontology. It complementarily uses one standard functional size measurement model and a linear regression technique. We report on a case study which illustrates the application of our solution approach in context and also helps evaluate our experiences in using it.

Lecture Notes in Computer Science

Alain Abran

Mohamad Kassab

Science Park Research Organization & Counselling

Functional size measurement is a very powerful tool for information technology practitioners since it's output is an invaluable information and used for several purposes. For instance functional size is vital in measurement of productivity and quality. It's also an important indicator in software projects cost estimation. However usage of functional sizing methodologies is not widespread in software industry and rollout within companies is still a big challange. This study focuses on the selection process of a functional sizing methodology in the context of a telecommunications company. It investigates the factors influencing sizing methodology selection decision and experienced issues in the course of proof of concept project. A case study was conducted measuring more than 40 projects from a Turkish telecommunications company using COSMIC and IFPUG methodologies as two candidates. After evaluations, the selected method within company was COSMIC.

Davide Taibi , Valentina Lenarduzzi

—In SCRUM projects, effort estimations are carried out at the beginning of each sprint, usually based on story points. The usage of functional size measures, specifically selected for the type of application and development conditions, is expected to allow for more accurate effort estimates. The goal of the work presented here is to verify this hypothesis, based on experimental data. The association of story measures to actual effort and the accuracy of the resulting effort model was evaluated. The study shows that developers' estimation is more accurate than those based on functional measurement. In conclusion, our study show that, easy to collect functional measures do not help developers in improving the accuracy of the effort estimation in Moonlight SCRUM.

Loading Preview

Sorry, preview is currently unavailable. You can download the paper by clicking the button above.

RELATED PAPERS

IWSM Mensura

Colin Hammond

2010 Eighth ACIS International Conference on Software Engineering Research, Management and Applications

Frank Vogelezang

Conference: 2nd International Conference on Information Society, Technology and Management, ICIST 2012, At Kopaonik, Serbia

Sonja Dimitrijevic

Luigi Lavazza

RELATED TOPICS

•   We're Hiring!
•   Help Center
• Find new research papers in:
• Health Sciences
• Earth Sciences
• Cognitive Science
• Mathematics
• Computer Science
• Academia ©2024

• Open access
• Published: 31 July 2024

Ozone therapy in musculoskeletal medicine: a comprehensive review

• Madhan Jeyaraman   ORCID: orcid.org/0000-0002-9045-9493 1 ,
• Naveen Jeyaraman   ORCID: orcid.org/0000-0002-4362-3326 1 ,
• Swaminathan Ramasubramanian   ORCID: orcid.org/0000-0001-8845-8427 2 ,
• Sangeetha Balaji   ORCID: orcid.org/0000-0002-1566-1333 2 ,
• Arulkumar Nallakumarasamy   ORCID: orcid.org/0000-0002-2445-2883 3 ,
• Bishnu Prasad Patro   ORCID: orcid.org/0000-0001-9497-9624 4 &
• Filippo Migliorini   ORCID: orcid.org/0000-0001-7220-1221 5 , 6 , 7  

European Journal of Medical Research volume  29 , Article number:  398 ( 2024 ) Cite this article

112 Accesses

1 Altmetric

Metrics details

Musculoskeletal disorders encompass a wide range of conditions that impact the bones, joints, muscles, and connective tissues within the body. Despite the ongoing debate on toxicity and administration, ozone demonstrated promise in managing several musculoskeletal disorders, modulating pain and inflammation. A literature search was conducted. The research design, methods, findings, and conclusions of the studies were then examined to evaluate the physiological effects, clinical application, controversies, and safety of the application of ozone in musculoskeletal medicine. Ozone application demonstrates considerable therapeutic applications in the management of musculoskeletal disorders, including fractures, osteoarthritis, and chronic pain syndromes. Despite these advantages, studies have raised concerns regarding its potential toxicity and emphasized the importance of adhering to stringent administration protocols to ensure safety. Additionally, heterogeneities in patient reactions and hazards from oxidizing agents were observed. Given its anti-inflammatory and analgesic qualities, ozone therapy holds potential in the management of several musculoskeletal disorders. Additional high-quality research with long follow-up is required to refine indications, efficacy and safety profile. Finally, for wider clinical acceptability and utilization, the development of international recommendations is essential.

Introduction

Ozone therapy is an innovative modality which gained growing attention in medical science, especially in musculoskeletal medicine [ 1 ]. Administered as a mixture of oxygen and ozone gases, it stimulates physiological responses, particularly anti-inflammatory and analgesic effects [ 2 , 3 ]. By doing so, ozone therapy is administered in various acute and chronic musculoskeletal disorders such as fractures [ 4 , 5 , 6 ], osteoarthritis (OA) [ 7 , 8 , 9 , 10 ], low back ache [ 11 , 12 ], osteomyelitis [ 13 , 14 , 15 , 16 ], and chronic pain syndromes [ 17 , 18 , 19 ]. Beyond orthopaedic applications, ozone therapy has shown potential in treating conditions such as chronic fatigue syndrome (CFS) and myalgic encephalomyelitis (ME) using different modalities such as ozonated water, ozone oil, and ozone gas [ 20 , 21 , 22 , 23 , 24 ]. Tailored routes of administration, ranging from enteral ozonated oil [ingestion], enteral ozonated water [ingestion], parenteral oxygen/ozone gas mixture [intramuscular, subcutaneous, intratonsillar, intrathecal, intraperitoneal, periganglionic, intraforaminal, paravertebral, intra-articular, intradiscal, oral submucous, supralaminar, epidural, penile, and intravenous], parenteral ozonated water [intra-tumoral], systemic [minor autohemotherapy (MiAH), major autohemotherapy (MAH), extracorporeal blood oxygenation and ozonation (EBOO), intravenous, and rectal insufflation], topical ozonated water [wound wash, ozonized balneotherapy, mouth wash, irrigation (nasal, sinus, otological, vaginal, intrauterine, intestinal, and intravesical), and sauna therapy, topical oxygen/ozone gas mixture [insufflation (ontological, intrafistula, intrauterine, and intravesical) and hyperbaric bagging], topical ozonated oil [embrocation and inhalation], and topical ozonated saline solution [wound wash, mouth wash, irrigation (saline, ontological, vaginal, intrauterine, and intravesical)], highlight its versatility [ 18 , 25 , 26 , 27 ]. In orthopaedic practice, intramuscular, subcutaneous, intrathecal, periganglionic, intraforaminal, paravertebral, intra-articular, intradiscal, oral submucous, supralaminar, epidural, intravenous routes are used. However, besides its therapeutic potential, concerns about ozone toxicity and the importance of adhering to safe administration protocols cannot be overlooked [ 21 , 25 , 26 , 28 ]. Prolonged inhalation of tropospheric ozone has been linked to detrimental impacts on the respiratory system and vital organs, inducing chronic oxidative stress and inflammation across multiple organs [ 29 ]. However, in controlled therapeutic settings, ozone therapy has demonstrated calculated oxidative stress induction, which activates therapeutic benefits without resulting in acute or chronic toxicity [ 29 ]. Caution is warranted to ensure that ozone doses do not surpass the blood’s antioxidant capacity to prevent potential toxicity [ 30 ]. While ozone therapy has the potential to enhance erythrocyte characteristics, correct hypoxia in diseases, and increase ATP levels through glycolysis activation, precise application is essential to avoid toxicity concerns [ 30 ]. The existing evidence is divided between its clinical benefits and potential risks as an oxidizing agent, further nuanced by variability in patient responses [ 28 ]. Despite its environmental implications as a pollutant, when administered under controlled conditions, ozone therapeutic potential in musculoskeletal conditions, including OA and herniated discs, is documented [ 20 , 25 , 26 ]. Therefore, ozone therapy emerges as a potent yet intricate intervention, necessitating further in-depth scientific scrutiny.

Numerous studies have explored ozone therapy, yet discrepancies persist, yielding variable results. Existing literature on ozone therapy reveals research gaps: limited comparative studies between treatments, unclear safety and effectiveness in various musculoskeletal conditions and insufficient understanding of anti-inflammatory effects [ 3 , 18 , 31 ]. Reviews frequently concentrate on specialized areas, prompting the necessity for a comprehensive review to address lacunae. In light of these controversies, this review critically evaluates the current evidence on the utilization of ozone in musculoskeletal disorders.

Effects of ozone

Ozone therapy, with its multifaceted therapeutic capacities, has demonstrated notable physiological effects across various medical arenas. Its primary mechanism centres on enhancing tissue oxygenation, an indispensable component of cellular metabolism, by augmenting oxygen delivery to hypoxic tissues, thereby fostering optimal conditions for metabolic and repair processes [ 20 , 26 , 32 ]. For systemic administration of ozone, such as in major autohemotherapy (MAH), the concentration range should fall within 10–40 μg/mL, with 10–20 μg/mL of blood considered biologically relevant. It is advisable to administer a total ozone amount of 500–1000 μg per 50 mL of blood during MAH treatment. This induces the production of pharmacologically active hydroxy hydroperoxides, referred to as “ozone peroxides” [ 15 , 25 ]. These compounds play a pivotal role in modulating endogenous antioxidant systems, particularly through interactions with the glutathione system [ 15 ]. A salient feature of ozone therapy is its proficiency in managing oxidative stress (Fig.  1 ), amplifying cellular antioxidant mechanisms while tempering the formation of reactive oxygen species (ROS) [ 3 , 33 ]. Specifically, ozone engages with cysteine residues and glutathione, catalysing signal processes that support the bioregulation of enzymatic antioxidants [ 15 ]. This aptitude for oxidative stress modulation is important in conditions marked by persistent inflammation and immune system anomalies [ 3 , 17 ]. Notably, ozone moderates inflammation by curtailing pro-inflammatory cytokines, such as interleukin 6 (IL-6), and simultaneously promotes anabolic activity, as evident by elevated insulin-like growth factor 1 (IGF-1) levels [ 33 , 34 ]. In addition, ozone therapy exerts a regulatory effect on key transcription factors: it activates Nrf2, endorsing antioxidant and anti-inflammatory responses, while concurrently inhibiting NF-kB, integral in inflammation regulation [ 18 , 33 , 35 , 36 ]. The therapy also modulates the synthesis of prostaglandins, impacts bradykinin release [ 26 ], and optimizes immune functions, particularly enhancing the activity of macrophages pivotal to immune reactions [ 3 , 17 ]. From a tissue repair and angiogenesis standpoint, ozone therapy stimulates the expression of proteins such as collagen I, α-SMA, and TGF-β1, facilitating fibroblast activity [ 22 ]. It also upregulates growth factors, including vascular endothelial growth factor (VEGF) and transforming growth factor-beta (TGF-β), fostering enhanced tissue regeneration [ 23 , 33 ]. Moreover, the activation of cellular signalling pathways like the PI3K/Akt/mTOR axis underscores its role in promoting the epithelial–mesenchymal transition (EMT) process [ 22 , 37 ].

Physiological effects of ozone

Clinically, ozone therapy has been effective in pain management, addressing conditions such as low back pain [ 18 ] and rotator cuff calcific tendinitis [ 38 ] (Fig. 2 ). Its influence extends to ameliorating microcirculatory disturbances seen in bone necrosis [ 1 ] and positively adjusting serum physiological markers, including total antioxidant capacity and lactate dehydrogenase [ 14 ], alongside inflammatory markers like the erythrocyte sedimentation rate [ 13 ]. The nuanced mechanisms of action underlying ozone therapy, ranging from improved oxygenation and oxidative stress modulation to anti-inflammatory effects, immune system enhancements, and tissue repair, validate its rising interest as a potential adjunctive treatment for several medical conditions. Its multifaceted physiological interactions render it an asset in contemporary clinical practice.

Applications of metamizole in musculoskeletal medicine

Clinical applications of ozone in orthopaedics

Fracture non-union posed challenges in orthopaedic medicine, prompting the exploration of novel approaches [ 39 ]. In this context, ozone therapy garnered attention for its potential to enhance bone healing. Experimental studies using animal models, especially those with critical size defects, have demonstrated its efficacy. Duman et al. documented improved aspects of bone healing in a rat femoral fracture model with ozone therapy, noting enhanced union, bone formation, and bone marrow organization [ 40 ]. These findings align with those of Irban et al., who observed increased periosteal thickness and trabecular formation, thereby solidifying ozone therapy prospective role in bone regeneration [ 41 ]. These preclinical insights are bolstered by clinical evidence underscoring the therapy benefits. Shah et al. detailed a case wherein ozone therapy expedited wound healing and infection resolution over a tibial area, resulting in the patient regained ability to walk within 20 months [ 42 ]. Beyond fracture healing, ozone therapy has demonstrated effectiveness in pain management. Sconza et al. identified a comparable efficacy between ozone therapy and hyaluronic acid injections for knee OA pain alleviation and functional improvement [ 8 ]. Additionally, in comparison with low-level laser therapy, ozone therapy evidenced superior performance in promoting bone formation in rat calvarial defects [ 5 ]. Irban et al. conducted a comprehensive examination revealing ozone therapy profound impact on bone healing. Beyond the observed increase in periosteal thickness and trabecular areas, there was a marked elevation in vital growth factors and proteins, such as VEGF, β-catenin, and TGF-β, in the ozone-administered group versus the control [ 41 ]. Nevertheless, while the existing evidence is promising, it is crucial to address the current methodological limitations. Bennett et al. systematic review highlighted concerns regarding the quality of studies on the prevention or treatment of fracture non-union, pinpointing issues like lack of method standardization, inconsistent outcome measures, and inadequate reporting, which currently hampers the feasibility of meta-analyses [ 4 ].

Osteoarthritis

The advent of ozone therapy modalities, specifically O 2 –O 3 therapy and oxygen–ozone therapy (OOT), has significantly enriched the therapeutic spectrum for knee OA. These interventions, recognized for their ability to modulate oxidative stress and suppress pro-inflammatory cytokines, offer pain relief and functional enhancement in patients with knee OA [ 3 , 43 , 44 ]. Ozone therapy yields outcomes in pain, stiffness, and overall function comparable to the traditionally favoured hyaluronic acid injections [ 8 ]. Fernandez et al. observed its effectiveness through intra-articular injections for knee OA, resulting in symptomatic relief and improved physical function [ 33 ]. Comparative trials, such as those juxtaposing the efficacy of ozone prolotherapy and intra-articular hypertonic saline, found both modalities to be therapeutically analogous [ 45 ]. Periodic intra-articular ozone injections have been advocated for their consistent pain amelioration, joint function enhancement, and overall quality of life improvement, alongside a commendable safety profile [ 10 ]. These favourable findings merit a balanced perspective. A systematic review of randomized controlled trials (RCTs) pinpointed the generally subpar methodological quality in the prevailing research on this topic [ 4 ]. A previous review on OOT for knee OA reinforced this finding, suggesting that while short to medium-term results were encouraging, the included RCTs exhibited discernible methodological constraints [ 8 ]. When juxtaposing ozone therapy with hyaluronic acid injections, both modalities manifested significant advancements in pain, stiffness, and function across various time points, including 1, 3, and 6 months of follow-ups [ 46 ]. The anti-inflammatory and anabolic attributes of ozone therapy have garnered notable attention. Fernandez-Cuadros et al. documented reductions in inflammatory markers such as interleukin-6 (IL-6), C-reactive protein (CRP), erythrocyte sedimentation rate (ESR), and uric acid, coupled with pain alleviation and functional improvements spanning all grades of knee OA [ 33 ]. Farpour et al. findings echoed this, revealing that both ozone prolotherapy and intra-articular hypertonic saline injections offered parallel therapeutic benefits [ 45 ]. Moreover, a specific RCT highlighted the potential of weekly ozone injections over 8 weeks, emphasizing discernible pain reduction, improved joint functionality, and enhanced life quality [ 10 ]. While the prevailing literature accentuates the prospective benefits of ozone therapy in knee OA management, it simultaneously underscores methodological inadequacies in the current research landscape. To unequivocally establish ozone therapy standing as an efficacious knee OA treatment, there is an undeniable need for comprehensive, multicentre RCTs with extended follow-ups to corroborate its long-term benefits and reliability.

Osteonecrosis of the femoral head

Ozone therapy has emerged as a promising alternative in treating osteonecrosis of the femoral head, especially for patients unresponsive to traditional therapeutic approaches. The therapy effectiveness is rooted in its capacity to promote neovascularization and immunoregulation, both pivotal for repairing hypoxic-ischaemic necrosis in the femoral head [ 1 , 47 ]. Elucidating the molecular underpinnings of this treatment, An et al. highlighted the influence of differentially expressed genes, suggesting that individualized ozone therapy regimens, tailored to specific molecular markers, could be developed [ 1 ]. Complementary findings by An et al. showcased the augmented benefits of conjoining intra-articular O 2 –O 3 hip injections with O 3 -AHT (autohemotherapy). Such a combination not only ameliorated enduring pain, but also aided in mitigating bone marrow oedema across diverse stages of osteonecrosis of the femoral head. Importantly, differential clinical outcomes between patients responsive and non-responsive to ozone therapy underscored the significance of immunoregulatory pathways, particularly spotlighting the impact of mitotically active lymphocytes in modulating treatment efficacy [ 1 ]. Branching out from osteonecrosis of the femoral head, the therapeutic scope of ozone therapy has been assessed in other conditions like avascular bisphosphonate-induced jaw osteonecrosis [ 48 , 49 , 50 , 51 ]. Agrillo et al. indicated that over half of the participants experienced full lesion healing and symptom abatement, with another 30% witnessing substantial lesion diminution and symptom relief. Even though lesions persisted relatively unchanged in 16% of the patients, they still reported some alleviation in symptoms, suggesting that ozone therapy might be an effective alternative to more invasive interventions, such as bone ablation surgery, particularly for bisphosphonate-induced related jaw osteonecrosis cases [ 51 ].

Osteomyelitis

Ozone therapy potential as a treatment for osteomyelitis, a challenging bone infection often unyielding to conventional modalities, has garnered increasing attention in both clinical and preclinical domains. Yasheng et al. demonstrated the effectiveness of a treatment approach combining ozonated water lavage, physiological saline irrigation, and vacuum-sealed drainage for chronic osteomyelitis. This regimen yielded significant bacterial clearance and stimulated granulation tissue formation, attesting to ozone therapy potential as a reliable intervention [ 16 ]. Further, Shetty et al. highlighted its adaptability in specialized cases, showcasing its efficacy in neonatal mandibular osteomyelitis when combined with intravenous vancomycin antibiotics. The route of ozone administration involves the local application of ozonated water to the lesional site. The ozonated water is prepared by infusing three cycles of ozonated gas into 100 mL of saline over a 24-h p24 hours [ 52 ]. Animal studies bolster these clinical observations. Bilge et al., using a rat osteomyelitis model, noted improved histopathological parameters, suggesting ozone therapy has potent anti-inflammatory attributes [ 14 ]. Focusing on inflammatory markers, Nabi et al. reported the therapy positive impact on indices like the erythrocyte sedimentation rate, documenting a commendable 86.66% recovery in a patient cohort [ 13 ]. This line of exploration aligns with Shah et al. emphasis on ozone inherent disinfectant and antibacterial properties, positioning it as a beneficial option for extensive orthopaedic conditions, including osteomyelitis [ 42 ]. Furthermore, ozone therapy role extends to conditions such as bisphosphonate-related jaw osteonecrosis. Agrillo et al. evidenced complete healing in a significant patient proportion and substantial symptom relief in others [ 51 ]. An RCT delving into ozone therapy for chronic osteomyelitis showed the treatment group exhibiting marginally improved recovery rates and inflammatory indices, without notable adverse events, although the observed difference was not statistically significant [ 13 ].

Adhesive capsulitis

Recent randomized controlled trials have sought to discern the optimal treatment strategy for adhesive capsulitis of the shoulder. Foula et al. compared the therapeutic merits of ultrasound-guided intra-articular injections, specifically contrasting ozone, steroid, and pulsed radiofrequency. The equipment utilized in the study employed platelet-rich fibrin (PRF) generators, which emit oscillating pulses spanning a frequency range of 420–500 kHz. These pulses have an amplitude of 45 V and a duration of 20 ms, followed by a silent period lasting 480 ms. Their results indicated the superior long-term efficacy of pulsed radiofrequency in symptom alleviation compared to its counterparts, ozone and steroid injections [ 53 ]. Another study echoed the benefits of these ultrasound-guided intra-articular injections in treating shoulder adhesive capsulitis. It documented marked improvements in pain, disability, range of motion (ROM), and inflammation across all intervention groups. Notably, visual analogue scale (VAS) scores during movement exhibited significant enhancements from the second week onward, persisting through the fourth and eighth weeks for all interventions. The steroid group, in particular, showed early improvements with VAS scores at rest enhancing from the initial week. ROM and the Shoulder Pain and Disability Index (SPADI) scores also reported progress from the second week. Moreover, consistent reductions in inflammatory markers, intercellular adhesion molecule-1 (ICAM-1), and high-sensitivity C-reactive protein (hs-CRP), were observed across the groups [ 53 ]. While both studies validate the efficacy of ultrasound-guided intra-articular injections in managing adhesive capsulitis symptoms, Foula et al. findings distinctly spotlight pulsed radiofrequency advantage for sustained symptom relief [ 53 ]. Concurrently, the subsequent study provides a more holistic perspective, underscoring the significant short-term benefits of both ozone and steroid injections, in addition to pulsed radiofrequency. Thus, pulsed radiofrequency stands out for its long-term therapeutic potential. Still, all modalities, including ozone and steroids, remain significant contenders in the short-term therapeutic landscape for adhesive capsulitis of the shoulder.

Tendinopathies

Ozone therapy is progressively gaining prominence in managing various musculoskeletal conditions, especially tendinopathies [ 20 , 38 , 54 , 55 , 56 ]. Hidalgo-Tallon et al., spotlighted the therapy efficacy in treating subacromial tendinopathy, illustrating its advantage over mesotherapy and steroid injections [ 2 ]. Complementary research by Dong et al. [ 38 ] and Atar et al. [ 56 ] further substantiated these findings, suggesting ozone therapy potential in managing conditions like rotator cuff calcific tendinitis and chronic supraspinatus tendinopathy. These studies suggest that ozone injections are comparable to corticosteroid injections in alleviating symptoms. Further broadening the scope of ozone therapy, the modality matches corticosteroid injections in delivering significant relief from chronic plantar fasciitis [ 55 ]. This not only accentuates ozone therapy versatility, but also positions it as a potent alternative to established treatments. In the realm of sports medicine, the ozone therapy utility is becoming apparent. Hidalgo-Tallon et al. showcased ozone therapy benefits in managing hamstring injuries among professional athletes, noting pronounced pain reduction and functional improvements [ 57 ]. Seyam et al. highlighted positive outcomes from ultrasound-guided ozone therapy for partial supraspinatus tendon tears [ 20 ]. Furthermore, with an emphasis on compassionate care and treatment-resistant cases, Hidalgo-Tallon et al. proposed ozone therapy as an invaluable tool [ 2 ]. While ongoing research is geared towards elucidating ozone therapy comprehensive efficacy relative to corticosteroid injections in treating rotator cuff calcific tendinitis, definitive outcomes remain anticipated [ 38 ]. Additionally, animal studies such as the one by Gurger et al. spotlight the augmented effect of combining ozone therapy with platelet-rich plasma (PRP) on tendon-to-bone healing. This particular study revealed the duo potential in enhancing biomechanical strength, collagen fibre continuity, and alignment, thus hinting at the possibility of using ozonized PRP as a biological catalyst in tendon healing processes [ 58 ]. Lastly, Atar et al. randomized controlled trial buttressed the comparable therapeutic potency of ozone and corticosteroid injections for chronic supraspinatus tendinopathy in terms of pain relief, life quality, and functionality, though emphasizing the need for extended studies to gauge long-term outcomes [ 56 ].

Rheumatic pathologies

Ozone therapy has emerged as a noteworthy intervention in the realm of rheumatic diseases, driven primarily by its capacities to modulate oxidative stress and attenuate pro-inflammatory cytokines [ 3 ]. Rheumatoid arthritis stands as a significant context where ozone therapy is increasingly being validated for methotrexate (MTX) efficacy for clinical response and improves cellular redox balance [ 59 , 60 , 61 ]. In animal models induced by Freund complete adjuvant and another experimental model, studies by Bozbas et al. and Zhao et al., respectively, observed marked attenuation of symptoms, histopathological signs of inflammation, and reductions in pro-inflammatory cytokines [ 62 , 63 ]. Furthermore, Patel et al. postulate the potential dual application of ozone therapy, both as a primary treatment and synergistically alongside stem cell therapy or natural medicines [ 64 ]. Complementing its standalone utility, ozone therapy demonstrates promise as an adjunctive approach in conventional treatment protocols for rheumatoid arthritis. Fernandez et al. explored its potential combined with methotrexate (MTX), finding enhanced therapeutic outcomes of MTX in rheumatoid arthritis [ 61 ]. This combined approach not only mitigated disease activity, but also reinforced the antioxidant system, accentuating ozone therapeutic complementarity [ 61 ]. Beyond rheumatoid arthritis, the application spectrum of ozone therapy expands to other rheumatic conditions. Studies by Seyam et al. and Tartari et al. also spotlighted ozone therapy role in modulating oxidative damage in systemic sclerosis and its potential applicability in systemic autoimmune rheumatic diseases via cytokine profile adjustments [ 20 , 60 ]. However, nuances exist in the broader application of ozone. Zhao et al. study pointed to a correlation between fine particulate matter (PM2.5) exposure and systemic autoimmune rheumatic diseases, yet did not firmly link ozone exposure with the onset of these diseases [ 63 ]. While another study affirmed ozone therapy benefits in reducing inflammation and arthritis severity in an animal model with rheumatoid arthritis, the alterations in oxidative stress markers remained statistically inconclusive [ 62 ].

Chronic pain syndromes

Ozone therapy has solidified its position as an effective therapeutic modality in managing an array of chronic pain conditions. Particularly, intramuscular injections of the O 2 –O 3 mixture have shown significant efficacy in alleviating cervicobrachial pain, with discernible reductions in VAS pain scores [ 65 , 66 ]. Parallel to this, intramuscular paravertebral lumbar injections of the mixture have proven fruitful in addressing low back pain, as reflected by reductions in VAS scores and enhancements in the Oswestry Disability Index (ODI) scores [ 11 ]. This is further substantiated by studies by Andrade et al. and Biazzo et al., which emphasize the minimally invasive and effective nature of ozone therapy in lumbar pain management [ 11 , 12 ]. A meta-analysis considering ozone therapy role in lumbar pain underscores its therapeutic potential but also signals the necessity for caution due to the high or uncertain risk of bias in some of the assessed trials [ 12 ]. Beyond specific pain syndromes, ozone therapy exhibits broader mechanisms, such as oxygenation, immune modulation, and anti-inflammatory action, and has been applied to other chronic conditions like ME/CFS and CFS [ 17 , 67 , 68 , 69 ]. Furthermore, its efficacy has been observed to be comparable, if not superior, to standard treatments in Pain Units, especially when conventional therapies fall short [ 2 ]. The treatment adaptability was further highlighted in a case report detailing its successful application in an 11-year-old with Complex Regional Pain Syndrome and pseudo-seizures, marking a full remission post-intervention [ 17 ]. Delving into synergistic applications, Patel presented the intriguing prospect of combining ozone therapy with stem cell interventions, particularly for rheumatoid arthritis. However, this proposal is nascent and requires in-depth exploration for both safety and effectiveness [ 64 ]. While ozone therapy therapeutic potential is increasingly acknowledged, there exists a clear mandate for more robust scientific investigation. Tartari et al. emphasize the need for clarity in indications, protocol optimization, and pinpointing the patient demographics most likely to benefit [ 60 ]. Thus, despite the growing advocacy for ozone therapy as a versatile and efficient treatment modality in chronic pain management, its broad clinical application demands further empirical rigour.

Fibromyalgia

Ozone therapy has emerged as a viable therapeutic option in addressing chronic conditions marked by fatigue and musculoskeletal pain, particularly myalgic encephalomyelitis (ME)/chronic fatigue syndrome (CFS) [ 67 , 68 , 69 , 70 ] and fibromyalgia [ 71 , 72 , 73 , 74 ]. In patients with ME/CFS, oxygen–ozone autohemotherapy (O 2 –O 3 -AHT) has been linked to significant clinical advantages. After O 2 –O 3 -AHT, approximately 43.5% of participants experienced a marked reduction in fatigue symptoms, with these benefits persisting for at least 3 months across diverse demographics [ 68 ]. The therapy impact on ME/CFS extends beyond symptom alleviation, showing potential in modulating metabolic pathways, oxidative stress, antioxidant systems, and immune and inflammatory responses [ 67 ]. Simultaneously, for fibromyalgia, a condition which shares many symptomatic parallels with ME/CFS, rectal insufflation ozone therapy has shown significant therapeutic potential [ 66 ]. Hidalgo-Tallon et al. observed substantial improvements in fibromyalgia patients’ physical symptoms, as reflected by decreased Fibromyalgia Impact Questionnaire scores within the initial 4 weeks of intervention [ 66 ]. In addition to physical improvements, the therapy also positively influenced psychological parameters, evident from reduced depression scores and enhanced Physical Summary Scores on the SF-12 questionnaire [ 66 ]. Subsequent research has affirmed these findings, with notable reductions in fatigue and psychological distress in fibromyalgia patients post-ozone therapy [ 25 , 41 ]. While transient meteorism was occasionally reported, the side effects did not negate the overall beneficial outcomes of the therapy [ 66 ]. Furthermore, in a distinct study focusing on CFS, an impressive 70% of patients reported significant symptom alleviation, with the treatment presenting no major adverse effects [ 67 ]. Collectively, ozone therapy, whether through O 2 –O 3 -AHT or rectal insufflation, provides a robust therapeutic approach to the comprehensive challenges presented by ME/CFS and fibromyalgia. The therapy multifaceted mechanisms of action, coupled with its evident safety profile [ 67 , 68 , 69 ], advocate for its consideration, especially for patients who have found limited relief with other treatments [ 69 ].

Miscellaneous

Ozone therapy has been identified as a versatile therapeutic modality, demonstrating efficacy across diverse medical conditions. For avascular bisphosphonate-related jaw osteonecrosis, approximately 54% of patients achieved complete lesion healing following an average treatment duration of 6.5 months, suggesting its potential in managing such conditions [ 51 ]. Furthermore, ozone therapy has demonstrated longer-lasting symptom relief for chronic plantar fasciitis compared to corticosteroids [ 55 ].

In orthopaedics, ozone therapy has been examined for its benefits in a range of conditions, from temporomandibular joint disorders to low back pain and carpal tunnel syndrome [ 8 , 11 , 15 ]. Its application has also been extended to chronic wound and ulcer management. Topical and injected ozone treatments have enhanced wound healing by reducing inflammation, facilitating rapid wound closure, and promoting angiogenesis and fibroblast proliferation, as evident from animal studies [ 22 , 23 ]. Case studies and systematic reviews, such as those by Fitzpatrick et al. and Romary et al., highlight the efficacy of ozone therapy in promoting wound healing, especially in chronic wound scenarios [ 24 , 75 ]. Particularly for diabetic foot ulcers, ozone therapy has been credited for reducing wound size and amputation rates [ 76 , 77 , 78 , 79 , 80 ].

The mechanistic attributes of ozone therapy in tissue repair have also been explored. Studies highlight its role in promoting fibroblast migration, stimulating epithelial–mesenchymal transition via the PI3K/Akt/mTOR pathway, and augmenting angiogenesis [ 22 , 23 ]. A notable case study indicated the potential of combined ozone therapy in promoting the healing of extensive tibial wounds, enhancing the patient mobility [ 42 ]. In sports medicine, ozone therapy has proven valuable in managing injuries such as those in professional athletes with hamstring injuries, resulting in diminished pain, improved mobility, and enhanced perfusion [ 57 , 77 , 78 ]. Its safety and effectiveness are also corroborated by systematic reviews in pain medicine for conditions like knee OA [ 2 ]. The significant research findings and merits and de-merits of ozone therapy for various musculoskeletal conditions are jotted in Tables 1 and 2 , respectively.

Complications of ozone therapy

Ozone therapy, increasingly recognized in the scientific literature for its therapeutic potential, has demonstrated a commendable safety profile across various clinical applications when administered following established guidelines and employing an atoxic system. Clinical trials predominantly document mild and self-limiting adverse effects such as abdominal distension, lower limb hypoesthesia, and transient pain exacerbation, which typically resolve without necessitating extensive medical intervention [ 4 , 21 , 25 , 26 , 66 ]. Specifically, within the realm of musculoskeletal disorders, ozone therapy, in conditions like knee OA, has showcased a safety profile commensurate with hyaluronic acid injections, with both modalities yielding only mild, transient adverse events [ 33 , 46 ]. This safety spectrum further extends to osteomyelitis, as evidenced in a rat model study, wherein ozone therapy augmented antioxidant mechanisms devoid of adverse reactions [ 14 ]. In treating chronic wounds and ulcers, studies have reiterated the minimal risk profile of ozone therapy, emphasizing its efficacy in wound healing [ 80 ]. Nonetheless, potential complications arising from specific administration methods, such as the use of polyvinyl chloride (PVC) auto-transfusion bags contaminated with excessive citrate–phosphate–dextrose (CPD), warrant vigilance [ 21 ]. Puncture accidents during therapy and risks in populations like the elderly or those with decompensated conditions merit careful consideration [ 10 , 12 ].

Contraindications to ozone therapy are unambiguously defined, encompassing conditions like latent hypoglycaemia, hyperthyroidism, favism (due to G-6PD deficiency), pregnancy, and sickle cell anaemia. The use of angiotensin-converting enzyme (ACE) inhibitors also mandates caution with ozone therapy [ 20 , 21 ]. These contraindications are rooted in potential risks such as the prospect of hemolysis in G-6PD deficiency, mutagenic concerns during early pregnancy, and asthmatic hypersensitivity.

Prolonged exposure to ozone may adversely affect the respiratory system, fostering the release of deleterious compounds into the bloodstream and potentially resulting in multiorgan damage [ 81 ]. Cells with deficient antioxidant activity are especially vulnerable to mutagenic alterations with sustained ozone exposure [ 82 ]. High concentrations of ozone can induce DNA oxidation and exhibit genotoxic effects [ 83 ]. Within the pulmonary milieu, ozone interaction with unsaturated fatty acids can yield lipid ozonation products, engendering lipid peroxidation, perturbed membrane permeability, and subsequent activation of inflammatory mediators [ 84 ]. When ozone commingles with nitrogen dioxide (NO 2 ), the ensuing photochemical smog can intensify detrimental effects. Fortunately, antioxidants like vitamins E and C, along with anti-inflammatory agents like indomethacin, can serve as protective countermeasures against such adversities [ 20 , 21 ].

Limitations of ozone therapy

Ozone therapy, emerging as a promising therapeutic modality across diverse medical conditions, including osteonecrosis of the femoral head (ONFH) and orthopaedic disorders, has garnered considerable attention. Despite promising clinical observations [ 1 , 5 , 20 , 28 , 51 , 69 ], the present study has methodological challenges which negatively impact the robustness of the findings. Notably, the prevalence of studies with small sample sizes underscores the imperative for investigations with expansive patient cohorts and prolonged follow-ups to affirm these preliminary insights [ 20 , 46 , 69 , 80 ]. The methodological quality of much extant research tempers the interpretability of results. A significant portion of these studies exhibit a high or uncertain risk of bias [ 8 , 12 ], highlighting the exigency for methodologically rigorous investigations to validate the therapeutic potential of ozone therapy [ 2 ]. A marked inconsistency in treatment protocols, as underscored by Fernandez-Cuadros et al. regarding knee OA [ 33 ], underlines the need for standardized approaches. Such consistency can lay the foundation for evidence-based guidelines, fostering the safe and efficacious deployment of ozone therapy in clinical contexts. A paramount lacuna in our current comprehension pertains to the elucidation of the exact biological and physiological mechanisms underpinning ozone therapy effects. This knowledge gap is accentuated in conditions like ONFH, where hypotheses around ozone role in neovascularization and immunoregulation remain speculative [ 1 ]. Sire et al. further highlight the necessity to broaden the research spectrum to encompass additional musculoskeletal disorders, enriching our understanding of ozone therapy applications in orthopaedics [ 18 ]. Deciphering these mechanisms is pivotal to refine and optimize treatment protocols. While the promise of ozone therapy in contexts such as wound healing in diabetic foot ulcers is evident [ 80 , 85 ], the call for high-quality randomized controlled trials (RCTs) reverberates, especially when venturing beyond specific wound types. This is emblematic of the overarching need for comprehensive, methodologically sound trials which shed light on ozone therapy influence on inflammatory mediators and its broader clinical implications [ 3 ]. Despite ozone therapy prospective benefits, it encounters significant barriers to its mainstream medical integration, most prominently the absence of FDA approval and a paucity of evidence corroborating its expansive efficacy [ 17 , 86 , 87 ]. Overcoming these challenges demands the undertaking of well-constructed research studies, adhering to rigorous scientific paradigms. By redressing these research inadequacies, the scientific community can proffer compelling evidence which vindicates the place of ozone therapy in contemporary healthcare, emphasizing its efficacy and safety vis-à-vis conventional treatments [ 24 , 75 , 80 ].

Level of evidence of ozone in orthopaedics

In this paper, the cited evidence is classified according to the Oxford Centre for Evidence-based Medicine hierarchy. Foremost, Level 2 evidence, encompassing randomized controlled trials (RCTs) and systematic reviews with meta-analyses, offers the most robust validation. This calibre of evidence prominently features in discussions on osteoarthritis, osteomyelitis, tendinopathies, rheumatic pathologies, and chronic pain syndromes. The methodological rigour of Level 2 studies engenders heightened confidence in the outcomes they present as mentioned in Table  3 . Subsequently, Level 3 evidence, characterized by non-randomized controlled trials and case–control studies, emerges in contexts such as avascular necrosis, osteomyelitis, adhesive capsulitis, chronic pain syndromes, and fibromyalgia. Though not mirroring the exacting standards of Level 2 research, these Level 3 studies furnish indispensable foundational knowledge, potentially guiding subsequent, more rigorous investigations.

Lastly, a solitary study, a systematic review assessing preclinical therapies for fracture non-union, is identified as Level 4 evidence. While systematic reviews are typically accorded a higher evidential standing, the specific methodological approach of this review positions it within the Level 4 category. In summary, the literature predominantly aligns with Level 2 and Level 3 evidence, reflecting a considerable degree of scientific rigour. The limited representation of Level 4 evidence, confined to a single systematic review of fractures, accentuates the depth of the findings while simultaneously highlighting domains warranting further high-level research to corroborate assertions across diverse medical contexts.

Future prospective

The advancement of ozone therapy scientific foundation in orthopaedics demands stringent, meticulously designed randomized controlled trials with expanded sample sizes and prolonged follow-up periods [ 4 , 8 , 11 , 15 , 22 ]. Such studies should prioritize consistent, objective outcome measures to elevate their methodological quality, enabling a clearer comparison of various treatments [ 4 ]. It is essential that research impartially delineates the pros and cons of ozone therapy. A focal area of inquiry should be the interplay between constant oxidative stress and the episodic acute stress invoked by ozone treatments, a realm presently marked by uncertainty [ 40 ]. Deepening our comprehension of the molecular underpinnings guiding therapeutic responses is pivotal, not only to academic discourse but also in refining and personalizing treatment regimens for specific orthopaedic afflictions [ 1 , 28 , 65 , 85 ]. For the therapy to achieve broader clinical acceptance, the establishment of standardized guidelines is paramount. These guidelines would encapsulate the best practices for ozone administration, thereby ensuring uniformity in therapeutic approaches and the consequent predictability of clinical outcomes [ 66 ]. The identification of differentially expressed genes between positive responders and non-responders to ozone therapy can offer insights into predicting therapeutic outcomes, facilitating a more personalized therapeutic strategy [ 1 ]. Research endeavours should encompass a detailed exploration of optimal dosages, session frequencies, and treatment durations, ensuring maximized therapeutic potential across varied orthopaedic conditions [ 3 , 66 ]. Furthermore, expansive multicentre prospective studies can enhance the validation of ozone therapy’s therapeutic and safety profiles, extending its reach to broader patient demographics with a multitude of clinical conditions [ 1 , 52 , 88 , 89 , 90 , 91 , 92 ]. Emphasis should also be placed on examining the specific impacts of topical ozone treatments, such as ozonated water and oils, which currently lack comprehensive evaluation in human studies [ 75 ]. The fortification of ozone therapy’s role in orthopaedics necessitates a multifaceted research approach. This encompasses the execution of rigorous trials, a deeper investigation into molecular mechanisms, and the formulation of standardized treatment protocols. Through such integrated efforts, the orthopaedic community can establish a solid evidence base, ensuring the effective and reliable clinical integration of ozone therapy.

Ozone therapy, with its multifaceted potential, is steadily gaining prominence across diverse clinical domains, notably within orthopaedics. At the physiological level, the therapy exhibits attributes crucial for orthopaedic interventions, including enhanced tissue oxygenation, modulation of oxidative stress, and anti-inflammatory properties. Such mechanisms provide potential relief in conditions such as knee OA, chronic osteomyelitis, and various chronic pain syndromes, positioning ozone therapy as a valuable adjunct or alternative when conventional treatments fall short. While the merits are evident, a comprehensive endorsement of ozone therapy in mainstream clinical practice awaits a more robust body of evidence. The imperative lies in rigorous research endeavours, particularly well-executed randomized controlled trials, to address current methodological limitations. Comprehensive studies must also delve into the precise mechanisms underpinning ozone therapy therapeutic impacts. Alongside this, the establishment of standardized treatment guidelines will be vital for its broader clinical acceptance and application. Until consolidated evidence emerges, clinicians should approach ozone therapy with circumspection, calibrating its use based on individual patient needs and clinical contexts. In essence, while ozone therapy offers considerable promise for a myriad of orthopaedic conditions, the onus is on the scientific community to further elucidate its efficacy, refine its application, and ascertain its safety profile.

Availability of data and materials

The datasets generated during and/or analysed during the current study are available throughout the manuscript.

An J-X, Wu G-P, Niu K, Wei Y-P, Liu H, Gao X-Y, et al. Treatment of femoral head osteonecrosis with ozone therapy: pilot trial of a new therapeutic approach. Pain Physician. 2022;25:E43–54.

PubMed   Google Scholar  

Hidalgo-Tallón FJ, Torres-Morera LM, Baeza-Noci J, Carrillo-Izquierdo MD, Pinto-Bonilla R. Updated review on ozone therapy in pain medicine. Front Physiol. 2022;13: 840623.

Article   PubMed   PubMed Central   Google Scholar  

de Sire A, Marotta N, Ferrillo M, Agostini F, Sconza C, Lippi L, et al. Oxygen–ozone therapy for reducing pro-inflammatory cytokines serum levels in musculoskeletal and temporomandibular disorders: a comprehensive review. Int J Mol Sci. 2022;23:2528.

Bennett PM, Stewart SK, Dretzke J, Bem D, Penn-Barwell JG. Preclinical therapies to prevent or treat fracture non-union: a systematic review. PLoS ONE. 2018;13: e0201077.

Kazancioglu HO, Ezirganli S, Aydin MS. Effects of laser and ozone therapies on bone healing in the calvarial defects. J Craniofac Surg. 2013;24:2141–6.

Article   PubMed   Google Scholar  

Irban A, Uslu S, Gereli A, Aydinlar EI, Karyemez PE, Luleci N, et al. The effect of ozone therapy on experimental bone fracture healing in rats [abstract]. J Ozone Ther. 2019;3:29.

Article   Google Scholar  

Feng X, Beiping L. Therapeutic efficacy of ozone injection into the knee for the osteoarthritis patient along with oral celecoxib and glucosamine. J Clin Diagn Res. 2017;11:UC01–3.

CAS   PubMed   PubMed Central   Google Scholar  

Sconza C, Respizzi S, Virelli L, Vandenbulcke F, Iacono F, Kon E, et al. Oxygen–ozone therapy for the treatment of knee osteoarthritis: a systematic review of randomized controlled trials. Arthrosc J Arthrosc Relat Surg. 2020;36:277–86.

Aliyev D, Akkemik U, Asik I. Efficacy of an intra-articular ozone injection for chronic knee pain due to osteoarthritis. Altern Ther Health Med. 2023;29:24–8.

Lopes De Jesus CC, Dos Santos FC, De Jesus LMOB, Monteiro I, SantAna MSSC, Trevisani VFM. Comparison between intra-articular ozone and placebo in the treatment of knee osteoarthritis: a randomized, double-blinded, placebo-controlled study. PLoS ONE. 2017;12: e0179185.

Biazzo A, Corriero AS, Confalonieri N. Intramuscular oxygen–ozone therapy in the treatment of low back pain. Acta Bio Medica Atenei Parmensis. 2018;89:41–6.

De Andrade RR, De Oliveira-Neto OB, Barbosa LT, Santos IO, De Sousa-Rodrigues CF, Barbosa FT. Effectiveness of ozone therapy compared to other therapies for low back pain: a systematic review with meta-analysis of randomized clinical trials. Braz J Anesthesiol. 2019;69:493–501 ( English Edition ).

Nabi BN, Sedighinejad A, Mirbolouk AR, Farzi F, Haghighi M, Atrkarroushan Z, et al. The effectiveness of ozone therapy in chronic osteomyelitis: a randomized controlled clinical trial. Arch Clin Infect Dis. 2018;13: e61320.

Google Scholar  

Bilge A, Öztürk Ö, Adali Y, Üstebay S. Could ozone treatment be a promising alternative for osteomyelitis? An experimental study. Acta Ortop Bras. 2018;26:67–71.

Viebahn-Hänsler R, León Fernández OS, Fahmy Z. Ozone in medicine: the low-dose ozone concept—guidelines and treatment strategies. Ozone Sci Eng. 2012;34:408–24.

Yasheng T, Mijiti A, Yushan M, Liu Z, Liu Y, Yusufu A. Ozonated water lavage and physiological saline irrigation combined with vacuum-sealed drainage in the treatment of 18 cases of chronic osteomyelitis. J Int Med Res. 2021;49:0300060521999530.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Rowen RJ, Robins H. Ozone therapy for complex regional pain syndrome: review and case report. Curr Pain Headache Rep. 2019;23:41.

De Sire A, Agostini F, Lippi L, Mangone M, Marchese S, Cisari C, et al. Oxygen–ozone therapy in the rehabilitation field: state of the art on mechanisms of action, safety and effectiveness in patients with musculoskeletal disorders. Biomolecules. 2021;11:356.

Alebouyeh M-R, Morsali SF, Zojaji F, Ebrahimi SA, Ahani A, Antar A. Refractory complex regional pain syndrome: a case report and review of literature. Anesth Pain Med. 2023;13: e135286.

Seyam O, Smith N, Reid I, Gandhi J, Jiang W, Khan S. Clinical utility of ozone therapy for musculoskeletal disorders. Med Gas Res. 2018;8:103.

Bocci V. The potential toxicity of ozone: side effects and contraindications of ozonetherapy. In: Ozone. Dordrecht: Springer; 2010. p. 75–84.

Chapter   Google Scholar  

Xiao W, Tang H, Wu M, Liao Y, Li K, Li L, et al. Ozone oil promotes wound healing by increasing the migration of fibroblasts via PI3K/Akt/mTOR signaling pathway. Biosci Rep. 2017;37:BSR20170658.

Pchepiorka R, Moreira MS, Lascane NADS, Catalani LH, Allegrini S Jr, De Lima NB, et al. Effect of ozone therapy on wound healing in the buccal mucosa of rats. Arch Oral Biol. 2020;119: 104889.

Article   CAS   PubMed   Google Scholar  

Fitzpatrick E, Holland OJ, Vanderlelie JJ. Ozone therapy for the treatment of chronic wounds: a systematic review. Int Wound J. 2018;15:633–44.

Serra MEG, Baeza-Noci J, Mendes Abdala CV, Luvisotto MM, Bertol CD, Anzolin AP. The role of ozone treatment as integrative medicine. An evidence and gap map. Front Public Health. 2023;10:1112296.

Akkawi I. Ozone therapy for musculoskeletal disorders Current concepts. Acta Biomed. 2020;91(4):e2020191.

Bocci V. How is ozone administered? Ozone. 2010. https://doi.org/10.1007/978-90-481-9234-2_5 .

Article   PubMed Central   Google Scholar  

Re L, Mawsouf MN, Menéndez S, León OS, Sánchez GM, Hernández F. Ozone therapy: clinical and basic evidence of its therapeutic potential. Arch Med Res. 2008;39:17–26.

Bocci VA. Tropospheric ozone toxicity vs. usefulness of ozone therapy. Arch Med Res. 2007;38:265–7.

Bocci V. Ozone as Janus: this controversial gas can be either toxic or medically useful. Mediat Inflamm. 2004;13:3–11.

Article   CAS   Google Scholar  

Li Q, Qi X, Zhang Z. Intra-articular oxygen–ozone versus hyaluronic acid in knee osteoarthritis: a meta-analysis of randomized controlled trials. Int J Surg. 2018;58:3–10.

Sagai M, Bocci V. Mechanisms of action involved in ozone therapy: is healing induced via a mild oxidative stress? Med Gas Res. 2011;1:29.

Fernández-Cuadros ME, Pérez-Moro OS, Albaladejo-Florín MJ, Tobar-Izquierdo MM, Magaña-Sánchez A, Jiménez-Cuevas P, et al. Intra articular ozone modulates inflammation and has anabolic effect on knee osteoarthritis: IL-6 and IGF-1 as pro-inflammatory and anabolic biomarkers. Processes. 2022;10:138.

Travagli V, Iorio EL. The biological and molecular action of ozone and its derivatives: state-of-the-art, enhanced scenarios, and quality insights. Int J Mol Sci. 2023;24:8465.

Galiè M, Covi V, Tabaracci G, Malatesta M. The role of Nrf2 in the antioxidant cellular response to medical ozone exposure. Int J Mol Sci. 2019;20:4009.

Viebahn-Haensler R, León Fernández OS. Ozone in medicine. The low-dose ozone concept and its basic biochemical mechanisms of action in chronic inflammatory diseases. Int J Mol Sci. 2021;22:7890.

Xu W, Yang Z, Lu N. A new role for the PI3K/Akt signaling pathway in the epithelial–mesenchymal transition. Cell Adhes Migr. 2015;9:317–24.

Dong J, Zhang L, Jia H, Zhu Y, Xiang R, Li P. Effects of adjuvant application of corticosteroid and ozone after ultrasound-guided puncture and lavage for the treatment of rotator cuff calcific tendinitis: study protocol for a non-inferiority randomized controlled trial. Trials. 2023;24:375.

Stewart S. Fracture non-union: a review of clinical challenges and future research needs. Malays Orthop J. 2019;13:1–10.

Duman IG, Davul S, Gokce H, Gonenci R, Özden R, Uruc V. Effects of gaseous ozone treatment on bone regeneration in femoral defect model in rats. J Hard Tissue Biol. 2017;26:7–12.

Irban A, Uslu S, Gereli A, Aydinlar EI, Karyemez PE, Luleci N, et al. The effect of ozone therapy on experimental bone fracture healing in rats. Int Res J Public Environ Health. 2015. https://doi.org/10.15739/irjpeh.034 .

Shah P, Shyam AK, Shah S. Adjuvant combined ozone therapy for extensive wound over tibia. IJOO. 2011;45:376–9.

Vaillant JD, Fraga A, Díaz MT, Mallok A, Viebahn-Hänsler R, Fahmy Z, et al. Ozone oxidative postconditioning ameliorates joint damage and decreases pro-inflammatory cytokine levels and oxidative stress in PG/PS-induced arthritis in rats. Eur J Pharmacol. 2013;714:318–24.

Sun P, Xu W, Zhao X, Zhang C, Lin X, Gong M, et al. Ozone induces autophagy by activating PPARγ/mTOR in rat chondrocytes treated with IL-1β. J Orthop Surg Res. 2022;17:351.

Farpour HR, Ashraf A, Hosseini SS. The efficacy of ozone prolotherapy compared to intra-articular hypertonic saline injection in reducing pain and improving the function of patients with knee osteoarthritis: a randomized clinical trial. Evid-Based Complement Altern Med. 2021;2021:1–7.

Sconza C, Di Matteo B, Queirazza P, Dina A, Amenta R, Respizzi S, et al. Ozone therapy versus hyaluronic acid injections for pain relief in patients with knee osteoarthritis: preliminary findings on molecular and clinical outcomes from a randomized controlled trial. IJMS. 2023;24:8788.

Iliakis E, Bonetti M, Iliakis A. Osteonecrosis of the femoral head: could oxygen–ozone therapy became a treatment option? J Ozone Ther. 2015;1(1).1-7.

Sacco R, Leeson R, Nissan J, Olate S, Bettoni Cruz de Castro CH, Acocella A, et al. A systematic review of oxygen therapy for the management of medication-related osteonecrosis of the jaw (MRONJ). Appl Sci. 2019;9:1026.

Ripamonti CI, Cislaghi E, Mariani L, Maniezzo M. Efficacy and safety of medical ozone (O 3 ) delivered in oil suspension applications for the treatment of osteonecrosis of the jaw in patients with bone metastases treated with bisphosphonates: preliminary results of a phase I–II study. Oral Oncol. 2011;47:185–90.

Di Fede O, Del Gaizo C, Panzarella V, La Mantia G, Tozzo P, Di Grigoli A, et al. Ozone infiltration for osteonecrosis of the jaw therapy: a case series. J Clin Med. 2022;11:5307.

Agrillo A, Ungari C, Filiaci F, Priore P, Iannetti G. Ozone therapy in the treatment of avascular bisphosphonate-related jaw osteonecrosis. J Craniofac Surg. 2007;18:1071–5.

Shetty L, Channe S, Londhe U, Bongulwar R. Potential use of local ozone therapy for neonatal mandibular osteomyelitis. World J Dent. 2018;9:343–4.

Foula AS, Sabry LS, Elmulla AF, Kamel MA, Hozien AI. Ultrasound-guided shoulder intraarticular ozone injection versus pulsed radiofrequency application for shoulder adhesive capsulitis: a randomized controlled trial. Pain Physician. 2023;26(4):E329.

Kizilkaya V, Uruc V, Levent A, Kanat O, Yildizgoren MT, Dogramaci Y, et al. Effectiveness of ozone therapy on tendon healing: an experimental study in generated Achilles tendon injury model in rats. J Hard Tissue Biol. 2018;27:309–14.

Babaei-Ghazani A, Karimi N, Forogh B, Madani SP, Ebadi S, Fadavi HR, et al. Comparison of ultrasound-guided local ozone (O 2 –O 3 ) injection vs corticosteroid injection in the treatment of chronic plantar fasciitis: a randomized clinical trial. Pain Med. 2019;20:314–22.

Atar MÖ, Korkmaz N, Aslan SG, Tezen Ö, Köylü SU, Demir Y, et al. Comparison of ultrasound-guided subacromial corticosteroid and ozone (O 2 –O 3 ) injections in the treatment of chronic rotator cuff tendinopathy: a randomized clinical trial. Korean J Pain. 2023;36:128–36.

Hidalgo-Tallón FJ, Pinto-Bonilla R, Baeza-Noci J, Menéndez-Cepero S, Cabizosu A. Medical ozone on hamstring injury in a professional athlete assessed by thermography: a clinical case report. BJR Case Rep. 2023;9:20220078.

PubMed   PubMed Central   Google Scholar  

Gurger M, Once G, Yilmaz E, Demir S, Calik I, Say Y, et al. The effect of the platelet-rich plasma and ozone therapy on tendon-to-bone healing in the rabbit rotator cuff repair model. J Orthop Surg Res. 2021;16:202.

Bozbaş GT, Yilmaz M, Paşaoğlu E, Gürer G, İvgin R, Demirci B. Effect of ozone in Freund’s complete adjuvant-induced arthritis. Arch Rheumatol. 2018;33:137–42.

Tartari APS, Moreira FF, Pereira MCDS, Carraro E, Cidral-Filho FJ, Salgado AI, et al. Anti-inflammatory effect of ozone therapy in an experimental model of rheumatoid arthritis. Inflammation. 2020;43:985–93.

León Fernández OS, Viebahn-Haensler R, Cabreja GL, Espinosa IS, Matos YH, Roche LD, et al. Medical ozone increases methotrexate clinical response and improves cellular redox balance in patients with rheumatoid arthritis. Eur J Pharmacol. 2016;789:313–8.

Taşçı BG. Effect of ozone in Freund’s complete adjuvant-induced arthritis. Arch Rheumatol. 2018;33:137–42.

Zhao N, Smargiassi A, Jean S, Gamache P, Laouan-Sidi E-A, Chen H, et al. Long-term exposure to fine particulate matter and ozone and the onset of systemic autoimmune rheumatic diseases: an open cohort study in Quebec, Canada. Arthritis Res Ther. 2022;24:151.

[PDF] Ozone and stem cell a possible cure for Rheumatoid Arthritis | Semantic Scholar. https://www.semanticscholar.org/paper/Ozone-and-stem-cell-a-possible-cure-for-Rheumatoid-Nd-Cht/710b110e67457449bf0fca17e1056db608ddf32d . Accessed 9 Oct 2023.

Rania V, Marcianò G, Casarella A, Vocca C, Palleria C, Calabria E, et al. Oxygen–ozone therapy in cervicobrachial pain: a real-life experience. JCM. 2022;12:248.

Hidalgo-Tallón J, Menéndez-Cepero S, Vilchez JS, Rodríguez-López CM, Calandre EP. Ozone therapy as add-on treatment in fibromyalgia management by rectal insufflation: an open-label pilot study. J Altern Complement Med. 2013;19:238–42.

Tirelli U, Cirrito C, Pavanello M. Oxygen–ozone therapy is an effective therapy in chronic fatigue syndrome: results in 100 patients. BJSTR. 2019;21:15890–2.

Tirelli U, Franzini M, Valdenassi L, Pandolfi S, Berretta M, Ricevuti G, et al. Patients with myalgic encephalomyelitis/chronic fatigue syndrome (ME/CFS) greatly improved fatigue symptoms when treated with oxygen–ozone autohemotherapy. J Clin Med. 2021;11:29.

Tirelli U, Cirrito C, Pavanello M. Ozone therapy is an effective therapy in chronic fatigue syndrome: result of an Italian study in 65 patients. Ozone Ther. 2018. https://doi.org/10.4081/ozone.2018.7812 .

Chirumbolo S, Valdenassi L, Franzini M, Pandolfi S, Ricevuti G, Tirelli U. Male vs. female differences in responding to oxygen–ozone autohemotherapy (O 2 –O 3 -AHT) in patients with myalgic encephalomyelitis/chronic fatigue syndrome (ME/CFS). J Clin Med. 2022;11:173.

Tirelli U, Cirrito C, Pavanello M. Ozone therapy in 40 patients with fibromyalgia: an effective therapy. Ozone Ther. 2018. https://doi.org/10.4081/ozone.2018.7969 .

Gazioğlu Türkyılmaz G, Rumeli Ş, Bakır M. Effects of major ozone autohemotherapy on physical functionality and quality of life in fibromyalgia syndrome: a prospective cross-sectional study. Altern Ther Health Med. 2021;27:8–12.

Sucuoğlu H, Soydaş N. Efficacy of ozone therapy as an add-on treatment in fibromyalgia: a randomized double-blind placebo-controlled study. J Back Musculoskelet Rehabil. 2023;36:357–66.

Tirelli U, Cirrito C, Pavanello M, Piasentin C, Lleshi A, Taibi R. Ozone therapy in 65 patients with fibromyalgia: an effective therapy. Eur Rev Med Pharmacol Sci. 2019;23:1786–8.

CAS   PubMed   Google Scholar  

Romary DJ, Landsberger SA, Bradner KN, Ramirez M, Leon BR. Liquid ozone therapies for the treatment of epithelial wounds: a systematic review and meta-analysis. Int Wound J. 2023;20:1235–52.

Faraji N, Goli R, Choobianzali B, Bahrami S, Sadeghian A, Sepehrnia N, et al. Ozone therapy as an alternative method for the treatment of diabetic foot ulcer: a case report. J Med Case Rep. 2021;15:234.

Astasio-Picado Á, Babiano AÁ, López-Sánchez M, Lozano RR, Cobos-Moreno P, Gómez-Martín B. Use of ozone therapy in diabetic foot ulcers. J Pers Med. 2023;13:1439.

Kushmakov R, Gandhi J, Seyam O, Jiang W, Joshi G, Smith NL, et al. Ozone therapy for diabetic foot. Med Gas Res. 2018;8:111–5.

Izadi M, Kheirjou R, Mohammadpour R, Aliyoldashi MH, Moghadam SJ, Khorvash F, et al. Efficacy of comprehensive ozone therapy in diabetic foot ulcer healing. Diabetes Metab Syndr. 2019;13:822–5.

Wen Q, Liu D, Wang X, Zhang Y, Fang S, Qiu X, et al. A systematic review of ozone therapy for treating chronically refractory wounds and ulcers. Int Wound J. 2022;19:853–70.

Kim S-Y, Kim E, Kim WJ. Health effects of ozone on respiratory diseases. Tuberc Respir Dis. 2020;83(Suppl 1):S6–11.

Lobo V, Patil A, Phatak A, Chandra N. Free radicals, antioxidants and functional foods: impact on human health. Pharmacogn Rev. 2010;4:118–26.

Palli D, Sera F, Giovannelli L, Masala G, Grechi D, Bendinelli B, et al. Environmental ozone exposure and oxidative DNA damage in adult residents of Florence. Italy Environ Pollut. 2009;157:1521–5.

Ballinger CA, Cueto R, Squadrito G, Coffin JF, Velsor LW, Pryor WA, et al. Antioxidant-mediated augmentation of ozone-induced membrane oxidation. Free Radic Biol Med. 2005;38:515–26.

Manoto SL, Maepa MJ, Motaung SK. Medical ozone therapy as a potential treatment modality for regeneration of damaged articular cartilage in osteoarthritis. Saudi J Biol Sci. 2018;25:672–9.

Secondary direct food additives permitted in food for human consumption (21 CFR 173.5-173.405). | FAOLEX. https://www.fao.org/faolex/results/details/en/c/LEX-FAOC156335/ . Accessed 14 Nov 2023.

Grangeat AM, de LA Erario M. The use of medical ozone in chronic intervertebral disc degeneration can be an etiological and conservative treatment. Int J Mol Sci. 2023;24:6538.

El Meligy OA, Elemam NM, Talaat IM. Ozone therapy in medicine and dentistry: a review of the literature. Dent J. 2023;11:187.

Bocci V, Zanardi I, Borrelli E, Travagli V. Reliable and effective oxygen–ozone therapy at a crossroads with ozonated saline infusion and ozone rectal insufflation. J Pharm Pharmacol. 2012;64:482–9.

Shah M, Captain J, Vaidya V, Kulkarni A, Valsangkar K, Nair PMK, et al. Safety and efficacy of ozone therapy in mild to moderate COVID-19 patients: a phase 1/11 randomized control trial (SEOT study). Int Immunopharmacol. 2021;91: 107301.

Epelle EI, Macfarlane A, Cusack M, Burns A, Okolie JA, Mackay W, et al. Ozone application in different industries: a review of recent developments. Chem Eng J. 2023;454: 140188.

Steppan J, Meaders T, Muto M, Murphy KJ. A metaanalysis of the effectiveness and safety of ozone treatments for herniated lumbar discs. J Vasc Interv Radiol. 2010;21:534–48.

Download references

Acknowledgements

The authors received no financial support for the research, authorship, and/or publication of this article.

Author information

Authors and affiliations.

Department of Orthopaedics, ACS Medical College and Hospital, Dr. MGR Educational and Research Institute, Chennai, Tamil Nadu, 600077, India

Madhan Jeyaraman & Naveen Jeyaraman

Department of Orthopaedics, Government Medical College, Omandurar Government Estate, Chennai, Tamil Nadu, 600002, India

Swaminathan Ramasubramanian & Sangeetha Balaji

Department of Orthopaedics, Jawaharlal Institute of Postgraduate Medical Education and Research (JIPMER)–Karaikal, Puducherry, 605006, India

Arulkumar Nallakumarasamy

Department of Orthopaedics, All India Institute of Medical Sciences, Bhubaneswar, Odisha, 751019, India

Bishnu Prasad Patro

Department of Orthopaedic, Trauma, and Reconstructive Surgery, RWTH University Medical Centre, Pauwelsstraße 30, 52074, Aachen, Germany

Filippo Migliorini

Department of Orthopaedic and Trauma Surgery, Academic Hospital of Bolzano (SABES-ASDAA), 39100, Bolzano, Italy

Department of Life Sciences, Health, and Health Professions, Link Campus University, Rome, Italy

You can also search for this author in PubMed   Google Scholar

Contributions

MJ: conception and design, drafting (original and revision); MJ, FM: supervision, drafting (revision); MJ, FM: drafting (revision); NJ, SB, SR, AN, BPP: drafting (original). All authors have agreed to the final version to be published and agree to be accountable for all aspects of the work.

Corresponding author

Correspondence to Filippo Migliorini .

Ethics declarations

Ethics approval and consent to participate.

This study complies with ethical standards. Registration and protocol: the present review was not registered.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have any competing interests in this article.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Jeyaraman, M., Jeyaraman, N., Ramasubramanian, S. et al. Ozone therapy in musculoskeletal medicine: a comprehensive review. Eur J Med Res 29 , 398 (2024). https://doi.org/10.1186/s40001-024-01976-4

Download citation

Received : 28 November 2023

Accepted : 14 July 2024

Published : 31 July 2024

DOI : https://doi.org/10.1186/s40001-024-01976-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Orthopaedics
• Traumatology
• Musculoskeletal medicine

European Journal of Medical Research

ISSN: 2047-783X

• Submission enquiries: [email protected]
• General enquiries: [email protected]

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Recommended Articles
• Google Scholar
• on Google Scholar
• Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

A survey on maximum ratio combination: applications, evaluation and future directions.

Share and Cite

Feng, X.; Tian, F.; Wang, J.; Zhou, M.; Li, D.; Sun, H.; Song, R. A Survey on Maximum Ratio Combination: Applications, Evaluation and Future Directions. Electronics 2024 , 13 , 3087. https://doi.org/10.3390/electronics13153087

Feng X, Tian F, Wang J, Zhou M, Li D, Sun H, Song R. A Survey on Maximum Ratio Combination: Applications, Evaluation and Future Directions. Electronics . 2024; 13(15):3087. https://doi.org/10.3390/electronics13153087

Feng, Xiao, Feng Tian, Junfeng Wang, Mingzhang Zhou, Dingzhao Li, Haixin Sun, and Ruiping Song. 2024. "A Survey on Maximum Ratio Combination: Applications, Evaluation and Future Directions" Electronics 13, no. 15: 3087. https://doi.org/10.3390/electronics13153087

Article Metrics

Article access statistics, further information, mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals

Applications of ICG in Breast Cancer for Sentinel Lymph Node Mapping: A Scoping Review Protocol

• Find this author on Google Scholar
• Find this author on PubMed
• Search for this author on this site
• ORCID record for Feryal Kurdi
• For correspondence: [email protected]
• ORCID record for Yahya Kurdi
• ORCID record for Igor Vladimirovich Reshetov
• Info/History
• Preview PDF

Objective: The objective of this scoping review is to evaluate the current literature on the use of Indocyanine Green (ICG) in sentinel lymph node (SLN) mapping for breast cancer patients. This review aims to assess the accuracy, efficacy, and safety of ICG in this context and to identify gaps in the existing research. The outcomes will contribute to the development of further research as part of a PhD project. Introduction: Breast cancer is the leading cause of morbidity and mortality worldwide. Accurate SLN mapping is crucial for staging and treatment planning in early-stage breast cancer. ICG has emerged as a promising agent for fluorescence imaging in SLN mapping. However, comprehensive assessment of its clinical utility, including accuracy and adverse effects, remains limited. This scoping review aims to consolidate evidence on the use of ICG in breast cancer SLN mapping. Inclusion criteria: Patients with early-stage breast cancer (T1, T2), selected T3 cases where sentinel lymph node biopsy is accurate, and clinically node-negative breast cancer. The intervention includes studies using ICG for SLN mapping and assessment of fluorescence imaging cameras. Methods: Five electronic databases will be searched (PubMed, EMBASE, MEDLINE, Web of Science, and SCOPUS) using search strategies developed in consultation with the academic supervisor. The search strategy is set to human studies published in English within the last 11 years. All retrieved citations will be imported to Zotero and then uploaded to Covidence for screening of titles, abstracts, and full text according to pre-specified inclusion criteria. Citations meeting the inclusion criteria for full-text review will have their data extracted by two independent reviewers, with disagreements resolved by discussion. A data extraction tool will be developed to capture full details about the participants, concept, and context, and findings relevant to the scoping review will be summarized. Keywords: Indocyanine Green, ICG, Sentinel Lymph Node, Breast Cancer, Fluorescence, Axillary Lymph Node Mapping, NIR

Competing Interest Statement

The authors have declared no competing interest.

Funding Statement

This study did not receive any funding

Author Declarations

I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.

I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.

I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).

I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.

I changed the orcid link of one of the authors, as well as changed the HTML styling of the abstract

Data Availability

All data produced in the present study are available upon reasonable request to the authors

View the discussion thread.

Thank you for your interest in spreading the word about medRxiv.

NOTE: Your email address is requested solely to identify you as the sender of this article.

Citation Manager Formats

• EndNote (tagged)
• EndNote 8 (xml)
• RefWorks Tagged
• Ref Manager
• Tweet Widget
• Facebook Like
• Google Plus One

Subject Area

• Addiction Medicine (336)
• Allergy and Immunology (658)
• Anesthesia (177)
• Cardiovascular Medicine (2573)
• Dentistry and Oral Medicine (310)
• Dermatology (218)
• Emergency Medicine (390)
• Endocrinology (including Diabetes Mellitus and Metabolic Disease) (915)
• Epidemiology (12088)
• Forensic Medicine (10)
• Gastroenterology (743)
• Genetic and Genomic Medicine (3994)
• Geriatric Medicine (376)
• Health Economics (667)
• Health Informatics (2578)
• Health Policy (992)
• Health Systems and Quality Improvement (959)
• Hematology (357)
• HIV/AIDS (825)
• Infectious Diseases (except HIV/AIDS) (13574)
• Intensive Care and Critical Care Medicine (783)
• Medical Education (396)
• Medical Ethics (107)
• Nephrology (423)
• Neurology (3759)
• Nursing (206)
• Nutrition (559)
• Obstetrics and Gynecology (718)
• Occupational and Environmental Health (688)
• Oncology (1958)
• Ophthalmology (569)
• Orthopedics (233)
• Otolaryngology (301)
• Pain Medicine (247)
• Palliative Medicine (72)
• Pathology (469)
• Pediatrics (1088)
• Pharmacology and Therapeutics (453)
• Primary Care Research (442)
• Psychiatry and Clinical Psychology (3353)
• Public and Global Health (6429)
• Radiology and Imaging (1365)
• Rehabilitation Medicine and Physical Therapy (793)
• Respiratory Medicine (859)
• Rheumatology (394)
• Sexual and Reproductive Health (401)
• Sports Medicine (336)
• Surgery (431)
• Toxicology (51)
• Transplantation (184)
• Urology (164)

A systematic literature review on the usability of mobile applications for visually impaired users

Interacting with mobile applications can often be challenging for people with visual impairments due to the poor usability of some mobile applications. The goal of this paper is to provide an overview of the developments on usability of mobile applications for people with visual impairments based on recent advances in research and application development. This overview is important to guide decision-making for researchers and provide a synthesis of available evidence and indicate in which direction it is worthwhile to prompt further research. We performed a systematic literature review on the usability of mobile applications for people with visual impairments. A deep analysis following the Preferred Reporting Items for SLRs and Meta-Analyses (PRISMA) guidelines was performed to produce a set of relevant papers in the field. We first identified 932 papers published within the last six years. After screening the papers and employing a snowballing technique, we identified 60 studies that were then classified into seven themes: accessibility, daily activities, assistive devices, navigation, screen division layout, and audio guidance. The studies were then analyzed to answer the proposed research questions in order to illustrate the different trends, themes, and evaluation results of various mobile applications developed in the last six years. Using this overview as a foundation, future directions for research in the field of usability for the visually impaired (UVI) are highlighted.

Introduction

The era of mobile devices and applications has begun. With the widespread use of mobile applications, designers and developers need to consider all types of users and develop applications for their different needs. One notable group of users is people with visual impairments. According to the World Health Organization, there are approximately 285 million people with visual impairments worldwide ( World Health Organization, 2020 ). This is a huge number to keep in mind while developing new mobile applications.

People with visual impairments have urged more attention from the tech community to provide them with the assistive technologies they need ( Khan & Khusro, 2021 ). Small tasks that we do daily, such as picking out outfits or even moving from one room to another, could be challenging for such individuals. Thus, leveraging technology to assist with such tasks can be life changing. Besides, increasing the usability of applications and developing dedicated ones tailored to their needs is essential. The usability of an application refers to its efficiency in terms of the time and effort required to perform a task, its effectiveness in performing said tasks, and its users’ satisfaction ( Ferreira et al., 2020 ). Researchers have been studying this field intensively and proposing different solutions to improve the usability of applications for people with visual impairments.

This paper provides a systematic literature review (SLR) on the usability of mobile applications for people with visual impairments. The study aims to find discussions of usability issues related to people with visual impairments in recent studies and how they were solved using mobile applications. By reviewing published works from the last six years, this SLR aims to update readers on the newest trends, limitations of current research, and future directions in the research field of usability for the visually impaired (UVI).

This SLR can be of great benefit to researchers aiming to become involved in UVI research and could provide the basis for new work to be developed, consequently improving the quality of life for the visually impaired. This review differs from previous review studies ( i.e.,   Khan & Khusro, 2021 ) because we classified the studies into themes in order to better evaluate and synthesize the studies and provide clear directions for future work. The following themes were chosen based on the issues addressed in the reviewed papers: “Assistive Devices,” “Navigation,” “Accessibility,” “Daily Activities,” “Audio Guidance,” and “Gestures.” Figure 1 illustrates the percentage of papers classified in each theme.

Figure 1: Percentages of classification themes.

The remainder of this paper is organized as follows: the next section specifies the methodology, following this, the results section illustrates the results of the data collection, the discussion section consists of the research questions with their answers and the limitations and potential directions for future work, and the final section summarizes this paper’s main findings and contribution.

Survey Methodology

This systematic literature review used the Meta-Analyses (PRISMA, 2009) guidelines to produce a set of relevant papers in the field. This SLR was undertaken to address the research questions described below. A deep analysis was performed based on a group of studies; the most relevant studies were documented, and the research questions were addressed.

A. Research questions

The research questions addressed by this study are presented in Table 1 with descriptions and the motivations behind them.

B. Search strategy

This review analysed and synthesised studies on usability for the visually impaired from a user perspective following a systematic approach. As proposed by Tanfield, Denyer & Smart (2003) , the study followed a three-stage approach to ensure that the findings were both reliable and valid. These stages were planning the review, conducting the review by analysing papers, and reporting emerging themes and recommendations. These stages will be discussed further in the following section.

1. Planning stage

The planning stage of this review included defining data sources and the search string protocol as well as inclusion and exclusion criteria.

Data sources.

We aimed to use two types of data sources: digital libraries and search engines. The search process was manually conducted by searching through databases. The selected databases and digital libraries are as follows:

ACM Library

IEEE Xplore

ScienceDirect

SpringerLink

ISI Web of Knowledge

The selected search engines were as follows:

DBLP (Computer Science Bibliography Website)

Google Scholar

Microsoft Academic

Search string.

The above databases were initially searched using the following keyword protocol: (“Usability” AND (”visual impaired” OR ”visually impaired” OR “blind” OR “impairment”) AND “mobile”). However, in order to generate a more powerful search string, the Network Analysis Interface for Literature Studies (NAILS) project was used. NAILS is an automated tool for literature analysis. Its main function is to perform statistical and social network analysis (SNA) on citation data ( Knutas et al., 2015 ). In this study, it was used to check the most important work in the relevant fields as shown in Fig. 2 .

NAILS produced a report displaying the most important authors, publications, and keywords and listed the references cited most often in the analysed papers ( Knutas et al., 2015 ) . The new search string was generated after using the NAILS project as follows: (“Usability” OR “usability model” OR “usability dimension” OR “Usability evaluation model” OR “Usability evaluation dimension”) AND (“mobile” OR “Smartphone”) AND (“Visually impaired” OR “Visual impairment” OR “Blind” OR “Low vision” OR “Blindness”).

Figure 2: NAILS output sample.

Figure 3: Number of papers per database.

Inclusion and exclusion criteria..

To be included in this systematic review, each study had to meet the following screening criteria:

The study must have been published between 2015 and 2020.

The study must be relevant to the main topic (Usability of Mobile Applications for Visually Impaired Users).

The study must be a full-length paper.

The study must be written in English because any to consider any other languages, the research team will need to use the keywords of this language in this topic and deal with search engines using that language to extract all studies related to our topic to form an SLR with a comprehensive view of the selected languages. Therefore, the research team preferred to focus on studies in English to narrow the scope of this SLR.

A research study was excluded if it did not meet one or more items of the criteria.

2. Conducting stage

The conducting stage of the review involved a systematic search based on relevant search terms. This consisted of three substages: exporting citations, importing citations into Mendeley, and importing citations into Rayyan.

Exporting citations.

First, in exporting the citations and conducting the search through the mentioned databases, a total of 932 studies were found. The numbers are illustrated in Fig. 3 below. The highest number of papers was found in Google Scholar, followed by Scopus, ISI Web of Knowledge, ScienceDirect, IEEE Xplore, Microsoft Academic, and DBLP and ACM Library with two studies each. Finally, SpringerLink did not have any studies that met the inclusion criteria.

The chance of encountering duplicate studies was determined to be high. Therefore, importing citations into Mendeley was necessary in order to eliminate the duplicates.

Figure 4: Search stages.

Importing citations into mendeley..

Mendeley is an open-source reference and citation manager. It can highlight paragraphs and sentences, and it can also list automatic references on the end page. Introducing the use of Mendeley is also expected to avoid duplicates in academic writing, especially for systematic literature reviews ( Basri & Patak, 2015 ). Hence, in the next step, the 932 studies were imported into Mendeley, and each study’s title and abstract were screened independently for eligibility. A total of 187 duplicate studies were excluded. 745 total studies remained after the first elimination process. The search stages are shown in Fig. 4 below.

Importing citations into rayyan.

Rayyan QCRI is a free web and mobile application that helps expedite the initial screening of both abstracts and titles through a semi-automated process while incorporating a high level of usability. Its main benefit is to speed up the most tedious part of the systematic literature review process: selecting studies for inclusion in the review ( Ouzzani et al., 2016 ). Therefore, for the last step, another import was done using Rayyan to check for duplications a final time. Using Rayyan, a total of 124 duplicate studies were found, resulting in a total of 621 studies. Using Rayyan, a two-step filtration was conducted to guarantee that the papers have met the inclusion criteria of this SLR. After filtering based on the abstracts, 564 papers did not meet the inclusion criteria. At this stage, 57 studies remained. The second step of filtration eliminated 11 more studies by reading the full papers; two studies were not written in the English language, and nine were inaccessible.

Snowballing.

Snowballing is an emerging technique used to conduct systematic literature reviews that are considered both efficient and reliable using simple procedures. The procedure for snowballing consisted of three phases in each cycle. The first phase is refining the start set, the second phase is backward snowballing, and the third is forward snowballing. The first step, forming the start set, is basically identifying relevant papers that can have a high potential of satisfying the criteria and research question. Backward snowballing was conducted using the reference list to identify new papers to include. It shall start by going through the reference list and excluding papers that do not fulfill the basic criteria; the rest that fulfil criteria shall be added to the SLR. Forward snowballing refers to identifying new papers based on those papers that cited the paper being examined ( Juneja & Kaur, 2019 ). Hence, in order to be sure that we concluded all related studies after we got the 46 papers, a snowballing step was essential. Forward and backward snowballing were conducted. Each of the 46 studies was examined by checking their references to take a look at any possible addition of sources and examining all papers that cited this study. The snowballing activity added some 38 studies, but after full reading, it became 33 that matched the inclusion criteria. A total of 79 studies were identified through this process.

Quality assessment.

A systematic literature review’s quality is determined by the content of the papers included in the review. As a result, it is important to evaluate the papers carefully ( Zhou et al., 2015 ). Many influential scales exist in the software engineering field for evaluating the validity of individual primary studies and grading the overall intensity of the body of proof. Hence, we adapted the comprehensive guidelines specified by Kitchenhand and Charters ( Keele, 2007 ), and the quasi-gold standard (QGS) ( Keele, 2007 ) was used to establish the quest technique, where a robust search strategy for enhancing the validity and reliability of a SLR’s search process is devised using the QGS. By applying this technique, our quality assessment questions were focused and aligned with the research questions mentioned earlier.

In our last step, we had to verify the papers’ eligibility; we conducted a quality check for each of the 79 studies. For quality assessment, we considered whether the paper answered the following questions:

QA1: Is the research aim clearly stated in the research?

QA2: Does the research contain a usability dimension or techniques for mobile applications for people with visual impairments?

QA3: Is there an existing issue with mobile applications for people with visual impairments that the author is trying to solve?

QA4: Is the research focused on mobile application solutions?

After discussing the quality assessment questions and attempting to find an answer in each paper, we agreed to score each study per question. If the study answers a question, it will be given 2 points; if it only partially answers a question, it will be given 1 point; and if there is no answer for a given question in the study, it will have 0 points.

The next step was to calculate the weight of each study. If the total weight was higher or equal to four points, the paper was accepted in the SLR; if not, the paper was discarded since it did not reach the desired quality level. Figure 5 below illustrates the quality assessment process. After applying the quality assessment, 39 papers were rejected since they received less than four points, which resulted in a final tally of 60 papers.

Figure 5: Quality assessment process.

To summarize, this review was conducted according to the Preferred Reporting Items for SLRs and Meta-Analyses (PRISMA) ( Liberati et al., 2009 ). The PRISMA diagram shown in Fig. 6 illustrates all systematic literature processes used in this study.

Figure 6: PRISMA flow diagram.

3. analysing stage.

All researchers involved in this SLR collected the data. The papers were distributed equally between them, and each researcher read each paper completely to determine its topic, extract the paper’s limitations and future work, write a quick summary about it, and record this information in an Excel spreadsheet.

All researchers worked intensively on this systematic literature review. After completing the previously mentioned steps, the papers were divided among all the researchers. Then, each researcher read their assigned papers completely and then classified them into themes according to the topic they covered. The researchers held several meetings to discuss and specify those themes. The themes were identified by the researchers based on the issues addressed in the reviewed papers. In the end, the researchers resulted in seven themes, as shown in Fig. 7 below. The references selected for each theme can be found in the Table A1 . Afterwards, each researcher was assigned one theme to summarize its studies and report the results. In this section, we review the results.

Figure 7: Results of the SLR.

A. accessibility.

Of a total of 60 studies, 10 focused on issues of accessibility. Accessibility is concerned with whether all users are able to have equivalent user experiences, regardless of abilities. Six studies, Darvishy, Hutter & Frei (2019) , Morris et al. (2016) , Qureshi & Hooi-Ten Wong (2020) , Khan, Khusro & Alam (2018) , Paiva et al. (2020) , and Pereda, Murillo & Paz (2020) , gave suggestions for increasing accessibility, ( Darvishy, Hutter & Frei, 2019 ; Morris et al., 2016 ), gave some suggestions for making mobile map applications and Twitter accessible to visually impaired users, and ( Qureshi & Hooi-Ten Wong, 2020 ; Khan, Khusro & Alam, 2018 ) focused on user interfaces and provided accessibility suggestions suitable for blind people. Paiva et al. (2020) and Pereda, Murillo & Paz (2020) proposed a set of heuristics to evaluate the accessibility of mobile applications. Two studies, Khowaja et al. (2019) and Carvalho et al. (2018) , focused on evaluating usability and accessibility issues on some mobile applications, comparing them, and identifying the number and types of problems that visually impaired users faced. Aqle, Khowaja & Al-Thani (2020) proposed a new web search interface designed for visually impaired users. One study, McKay (2017) , focused on accessibility challenges by applying usability tests on a hybrid mobile app with some visually impaired university students.

B. Assistive devices

People with visual impairments have an essential need for assistive technology since they face many challenges when performing activities in daily life. Out of the 60 studies reviewed, 13 were related to assistive technology. The studies Smaradottir, Martinez & Håland (2017) , Skulimowski et al. (2019) , Barbosa, Hayes & Wang, (2016) , Rosner & Perlman (2018) , Csapó et al. (2015) , Khan & Khusro (2020) , Sonth & Kallimani (2017) , Kim et al. (2016) , Vashistha et al. (2015) ; Kameswaran et al. (2020) , Griffin-Shirley et al. (2017) , and Rahman, Anam & Yeasin (2017) were related to screen readers (voiceovers). On the other hand, Bharatia, Ambawane & Rane (2019) , Lewis et al. (2016) were related to proposing an assistant device for the visually impaired. Of the studies related to screening readers, Sonth & Kallimani, (2017) , Vashistha et al. (2015) , Khan & Khusro (2020) Lewis et al. (2016) cited challenges faced by visually impaired users. Barbosa, Hayes & Wang (2016) , Kim et al. (2016) , Rahman, Anam & Yeasin (2017) suggested new applications, while Smaradottir, Martinez & Håland (2017) , Rosner & Perlman (2018) , Csapó et al. (2015) and Griffin-Shirley et al. (2017) evaluated current existing work. The studies Bharatia, Ambawane & Rane (2019) , Lewis et al. (2016) proposed using wearable devices to improve the quality of life for people with visual impairments.

C. Daily activities

In recent years, people with visual impairments have used mobile applications to increase their independence in their daily activities and learning, especially those based on the braille method. We divide the daily activity section into braille-based applications and applications designed to enhance the independence of the visually impaired. Four studies, Nahar, Sulaiman & Jaafar (2020) , Nahar, Jaafar & Sulaiman (2019) , Araújo et al. (2016) and Gokhale et al. (2017) , implemented and evaluated the usability of mobile phone applications that use braille to help visually impaired people in their daily lives. Seven studies, Vitiello et al. (2018) , Kunaratana-Angkul, Wu & Shin-Renn (2020) , Ghidini et al. (2016) , Madrigal-Cadavid et al. (2019) , Marques, Carriço & Guerreiro (2015) , Oliveira et al. (2018) and Rodrigues et al. (2015) , focused on building applications that enhance the independence and autonomy of people with visual impairments in their daily life activities.

D. Screen division layout

People with visual impairments encounter various challenges in identifying and locating non-visual items on touch screen interfaces like phones and tablets. Incidents of accidentally touching a screen element and frequently following an incorrect pattern in attempting to access objects and screen artifacts hinder blind people from performing typical activities on smartphones ( Khusro et al., 2019 ). In this review, 9 out of 60 studies discuss screen division layout: ( Khusro et al., 2019 ; Khan & Khusro, 2019 ; Grussenmeyer & Folmer, 2017 ; Palani et al., 2018 ; Leporini & Palmucci, 2018 ) discuss touch screen (smartwatch tablets, mobile phones, and tablet) usability among people with visual impairments, while ( Cho & Kim, 2017 ; Alnfiai & Sampalli, 2016 ; Niazi et al., 2016 ; Alnfiai & Sampalli, 2019 ) concern text entry methods that increase the usability of apps among visually impaired people. Khusro et al. (2019) provides a novel contribution to the literature regarding considerations that can be used as guidelines for designing a user-friendly and semantically enriched user interface for blind people. An experiment in Cho & Kim (2017) was conducted comparing the two-button mobile interface usability with the one-finger method and voiceover. Leporini & Palmucci (2018) gathered information on the interaction challenges faced by visually impaired people when answering questions on a mobile touch-screen device, investigated possible solutions to overcome the accessibility and usability challenges.

E. Gestures

In total, 3 of 60 studies discuss gestures in usability. Alnfiai & Sampalli (2017) compared the performance of BrailleEnter, a gesture based input method to the Swift Braille keyboard, a method that requires finding the location of six buttons representing braille dot, while Buzzi et al. (2017) and Smaradottir, Martinez & Haland (2017) provide an analysis of gesture performance on touch screens among visually impaired people.

F. Audio guidance

People with visual impairment primarily depend on audio guidance forms in their daily lives; accordingly, audio feedback helps guide them in their interaction with mobile applications.

Four studies discussed the use of audio guidance in different contexts: one in navigation ( Gintner et al., 2017 ), one in games ( Ara’ujo et al., 2017 ), one in reading ( Sabab & Ashmafee, 2016 ), and one in videos ( Façanha et al., 2016 ). These studies were developed and evaluated based on usability and accessibility of the audio guidance for people with visual impairments and aimed to utilize mobile applications to increase the enjoyment and independence of such individuals.

G. Navigation

Navigation is a common issue that visually impaired people face. Indoor navigation is widely discussed in the literature. Nair et al. (2020) , Al-Khalifa & Al-Razgan (2016) and De Borba Campos et al. (2015) discuss how we can develop indoor navigation applications for visually impaired people. Outdoor navigation is also common in the literature, as seen in Darvishy et al. (2020) , Hossain, Qaiduzzaman & Rahman (2020) , Long et al. (2016) , Prerana et al. (2019) and Bandukda et al. (2020) . For example, in Darvishy et al. (2020) , Touch Explorer, an accessible digital map application, was presented to alleviate many of the problems faced by people with visual impairments while using highly visually oriented digital maps. Primarily, it focused on using non-visual output modalities like voice output, everyday sound, and vibration feedback. Issues with navigation applications were also presented in Maly et al. (2015) . Kameswaran et al. (2020) discussed commonly used technologies in navigation applications for blind people and highlighted the importance of using complementary technologies to convey information through different modalities to enhance the navigation experience. Interactive sonification of images for navigation has also been shown in Skulimowski et al. (2019) .

In this section, the research questions are addressed in detail to clearly achieve the research objective. Also, a detailed overview of each theme will be mentioned below.

Answers to the research questions

This section will answer the research question proposed:

RQ1: What existing UVI issues did authors try to solve with mobile devices?

Mobile applications can help people with visual impairments in their daily activities, such as navigation and writing. Additionally, mobile devices may be used for entertainment purposes. However, people with visual impairments face various difficulties while performing text entry operations, text selection, and text manipulation on mobile applications ( Niazi et al., 2016 ). Thus, the authors of the studies tried to increase touch screens’ usability by producing prototypes or simple systems and doing usability testing to understand the UX of people with visual impairments.

RQ2: What is the role of mobile devices in solving those issues?

Mobile phones are widely used in modern society, especially among users with visual impairments; they are considered the most helpful tool for blind users to communicate with people worldwide ( Smaradottir, Martinez & Håland, 2017 ). In addition, the technology of touch screen assistive technology enables speech interaction between blind people and mobile devices and permits the use of gestures to interact with a touch user interface. Assistive technology is vital in helping people living with disabilities perform actions or interact with systems ( Niazi et al., 2016 ).

RQ3: What are the publication trends on the usability of mobile applications among the visually impaired?

As shown in Fig. 8 below, research into mobile applications’ usability for the visually impaired has increased in the last five years, with a slight dip in 2018. Looking at the most frequent themes, we find that “Assistive Devices” peaked in 2017, while “Navigation” and “Accessibility” increased significantly in 2020. On the other hand, we see that the prevalence of “Daily Activities” stayed stable throughout the research years. The term “Audio Guidance” appeared in 2016 and 2017 and has not appeared in the last three years. “Gestures” also appeared only in 2017. “Screen Layout Division” was present in the literature in the last five years and increased in 2019 but did not appear in 2020.

Figure 8: Publication trends over time.

Rq4: what are the current research limitations and future research directions regarding usability among the visually impaired.

We divide the answer to this question into two sections: first, we will discuss limitations; then, we will discuss future work for each proposed theme.

A. Limitations

Studies on the usability of mobile applications for visually impaired users in the literature have various limitations, and most of them were common among the studies. These limitations were divided into two groups. The first group concerns proposed applications; for example, Rahman, Anam & Yeasin (2017) , Oliveira et al. (2018) and Madrigal-Cadavid et al. (2019) faced issues regarding camera applications in mobile devices due to the considerable effort needed for its usage and being heavily dependent on the availability of the internet. The other group of studies, Rodrigues et al. (2015) , Leporini & Palmucci (2018) , Alnfiai & Sampalli (2016) , and Ara’ujo et al. (2017) , have shown limitations in visually impaired users’ inability to comprehend a graphical user interface. Alnfiai & Sampalli (2017) and Alnfiai & Sampalli (2019) evaluated new braille input methods and found that the traditional braille keyboard, where knowing the exact position of letters QWERTY is required, is limited in terms of usability compared to the new input methods. Most studies faced difficulties regarding the sample size and the fact that many of the participants were not actually blind or visually impaired but only blindfolded. This likely led to less accurate results, as blind or visually impaired people can provide more useful feedback as they experience different issues on a daily basis and are more ideal for this type of study. So, the need for a good sample of participants who actually have this disability is clear to allow for better evaluation results and more feedback and recommendations for future research.

B. Future work

A commonly discussed future work in the chosen literature is to increase the sample sizes of people with visual impairment and focus on various ages and geographical areas to generalize the studies. Table 2 summarizes suggestions for future work according to each theme. Those future directions could inspire new research in the field.

RQ5: What is the focus of research on usability for visually impaired people, and what are the research outcomes in the studies reviewed?

There are a total of 60 outcomes in this research. Of these, 40 involve suggestions to improve usability of mobile applications; four of them address problems that are faced by visually impaired people that reduce usability. Additionally, 16 of the outcomes are assessments of the usability of the prototype or model. Two of the results are recommendations to improve usability. Finally, the last two outcomes are hardware solutions that may help the visually impaired perform their daily activities. Figure 9 illustrates these numbers.

Figure 9: Outcomes of studies.

Overview of the reviewed studies.

In the following subsections, we summarize all the selected studies based on the classified theme: accessibility, assistive devices, daily activities, screen division layout, gestures, audio guidance, and navigation. The essence of the studies will be determined, and their significance in the field will be explored.

For designers dealing with mobile applications, it is critical to determine and fix accessibility issues in the application before it is delivered to the users ( Khowaja et al., 2019 ). Accessibility refers to giving the users the same user experience regardless of ability. In Khowaja et al. (2019) and Carvalho et al. (2018) , the researchers focused on comparing the levels of accessibility and usability in different applications. They had a group of visually impaired users and a group of sighted users test out the applications to compare the number and type of problems they faced and determine which applications contained the most violations. Because people with visual impairments cannot be ignored in the development of mobile applications, many researchers have sought solutions for guaranteeing accessibility. For example, in Qureshi & Hooi-Ten Wong (2020) , the study contributed to producing a new, effective design for mobile applications based on the suggestions of people with visual impairments and with the help of two expert mobile application developers. In Khan, Khusro & Alam (2018) , an adaptive user interface model for visually impaired people was proposed and evaluated in an empirical study with 63 visually impaired people. In Aqle, Khowaja & Al-Thani (2020) , the researchers proposed a new web search interface for users with visual impairments that is based on discovering concepts through formal concept analysis (FCA). Users interact with the interface to collect concepts, which are then used as keywords to narrow the search results and target the web pages containing the desired information with minimal effort and time. The usability of the proposed search interface (InteractSE) was evaluated by experts in the field of HCI and accessibility, with a set of heuristics by Nielsen and a set of WCAG 2.0 guidelines.

In Darvishy, Hutter & Frei (2019) , the researchers proposed a solution for making mobile map applications accessible for people with blindness or visual impairment. They suggested replacing forests in the map with green color and birds’ sound, replacing water with blue color and water sounds, replacing streets with grey color and vibration, and replacing buildings with yellow color and pronouncing the name of the building. The prototype showed that it was possible to explore a simple map through vibrations, sounds, and speech.

In Morris et al. (2016) the researchers utilized a multi-faceted technique to investigate how and why visually impaired individuals use Twitter and the difficulties they face in doing so. They noted that Twitter had become more image-heavy over time and that picture-based tweets are largely inaccessible to people with visual impairments. The researchers then made several suggestions for how Twitter could be amended to continue to be usable for people with visual impairments.

The researchers in Paiva et al. (2020) focused on how to evaluate proposed methods for ensuring the accessibility and usability of mobile applications. Their checklist, Acc-MobileCheck, contains 47 items that correspond to issues related to comprehension (C), operation (O), perception (P), and adaptation (A) in mobile interface interaction. To validate Acc-MobileCheck, it was reviewed by five experts and three developers and determined to be effective. In Pereda, Murillo & Paz (2020) , the authors also suggest a set of heuristics to evaluate the accessibility of mobile e-commerce applications for visually impaired people. Finally, McKay (2017) conducted an accessibility test for hybrid mobile apps and found that students with blindness faced many barriers to access based on how they used hybrid mobile applications. While hybrid apps can allow for increased time for marketing, this comes at the cost of app accessibility for people with disabilities.

A significant number of people with visual impairments use state-of-the-art software to perform tasks in their daily lives. These technologies are made up of electronic devices equipped with sensors and processors that can make intelligent decisions.

One of the most important and challenging tasks in developing such technologies is to create a user interface that is appropriate for the sensorimotor capabilities of users with blindness ( Csapó et al., 2015 ). Several new hardware tools have proposed to improve the quality of life for people with visual impairments. Three tools were presented in this SLR: a smart stick that can notify the user of any obstacle, helping them to perform tasks easily and efficiently ( Bharatia, Ambawane & Rane, 2019 ), and an eye that can allow users to detect colors (medical evaluation is still required) ( Lewis et al., 2016 ).

The purpose of the study in Griffin-Shirley et al. (2017) was to understand how people with blindness use smartphone applications as assistive technology and how they perceive them in terms of accessibility and usability. An online survey with 259 participants was conducted, and most of the participants rated the applications as useful and accessible and were satisfied with them.

The researchers in Rahman, Anam & Yeasin (2017) designed and implemented EmoAssist, which is a smartphone application that assists with natural dyadic conversations and aims to promote user satisfaction by providing options for accessing non-verbal communication that predicts behavioural expressions and contains interactive dimensions to provide valid feedback. The usability of this application was evaluated in a study with ten people with blindness where several tools were applied in the application. The study participants found that the usability of EmoAssist was good, and it was an effective assistive solution.

This theme contains two main categories: braille-based application studies and applications to enhance the independence of VIU. Both are summarized below.

1- Braille-based applications

Braille is still the most popular method for assisting people with visual impairments in reading and studying, and most educational mobile phone applications are limited to sighted people. Recently, however, some researchers have developed assistive education applications for students with visual impairments, especially those in developing countries. For example, in India, the number of children with visual impairments is around 15 million, and only 5% receive an education ( Gokhale et al., 2017 ). Three of the braille studies focused on education: ( Nahar, Sulaiman & Jaafar, 2020 ; Nahar, Jaafar & Sulaiman, 2019 , and Araújo et al., 2016 ). These studies all used smartphone touchscreens and action gestures to gain input from the student, and then output was provided in the form of audio feedback. In Nahar, Sulaiman & Jaafar (2020) , vibrational feedback was added to guide the users. The participants in Nahar, Sulaiman & Jaafar (2020) ; Nahar, Jaafar & Sulaiman (2019) , and Araújo et al. (2016) included students with blindness of visual impairment and their teachers. The authors in Nahar, Sulaiman & Jaafar (2020) , Nahar, Jaafar & Sulaiman (2019) evaluated the usability of their applications following the same criteria (efficiency, learnability, memorability, errors, and satisfaction). The results showed that in Nahar, Sulaiman & Jaafar (2020) , Nahar, Jaafar & Sulaiman (2019) , and Araújo et al. (2016) , the applications met the required usability criteria. The authors in Gokhale et al. (2017) presented a braille-based solution to help people with visual impairments call and save contacts. A braille keypad on the smartphone touchscreen was used to gain input from the user, which was then converted into haptic and auditory feedback to let the user know what action was taken. The usability of this application was considered before it was designed. The participants’ responses were positive because this kind of user-centric design simplifies navigation and learning processes.

2- Applications to enhance the independence of people with visual impairments

The authors in the studies explored in this section focused on building applications that enhance independence and autonomy in daily life activities for users with visual impairments.

In Vitiello et al. (2018) , the authors presented their mobile application, an assistive solution for visually impaired users called “Crania”, which uses machine learning techniques to help users with visual impairments get dressed by recognizing the colour and texture of their clothing and suggesting suitable combinations. The system provides feedback through voice synthesis. The participants in the study were adults and elderly people, some of whom were completely blind and the rest of whom had partial sight. After testing for usability, all the participants with blindness agreed that using the application was better than their original method, and half of the participants with partial sight said the same thing. At the end of the study, the application was determined to be accessible and easy to use.

In Kunaratana-Angkul, Wu & Shin-Renn (2020) , an application which allows elderly people to measure low vision status at home through their smartphones instead of visiting hospitals was tested, and most of the participants considered it to be untrustworthy because the medical information was insufficient. Even when participants were able to learn how to use the application, most of them were still confused while using it and needed further instruction.

In Ghidini et al. (2016) , the authors studied the habits of people with visual impairments when using their smartphones in order to develop an electronic calendar with different interaction formats, such as voice commands, touch, and vibration interaction. The authors presented the lessons learned and categorized them based on usability heuristics such as feedback, design, user freedom and control, and recognition instead of remembering.

In Madrigal-Cadavid et al. (2019) , the authors developed a drug information application for people with visual impairments to help them access the labels of medications. The application was developed based on a user-centered design process. By conducting a usability test, the authors recognized some usability issues for people with visual impairments, such as difficulty in locating the bar code. Given this, a new version will include a search function that is based on pictures. The application is searched by capturing the bar code or text or giving voice commands that allow the user to access medication information. The participants were people with visual impairments, and most of them required assistance using medications before using the application. This application will enhance independence for people with visual impairments in terms of using medications.

In Marques, Carriço & Guerreiro (2015) , an authentication method is proposed for users with visual impairments that allows them to protect their passwords. It is not secure when blind or visually impaired users spell out their passwords or enter the numbers in front of others, and the proposed solution allows the users to enter their password with one hand by tapping the screen. The blind participants in this study demonstrated that this authentication method is usable and supports their security needs.

In Oliveira et al. (2018) , the author noted that people with visual impairments face challenges in reading, thus he proposed an application called LeR otulos. This application was developed and evaluated for the Android operating system and recognizes text from photos taken by the mobile camera and converts them into an audio description. The prototype was designed to follow the guidelines and recommendations of usability and accessibility. The requirements of the application are defined based on the following usability goals: the steps are easy for the user to remember; the application is efficient, safe, useful, and accessible; and user satisfaction is achieved.

Interacting with talkback audio devices is still difficult for people with blindness, and it is unclear how much benefit they provide to people with visual impairments in their daily activities. The author in Rodrigues et al. (2015) investigates the smartphone adoption process of blind users by conducting experiments, observations, and weekly interviews. An eight-week study was conducted with five visually impaired participants using Samsung and an enabled talkback 2 screen reader. Focusing on understanding the experiences of people with visual impairments when using touchscreen smartphones revealed accessibility and usability issues. The results showed that the participants have difficulties using smartphones because they fear that they cannot use them properly, and that impacts their ability to communicate with family. However, they appreciate the benefits of using smartphones in their daily activities, and they have the ability to use them.

People with visual impairments encounter various challenges identifying and locating non-visual items on touch screen interfaces, such as phones and tablets. Various specifications for developing a user interface for people with visual impairments must be met, such as having touch screen division to enable people with blindness to easily and comfortably locate objects and items that are non-visual on the screen ( Khusro et al., 2019 ). Article ( Khusro et al., 2019 ) highlighted the importance of aspects of the usability analysis, such as screen partitioning, to meet specific usability requirements, including orientation, consistency, operation, time consumption, and navigation complexity when users want to locate objects on their touchscreen. The authors of Khan & Khusro (2019) describe the improvements that people with blindness have experienced in using the smartphone while performing their daily tasks. This information was determined through an empirical study with 41 people with blindness who explained their user and interaction experiences operating a smartphone.

The authors in Palani et al. (2018) provide design guidelines governing the accurate display of haptically perceived graphical materials. Determining the usability parameters and the various cognitive abilities required for optimum and accurate use of device interfaces is crucial. Also the authors of Grussenmeyer & Folmer (2017) highlight the importance of usability and accessibility of smartphones and touch screens for people with visual impairments. The primary focus in Leporini & Palmucci (2018) is on interactive tasks used to finish exercises and to answer questionnaires or quizzes. These tools are used for evaluation tests or in games. When using gestures and screen readers to interact on a mobile device, difficulties may arise ( Leporini & Palmucci, 2018 ), The study has various objectives, including gathering information on the difficulties encountered by people with blindness during interactions with mobile touch screen devices to answer questions and investigating practicable solutions to solve the detected accessibility and usability issues. A mobile app with an educational game was used to apply the proposed approach. Moreover, in Alnfiai & Sampalli (2016) and Niazi et al. (2016) , an analysis of the single-tap braille keyboard created to help people with no or low vision while using touch screen smartphones was conducted. The technology used in Alnfiai & Sampalli (2016) was the talkback service, which provides the user with verbal feedback from the application, allowing users with blindness to key in characters according to braille patterns. To evaluate single tap braille, it was compared to the commonly used QWERTY keyboard. In Niazi et al. (2016) , it was found that participants adapted quickly to single-tap Braille and were able to type on the touch screen within 15 to 20 min of being introduced to this system. The main advantage of single tap braille is that it allows users with blindness to enter letters based on braille coding, which they are already familiar with. The average error rate is lower using single-tap Braille than it is on the QWERTY keyboard. The authors of Niazi et al. (2016) found that minimal typing errors were made using the proposed keypad, which made it an easier option for people with blindness ( Niazi et al., 2016 ). In Cho & Kim (2017) , the authors describe new text entry methods for the braille system including a left touch and a double touch scheme that form a two-button interface for braille input so that people with visual impairments are able to type textual characters without having to move their fingers to locate the target buttons.

One of the main problems affecting the visually impaired is limited mobility for some gestures. We need to know what gestures are usable by people with visual impairments. Moreover, the technology of assistive touchscreen-enabled speech interaction between blind people and mobile devices permits the use of gestures to interact with a touch user interface. Assistive technology is vital in helping people living with disabilities to perform actions or interact with systems. Smaradottir, Martinez & Haland (2017) analyses a voiceover screen reader used in Apple Inc.’s products. An assessment of this assistive technology was conducted with six visually impaired test participants. The main objectives were to pinpoint the difficulties related to the performance of gestures applicable in screen interactions and to analyze the system’s response to the gestures. In this study, a user evaluation was completed in three phases. The first phase entailed training users regarding different hand gestures, the second phase was carried out in a usability laboratory where participants were familiarized with technological devices, and the third phase required participants to solve different tasks. In Knutas et al. (2015) , the vital feature of the system is that it enables the user to interactively select a 3D scene region for sonification by merely touching the phone screen. It uses three different modes to increase usability. Alnfiai & Sampalli (2017) explained a study done to compare the use of two data input methods to evaluate their efficiency with completely blind participants who had prior knowledge of braille. The comparison was made between the braille enter input method that uses gestures and the swift braille keyboard, which necessitates finding six buttons representing braille dots. Blind people typically prefer rounded shapes to angular ones when performing complex gestures, as they experience difficulties performing straight gestures with right angles. Participants highlighted that they experienced difficulties particularly with gestures that have steep or right angles. In Buzzi et al. (2017) , 36 visually impaired participants were selected and split into two groups of low-vision and blind people. They examined their touch-based gesture preferences in terms of the number of strokes, multitouch, and shape angles. For this reason, a wireless system was created to record sample gestures from various participants simultaneously while monitoring the capture process.

People with visual impairment typically cannot travel without guidance due to the inaccuracy of current navigation systems in describing roads and especially sidewalks. Thus, the author of Gintner et al. (2017) aims to design a system to guide people with visual impairments based on geographical features and addresses them through a user interface that converts text to audio using a built-in voiceover engine (Apple iOS). The system was evaluated positively in terms of accessibility and usability as tested in a qualitative study involving six participants with visual impairment.

Based on challenges faced by visually impaired game developers, Ara’ujo et al. (2017) provides guidance for developers to provide accessibility in digital games by using audio guidance for players with visual impairments. The interactions of the player can be conveyed through audio and other basic mobile device components with criteria focused on the game level and speed adjustments, high contrast interfaces, accessible menus, and friendly design. Without braille, people with visual impairments cannot read, but braille is expensive and takes effort, and so it is important to propose technology to facilitate reading for them. In Sabab & Ashmafee (2016) , the author proposed developing a mobile application called “Blind Reader” that reads an audio document and allows the user to interact with the application to gain knowledge. This application was evaluated with 11 participants, and the participants were satisfied with the application. Videos are an important form of digital media, and unfortunately people with visual impairment cannot access these videos. Therefore, Façanha et al. (2016) aims to discover sound synthesis techniques to maximize and accelerate the production of audio descriptions with low-cost phonetic description tools. This tool has been evaluated based on usability with eight people and resulted in a high acceptance rate among users.

1- Indoor navigation

Visually impaired people face critical problems when navigating from one place to another. Whether indoors or outdoors, they tend to stay in one place to avoid the risk of injury or seek the help of a sighted person before moving ( Al-Khalifa & Al-Razgan, 2016 ). Thus, aid in navigation is essential for those individuals. In Nair et al. (2020) , Nair developed an application called ASSIST, which leverages Bluetooth low energy (BLE) beacons and augmented reality (AR) to help visually impaired people move around cluttered indoor places ( e.g. , subways) and provide the needed safe guidance, just like having a sighted person lead the way. In the subway example, the beacons will be distributed across the halls of the subway and the application will detect them. Sensors and cameras attached to the individual will detect their exact location and send the data to the application. The application will then give a sequence of audio feedback explaining how to move around the place to reach a specific point ( e.g. , “in 50 ft turn right”, “now turn left”, “you will reach the destination in 20 steps”). The application also has an interface for sighted and low-vision users that shows the next steps and instructions. A usability study was conducted to test different aspects of the proposed solution. The majority of the participants agreed that they could easily reach a specified location using the application without the help of a sighted person. A survey conducted to give suggestions from the participants for future improvements showed that most participants wanted to attach their phones to their bodies and for the application to consider the different walking speeds of users. They were happy with the audio and vibration feedback that was given before each step or turn they had to take.

In Al-Khalifa & Al-Razgan (2016) , the main purpose of the study was to provide an Arabic-language application for guidance inside buildings using Google Glass and an associated mobile application. First, the building plan must be set by a sighted person who configures the different locations needed. Ebsar will ask the map builder to mark each interesting location with a QR code and generate a room number, and the required steps and turns are tracked using the mobile device’s built-in compass and accelerometer features. All of these are recorded in the application for the use of a visually impaired individual, and at the end, a full map is generated for the building. After setting the building map, a user can navigate inside the building with the help of Ebsar, paired with Google Glass, for input and output purposes. The efficiency, effectiveness, and levels of user satisfaction with this solution were evaluated. The results showed that the errors made were few, indicating that Ebsar is highly effective. The time consumed in performing tasks ranged from medium to low depending on the task; this can be improved later. Interviews with participants indicated the application’s ease of use. De Borba Campos et al. (2015) shows an application simulating a museum map for people with visual impairments. It discusses whether mental maps and interactive games can be used by people with visual impairments to recognize the space around them. After multiple usability evaluation sessions, the mobile application showed high efficiency among participants in understanding the museum’s map without repeating the visitation. The authors make a few suggestions based on feedback from the participants regarding enhancing usability, including using audio cues, adding contextual help to realise the activities carried around in a space, and focusing on audio feedback instead of graphics.

2- Outdoor navigation

Outdoor navigation is also commonly discussed in the literature. In Darvishy et al. (2020) , Touch Explorer was presented to alleviate many of the problems faced by visually impaired people in navigation by developing a non-visual mobile digital map. The application relies on three major methods of communication with the user: voice output, vibration feedback, and everyday sounds. The prototype was developed using simple abstract visuals and mostly relies on voice for explanation of the content. Usability tests show the great impact the prototype had on the understanding of the elements of the map. Few suggestions were given by the participants to increase usability, including GPS localization to locate the user on the map, a scale element for measuring the distance between two map elements, and an address search function.

In Hossain, Qaiduzzaman & Rahman (2020) , a navigation application called Sightless Helper was developed to provide a safe navigation method for people with visual impairments. It relies on footstep counting and GPS location to provide the needed guidance. It can also ensure safe navigation by detect objects and unsafe areas and can detect unusual shaking of the user and alert an emergency contact about the problem. The user interaction categories are voice recognition, touchpad, buttons, and shaking sensors. After multiple evaluations, the application was found to be useful in different scenarios and was considered usable by people with visual impairments. The authors in Long et al. (2016) propose an application that uses both updates from users and information about the real world to help visually impaired people navigate outdoor settings. After interviews with participants, some design goals were set, including the ability to tag an obstacle on the map, check the weather, and provide an emergency service. The application was evaluated and was found to be of great benefit; users made few errors and found it easy to use. In Prerana et al. (2019) , a mobile application called STAVI was presented to help visually impaired people navigate from a source to a destination safely and avoid issues of re-routing. The application depends on voice commands and voice output. The application also has additional features, such as calling, messages, and emergency help. The authors in Bandukda et al. (2020) helped people with visual impairments explore parks and natural spaces using a framework called PLACES. Different interviews and surveys were conducted to identify the issues visually impaired people face when they want to do any leisure activity. These were considered in the development of the framework, and some design directions were presented, such as the use of audio to share an experience.

3- General issues

The authors in Maly et al. (2015) discuss implementing an evaluation model to assess the usability of a navigation application and to understand the issues of communication with mobile applications that people with visual impairments face. The evaluation tool was designed using a client–server architecture and was applied to test the usability of an existing navigation application. The tool was successful in capturing many issues related to navigation and user behavior, especially the issue of different timing between the actual voice instruction and the position of the user. The authors in Kameswaran et al. (2020) conducted a study to find out which navigation technologies blind people can use and to understand the complementarity between navigation technologies and their impact on navigation for visually impaired users. The results of the study show that visually impaired people use both assistive technologies and those designed for non-visually impaired users. Improving voice agents in navigation applications was discussed as a design implication for the visually impaired. In Skulimowski et al. (2019) , the authors show how interactive sonification can be used in simple travel aids for the blind. It uses depth images and a histogram called U-depth, which is simple auditory representations for blind users. The vital feature of this system is that it enables the user to interactively select a 3D scene region for sonification by touching the phone screen. This sonic representation of 3D scenes allows users to identify the environment’s general appearance and determine objects’ distance. The prototype structure was tested by three blind individuals who successfully performed the indoor task. Among the test scenes used included walking along an empty corridor, walking along a corridor with obstacles, and locating an opening between obstacles. However, the results showed that it took a long time for the testers to locate narrow spaces between obstacles.

RQ6: What evaluation methods were used in the studies on usability for visually impaired people that were reviewed?

The most prevalent methods to evaluate the usability of applications were surveys and interviews. These were used to determine the usability of the proposed solutions and obtain feedback and suggestions regarding additional features needed to enhance the usability from the participants’ points of view. Focus groups were also used extensively in the literature. Many of the participants selected were blindfolded and were not actually blind or visually impaired. Moreover, the samples selected for the evaluation methods mentioned above considered the age factor depending on the study’s needs.

Limitation and future work

The limitations of this paper are mainly related to the methodology followed. Focusing on just eight online databases and restricting the search with the previously specified keywords and string may have limited the number of search results. Additionally, a large number of papers were excluded because they were written in other languages. Access limitations were also faced due to some libraries asking for fees to access the papers. Therefore, for future works, a study to expand on the SLR results and reveal the current usability models of mobile applications for the visually impaired to verify the SLR results is needed so that this work contributes positively to assessing difficulties and expanding the field of usability of mobile applications for users with visual impairments.

Conclusions

In recent years, the number of applications focused on people with visual impairments has grown, which has led to positive enhancements in those people’s lives, especially if they do not have people around to assist them. In this paper, the research papers focusing on usability for visually impaired users were analyzed and classified into seven themes: accessibility, daily activities, assistive devices, gestures, navigation, screen division layout, and audio guidance. We found that various research studies focus on accessibility of mobile applications to ensure that the same user experience is available to all users, regardless of their abilities. We found many studies that focus on how the design of the applications can assist in performing daily life activities like braille-based application studies and applications to enhance the independence of VI users. We also found papers that discuss the role of assistive devices like screen readers and wearable devices in solving challenges faced by VI users and thus improving their quality of life. We also found that some research papers discuss limited mobility of some gestures for VI users and investigated ways in which we can know what gestures are usable by people with visual impairments. We found many research papers that focus on improving navigation for VI users by incorporating different output modalities like sound and vibration. We also found various studies focusing on screen division layout. By dividing the screen and focusing on visual impairment-related issues while developing user interfaces, visually impaired users can easily locate the objects and items on the screens. Finally, we found papers that focus on audio guidance to improve usability. The proposed applications use voice-over and speech interactions to guide visually impaired users in performing different activities through their mobiles. Most of the researchers focused on usability in different applications and evaluated the usability issues of these applications with visually impaired participants. Some of the studies included sighted participants to compare the number and type of problems they faced. The usability evaluation was generally based on the following criteria: accessibility, efficiency, learnability, memorability, errors, safety, and satisfaction. Many of the studied applications show a good indication of these applications’ usability and follow the participants’ comments to ensure additional enhancements in usability. This paper aims to provide an overview of the developments on usability of mobile applications for people with visual impairments and use this overview to highlight potential future directions.

Download article

Report a problem.

Common use cases Typos, corrections needed, missing information, abuse, etc

Our promise PeerJ promises to address all issues as quickly and professionally as possible. We thank you in advance for your patience and understanding.

Typo Missing or incorrect metadata Quality: PDF, figure, table, or data quality Download issues Abusive behavior Research misconduct Other issue not listed above

Follow this publication for updates

You can also choose to receive updates via daily or weekly email digests. If you are following multiple publications then we will send you no more than one email per day or week based on your preferences.

Note: You are now also subscribed to the subject areas of this publication and will receive updates in the daily or weekly email digests if turned on. You can add specific subject areas through your profile settings.

Change notification settings or unfollow

Loading ...

Usage since published - updated daily

Top referrals unique visitors

Share this publication, articles citing this paper.

UN sustainable development goals and management of non-instantaneous deteriorating items: a literature review and applications framework

• Original Research
• Published: 01 August 2024

Cite this article

• Pratik Maheshwari 1 ,
• Sachin Kamble 2 ,
• Amine Belhadi 3 &
• Satish Kumar   ORCID: orcid.org/0000-0001-5200-1476 4  

23 Accesses

Explore all metrics

Recent decades have seen a surge in reports of mismanagement and mishandling of non-instantaneous deteriorating items (NIDIs), highlighting sustainability issues, environmental impact, and socio-economic changes in academia and industry. High complexity, continuous processes, integrated dynamics, and adaptability issues challenge the sustainable decision-making process for NIDIs’. Hence, the United Nations has directed 17 Sustainable Development Goals (SDGs) to overcome this ambiguity. This paper examines the challenges in monitoring the targets outlined in SDG 2 (2.3, 2.4, 2(a), 2(b), and 2(c)) and SDG 12 (12.1, 12.3, 12.5, 12.6, 12.7, and 12(a)) of the United Nations’s 2030 goals. The proposed goals could effectively address inventory stocks, appropriate policy selections, perishability, and stochastic market demand. Therefore, this study develops a novel framework by hand-mapping the SDGs and their targets with NIDIs sustainability practices, reflected in its environmental, social, and governance aspects. This paper also includes a systematic literature review (2006 to 2022) to identify sustainable models of NIDIs along with variables, barriers, different mathematical programming approaches, optimal performance criteria, and proposed application framework. Moreover, the findings offer a dual contribution: aiding academic analysis of NIDIs about SDGs and proposing a sustainable modeling framework for practitioners, fostering stakeholder collaboration, and integrating current research trends and agendas for future research directions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save.

• Get 10 units per month
• Download Article/Chapter or eBook
• 1 Unit = 1 Article or 1 Chapter
• Cancel anytime

Price excludes VAT (USA) Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

( Source Scopus database)

Similar content being viewed by others

Value Stream Mapping and Process Indicators Supporting Sustainable Development in Organizations – A Systematic Literature Review

An assessment of requirements in investments, new technologies, and infrastructures to achieve the SDGs

Sustainable Development Goals as a Guideline for Multinational Corporations

Ahmad, B., & Benkherouf, L. (2018). Economic-order-type inventory models for non-instantaneous deteriorating items and backlogging. RAIRO—Operations Research, 52 (3), 895–901.

Article   Google Scholar  

Ai, X.-Y., Zhang, J.-L., & Wang, L. (2017). Optimal joint replenishment policy for multiple non-instantaneous deteriorating items. International Journal of Production Research, 55 (16), 4625–4642.

Ai, X.-Y., Zhang, J.-L., Xu, H.-X., & Wang, L. (2018). Optimal joint replenishment policy for multiple non-instantaneous deteriorating products with deterioration-time-dependent demand. Kongzhi Yu Juece/control and Decision, 33 (4), 671–678.

Google Scholar  

Akhtar, M., Manna, A. K., & Bhunia, A. K. (2023). Optimization of a non-instantaneous deteriorating inventory problem with time and price-dependent demand over a finite time horizon via a hybrid DESGO algorithm. Expert Systems with Applications . https://doi.org/10.1016/j.eswa.2022.118676

Aliabadi, L., Yazdanparast, R., & Nasiri, M. M. (2018). An inventory model for non-instantaneous deteriorating items with credit period and carbon emission sensitive demand: A signal geometric programming approach. International Journal of Management Science and Engineering Management . https://doi.org/10.1080/17509653.2018.1504331

Almathkour, F., & Benkherouf, L. (2022). Optimal policies for finite horizon model with time-varying demand rate, non-instantaneous deterioration, and backlogging. International Journal of Systems Science Operations and Logistics . https://doi.org/10.1080/23302674.2022.2041128

Aly, E., Elsawah, S., & Ryan, M. J. (2022). A review and catalogue to the use of models in enabling the achievement of sustainable development goals (SDG). Journal of cleaner production, 340, 130803

Ardra, S., & Barua, M. K. (2022). Halving food waste generation by 2030: The challenges and strategies of monitoring UN sustainable development goal target 12.3. Journal of Cleaner Production, 380 , 135042.

Bakker, M., Riezebos, J., & Teunter, RuudH. (2012). Review of inventory systems with deterioration since 2001. European Journal of Operational Research, 221 , 275–284.

Bardhan, S., Pal, H., Giri, B.C. (2017). Optimal replenishment policy and preservation technology investment for a non-instantaneous deteriorating item with stock-dependent demand. Operational Research, pp. 1–22.

Becerra, P., Mula, J., & Sanchis, R. (2021). Green supply chain quantitative models for sustainable inventory management: A review. Journal of Cleaner Production, 328 , 129544.

Biglari, S., Beiglary, S., & Arthanari, T. (2022). Achieving sustainable development goals: Fact or fiction?. Journal of Cleaner Production, 332 , 130032.

Burek, J., & Nutter, D. (2018). Life cycle assessment of grocery, perishable, and general merchandise multi-facility distribution center networks. Energy and Buildings, 174 , 388-401.

Cao, Y., Li, Y. M., & Li, Q. S. (2018). Research on pricing and inventory lot-size policies for non-instantaneous deteriorating items with stochastic demand and promotional efforts based on advance payments. Control and Decision, 33 (2), 301–308.

Chakraborty, D., Garai, T., Jana, D. K., & Roy, T. K. (2017). A three-layer supply chain inventory model for non-instantaneous deteriorating items with inflation and delay in payments in a random fuzzy environment. Journal of Industrial and Production Engineering, 34 (6), 407–424.

Chang, C.-T., Cheng, M.-C., & Ouyang, L.-Y. (2011). Optimal pricing and ordering policies for non-instantaneously deteriorating items under order-size-dependent payment delay. Applied Mathematical Modelling, 39 (2), 747–763.

Chang, C.-T., Teng, J.-T., & Goyal, S. K. (2010). Optimal replenishment policies for non-instantaneous deteriorating items with stock-dependent demand. International Journal of Production Economics, 123 (1), 62–68.

Chang, H. J., & Lin, W. F. (2010). A partial backlogging inventory model for non-instantaneous deteriorating items with stock-dependent consumption rate under inflation. Yugoslav Journal of Operations Research, 20 (1), 35–54.

Chapman, C. B., Ward, S. C., Cooper, D. F., & Page, M. J. (1984). Credit policy and inventory control. Journal of Operational Research Society, 35 (12), 1055–1065.

Chatterjee, S., Chaudhuri, R., Shah, M., & Maheshwari, P. (2022). Big data driven innovation for sustaining SME supply chain operation in post COVID-19 scenario: Moderating role of SME technology leadership. Computers & Industrial Engineering, 168 , 108058. https://doi.org/10.1016/j.cie.2022.108058

Chung, K.-J. (2009). A complete proof of the solution procedure for non-instantaneous deteriorating items with permissible delay in payment. Computers and Industrial Engineering, 56 (1), 267–273.

Chung, K.-J. (2010). Using the convexities of total annual relevant costs to determine the optimal cycle times of inventory models for deteriorating items with permissible delay in payments. Computers and Industrial Engineering, 58 (4), 801–808.

Chung, K.-J., Cárdenas-Barrón, L. E., & Ting, P.-S. (2014). An inventory model with non-instantaneous receipt and exponentially deteriorating items for an integrated three layer supply chain system under two levels of trade credit. International Journal of Production Economics, 155 , 310–317.

Daghigh, R., Jabalameli, M., Amiri, A., & Pishvaee, M. (2016). A multi-objective location-inventory model for 3PL providers with sustainable considerations under uncertainty. International Journal of Industrial Engineering Computations, 7 (4), 615–634.

Das, S. C., Manna, A. K., Rahman, M. S., Shaikh, A. A., & Bhunia, A. K. (2021). An inventory model for non-instantaneous deteriorating items with preservation technology and multiple credit periods-based trade credit financing via particle swarm optimization. Soft Computing, 25 (7), 5365–5384. https://doi.org/10.1007/s00500-020-05535-x

Dye, C.-Y. (2013). The effect of preservation technology investment on a non-instantaneous deteriorating inventory model. Omega (united Kingdom), 41 (5), 872–880.

Farughi, H., Khanlarzade, N., & Yegane, B. Y. (2014). Pricing and inventory control policy for non-instantaneous deteriorating items with time- and price-dependent demand and partial backlogging. Decision Science Letters, 3 (3), 325–334.

Galal, N. M., & El-Kilany, K. S. (2016). Sustainable agri-food supply chain with uncertain demand and lead time. International Journal of Simulation Modelling, 15 (3), 485–496.

Garg, C. P. (2022). Modelling and optimisation of reverse logistics network under fuzzy environment. International Journal of Operational Research, 44 (3), 360–388.

Geetha, K. V., & Uthayakumar, R. (2009). Optimal inventory control policy for items with time-dependent demand. American Journal of Mathematical and Management Sciences, 29 (3–4), 457–476.

Geetha, K. V., & Uthayakumar, R. (2010). Economic design of an inventory policy for non-instantaneous deteriorating items under permissible delay in payments. Journal of Computational and Applied Mathematics, 233 (10), 2492–2505.

Ghasemi, N. (2015). Developing EPQ models for non-instantaneous deteriorating items. Journal of Industrial Engineering International, 11 (3), 427–437.

Ghobadi, S. N., Kamalabadi, I. N., & Aghdasi, M. (2012). Joint pricing and inventory control for non instantaneous deteriorating items with stochastic demand. International Journal of Supply Chain Management, 1 (3), 43–53.

Ghoreishi, M., Mirzazadeh, A., & Kamalabadi, I. N. (2014a). Optimal pricing and lot-sizing policies for an economic production quantity model with non-instantaneous deteriorating items, permissible delay in payments, customer returns, and inflation. Proceedings of the Institution of Mechanical Engineers, Part B: Journal of Engineering Manufacture, 228 (12), 1653–1663.

Ghoreishi, M., Mirzazadeh, A., & Weber, G.-W. (2013). Optimal pricing and ordering policy for non-instantaneous deteriorating items under inflation and customer returns. Optimization, 63 (12), 1785–1804.

Ghoreishi, M., Weber, G.-W., & Mirzazadeh, A. (2014b). An inventory model for non-instantaneous deteriorating items with partial backlogging, permissible delay in payments, inflation- and selling price-dependent demand and customer returns. Annals of Operations Research, 226 (1), 221–238.

Gite, S. P. (2011). An inventory model for deteriorating items with stock-dependent demand under inflation and time value of money. International Journal of Agricultural and Statistical Sciences, 7 (2), 463–471.

Gor, A. S., & Patel, H. R. (2018). Optimal selling price and lot size for non-instantaneous deteriorating items with different demand rates and partial backlogging. International Journal of Procurement Management, 11 (3), 295–312.

Goyal, A. K., & Chauhan, A. (2016). An EOQ model for deteriorating items with selling price dependent demand rate with learning effect. Nonlinear Studies, 23 (4), 543–552.

Goyal, S. K., & Giri, B. C. (2001). Recent trends in modeling of deteriorating inventory. European Journal of operational research, 134 (1), 1–16.

Goyal, K., & Kumar, S. (2021). Financial literacy: A systematic review and bibliometric analysis. International Journal of Consumer Studies, 45 (1), 80–105.

Hao, J., Chen, G., Chen, P., Yao, Y., & Zhang, Y. (2013). An inventory model for non-instantaneous deteriorating items under two levels of trade credit and time value of money. Journal of Theoretical and Applied Information Technology, 48 (1), 87–93.

Hasan, M. R., Daryanto, Y., Roy, T. C., & Feng, Y. (2020). Inventory management with online payment and preorder discounts. Industrial Management and Data Systems, 120 (11), 2001–2023. https://doi.org/10.1108/IMDS-05-2020-0314

Hasan, S. T., Wood, M. O., & Singh, S. (2022). Revealing embedded carbon emissions within the comprehensive and progressive agreement for trans-pacific partnership. Economic Systems Research, 34 (3), 294–319.

He, Y., Gong, Y., & Hong, X. (2022). Demand information acquisition and disclosure in a non-instantaneous deteriorating items supply chain. Computers and Industrial Engineering . https://doi.org/10.1016/j.cie.2022.108250

Hosseini, Z. S., Flapper, S. D., & Pirayesh, M. (2022). Sustainable supplier selection and order allocation under demand, supplier availability and supplier grading uncertainties. Computers & Industrial Engineering, 165 , 107811.

Jaggi, C. K., Goel, S. K., & Tiwari, S. (2017a). Two-warehouse inventory model for non-instantaneous deteriorating items under different dispatch policies. Investigation Operational, 38 (4), 343–365.

Jaggi, C. K., Sharma, A., & Tiwari, S. (2015a). Credit financing in economic ordering policies for non-instantaneous deteriorating items with price dependent demand under permissible delay in payments: A new approach. International Journal of Industrial Engineering Computations, 6 (4), 481–502.

Jaggi, C. K., Tiwari, S., & Goel, S. K. (2016). Replenishment policy for non-instantaneous deteriorating items in a two storage facilities under inflationary conditions. International Journal of Industrial Engineering Computations, 7 (3), 489–506.

Jaggi, C. K., Tiwari, S., & Goel, S. K. (2017b). Credit financing in economic ordering policies for non-instantaneous deteriorating items with price dependent demand and two storage facilities. Annals of Operations Research, 248 (1–2), 253–280.

Jaggi, C. K., & Verma, P. (2010). An optimal replenishment policy for non-instantaneous deteriorating items with two storage facilities. International Journal of Services Operations and Informatics, 5 (3), 209–230.

Jaggi, C. K., Verma, P., & Gupta, M. (2015b). Ordering policy for non-instantaneous deteriorating items in two warehouse environment with shortages. International Journal of Logistics Systems and Management, 22 (1), 103–124.

Janssen, T. C., & Sauer, J. (2016). Literature review of deteriorating inventory models by key topics from 2012 to 2015. International Journal of Production Economics, 182 , 86–112.

Johri, A., Joshi, P., Kumar, S., & Joshi, G. (2024). Metaverse for sustainable development in a bibliometric analysis and systematic literature review. Journal of Cleaner Production, 435 , 140610.

Kamble, S., Gunasekaran, A., & Shradha, G. (2019). Achieving sustainable performance in a data-driven agriculture supply chain: A review for research and applications. International Journal of Production Economics . https://doi.org/10.1016/j.ijpe.2019.05.022

Kaur, J., Sharma, R., & Singh, A. P. (2013). An optimal ordering policy for inventory model with non-instantaneous deteriorating items and stock-dependent demand. Applied Mathematical Sciences, 7 (81–84), 4073–4080.

Krommyda, I.-P., Skouri, K., Konstantaras, I., & Ganas, I. (2013). Optimal pricing and replenishment policy for non instantaneous deteriorating items and two levels of storage. Asia-Pacific Journal of Operational Research, 30 (4), 1350001.

Kumar, N., Singh, S. R., & Kumari, R. (2012). Three echelon supply chain inventory model for deteriorating items with limited storage facility and lead-time under inflation. International Journal of Services and Operations Management, 13 (1), 98–118.

Kumar, N., Singh, S. R., & Kumari, R. (2013). Learning effect on an inventory model with two-level storage and partial backlogging under inflation. International Journal of Services and Operations Management, 16 (1), 105–122.

Kumar, S., Colombage, S., & Rao, P. (2017). Research on capital structure determinants: A review and future directions. International Journal of Managerial Finance, 13 (2), 106–132.

Kumar, S., & Rajput, U. S. (2014). A two warehouse inventory model for non-instantaneous deteriorating items with time dependent demand and partial backlogging. International Journal of Applied Engineering Research, 9 (11 SPEC ISSUE), 1347–1356.

Kumar, S., & Singh, A. K. (2015). An optimal replenishment policy for non instantaneous deteriorating items with stock dependent, price decreasing demand and partial backlogging. Indian Journal of Science and Technology, 8 (35), 12.

La Scalia, G., Micale, R., Miglietta, P. P., & Toma, P. (2019). Reducing waste and ecological impacts through a sustainable and efficient management of perishable food based on the Monte Carlo simulation. Ecological Indicators, 97 , 363–371.

Lashgari, M., Taleizadeh, A.A., & Ahmadi, A. (2016). Partial up-stream advanced payment and partial down-stream delayed payment in a three-level supply chain. Annals of Operations Research, 238 , 329–354.

Lashgari, M., Taleizadeh, A. A., & Sadjadi, S. J. (2018). Ordering policies for non-instantaneous deteriorating items under hybrid partial prepayment, partial trade credit and partial backordering. Journal of the Operational Research Society, 69 (8), 1167–1196.

Li, J., Lai, K. K., & Li, Y. (2022). Remanufacturing and low-carbon investment strategies in a closed-loop supply chain under multiple carbon policies. International Journal of Logistics Research and Applications, 1–23.

Li, G., He, X., Zhou, J., & Wu, H. (2019). Pricing, replenishment and preservation technology investment decisions for non-instantaneous deteriorating items. Omega (united Kingdom), 84 , 114–126.

Lim, M. K., Bahr, W., & Leung, S. C. (2013). RFID in the warehouse: A literature analysis (1995–2010) of its applications, benefits, challenges and future trends. International Journal of Production Economics, 145 (1), 409–430.

Mahato, C., & Mahata, G. C. (2022). Optimal replenishment, pricing and preservation technology investment policies for non-instantaneous deteriorating items under two-level trade credit policy. Journal of Industrial and Management Optimization, 18 (5), 3499–3537. https://doi.org/10.3934/jimo.2021123

Maheshwari, P., & Kamble, S. (2022). the application of supply chain digital twin to measure optimal inventory policy. IFAC-PapersOnLine, 55 (10), 2324–2329. https://doi.org/10.1016/j.ifacol.2022.10.055

Maheshwari, P., Kamble, S., Pundir, A., Belhadi, A., Ndubisi, N. O., & Tiwari, S. (2021). Internet of things for perishable inventory management systems: an application and managerial insights for micro, small and medium enterprises. Annals of Operations Research . https://doi.org/10.1007/s10479-021-04277-9

Maihami, R., Ghalehkhondabi, I., & Ahmadi, E. (2021). Pricing and inventory planning for non-instantaneous deteriorating products with greening investment: A case study in beef industry. Journal of Cleaner Production . https://doi.org/10.1016/j.jclepro.2021.126368

Maihami, R., & Karimi, B. (2014). Optimizing the pricing and replenishment policy for non-instantaneous deteriorating items with stochastic demand and promotional efforts. Computers and Operations Research, 51 , 302–312.

Maihami, R., Karimi, B., & Fatemi Ghomi, S. M. T. (2017). Effect of two-echelon trade credit on pricing-inventory policy of non-instantaneous deteriorating products with probabilistic demand and deterioration functions. Annals of Operations Research, 257 (1–2), 237–273.

Magnanini, M. C., & Tolio, T. (2018). A threshold-based control policy for scrap reduction of perishable in-process inventories. Procedia CIRP, 78 , 184–189.

Maihami, R., & Nakhai Kamalabadi, I. (2012). Joint pricing and inventory control for non-instantaneous deteriorating items with partial backlogging and time and price dependent demand. International Journal of Production Economics, 136 (1), 116–122.

Malladi, K. T., & Sowlati, T. (2018). Sustainability aspects in inventory routing problem: A review of new trends in the literature. Journal of Cleaner Production, 197 , 804–814.

Manna, S. K., Lee, C. C., & Chiang, C. (2009). EOQ model for non-instantaneous deteriorating items with time-varying demand and partial backlogging. International Journal of Industrial and Systems Engineering, 4 (3), 241–254.

Mishra, U., Wu, J. Z., Tsao, Y. C., & Tseng, M. L. (2020). Sustainable inventory system with controllable non-instantaneous deterioration and environmental emission rates. Journal of Cleaner Production . https://doi.org/10.1016/j.jclepro.2019.118807

Mishra, V. K. (2014). Deteriorating inventory model with controllable deterioration rate for time-dependent demand and time-varying holding cost. Yugoslav Journal of Operations Research, 24 (1), 87–98.

Mohanty, D. J., Kumar, R. S., & Goswami, A. (2016). A two-warehouse inventory model for non-instantaneous deteriorating items over stochastic planning horizon. Journal of Industrial and Production Engineering, 33 (8), 516–532.

Mu, X., & Antwi-Afari, M. F. (2024). The applications of Internet of Things (IoT) in industrial management: A science mapping review. International Journal of Production Research, 62 (5), 1928–1952.

Nagare, M., Dutta, P., & Suryawanshi, P. (2017). Optimal procurement and discount pricing for single-period non-instantaneous deteriorating products with promotional efforts. Operational Research, 20 , 1–29.

Nagare, M., Dutta, P., & Suryawanshi, P. (2020). Optimal procurement and discount pricing for single-period non-instantaneous deteriorating products with promotional efforts. Operational Research, 20 (1), 89–117. https://doi.org/10.1007/s12351-017-0318-5

Nahmias, S. (1982). Perishable inventory theory: A review. Operations research, 30 (4), 680–708.

Nylund, P. A., Agarwal, N., Probst, C., & Brem, A. (2022). Firm engagement in UN sustainable development goals: Introduction of a constraints map from a corporate reports content analysis. Journal of Cleaner Production, 371 , 133446.

Ouyang, L.-Y., Wu, K.-S., & Yang, C.-T. (2006). A study on an inventory model for non-instantaneous deteriorating items with permissible delay in payments. Computers and Industrial Engineering, 51 (4), 637–651.

Ouyang, L.-Y., Wu, K.-S., & Yang, C.-T. (2008). Retailer’s ordering policy for non-instantaneous deteriorating items with quantity discount, stock-dependent demand and stochastic backorder rate. Journal of the Chinese Institute of Industrial Engineers, 25 (1), 62–72.

Pal, H., Bardhan, S., & Giri, B. C. (2018). Optimal replenishment policy for non-instantaneously perishable items with preservation technology and random deterioration start time. International Journal of Management Science and Engineering Management, 13 (3), 188–199.

Palanivel, M., Mirzazadeh, A., Weber, G.-W., & Nakhai-Kamalabadi, I. (2014). Joint pricing and replenishment decisions for non-instantaneous deteriorating items with partial backlogging, inflation- and selling price-dependent demand and customer returns. Journal of Industrial and Management Optimization, 11 (3), 933–949.

Palanivel, M., Priyan, S., & Mala, P. (2018). Two-warehouse system for non-instantaneous deterioration products with promotional effort and inflation over a finite time horizon. Journal of Industrial Engineering International, 14 (3), 603–612.

Palanivel, M., Sundararajan, R., & Uthayakumar, R. (2016). Two-warehouse inventory model with non-instantaneously deteriorating items, stock-dependent demand, shortages and inflation. Journal of Management Analytics, 3 (2), 152–173.

Palanivel, M., & Uthayakumar, R. (2015). Finite horizon EOQ model for non-instantaneous deteriorating items with price and advertisement dependent demand and partial backlogging under inflation. International Journal of Systems Science, 46 (10), 1762–1773.

Palanivel, M., & Uthayakumar, R. (2016a). Finite horizon EOQ model for non-instantaneous deteriorating items with probabilistic deterioration and partial backlogging under inflation. International Journal of Mathematics in Operational Research, 8 (4), 449–476.

Palanivel, M., & Uthayakumar, R. (2016b). Two-warehouse inventory model for non-instantaneous deteriorating items with stock dependent demand, partial backlogging and inflation over a finite time horizon. International Journal of Services and Operations Management, 25 (2), 206–235.

Palanivel, M., & Uthayakumar, R. (2016c). Two-warehouse inventory model for non-instantaneous deteriorating items with partial backlogging and inflation over a finite time horizon. Opsearch, 53 (2), 278–302.

Palanivel, M., & Uthayakumar, R. (2016d). Two-warehouse inventory model for non–instantaneous deteriorating items with optimal credit period and partial backlogging under inflation. Journal of Control and Decision, 3 (2), 132–150.

Palanivel, M., & Uthayakumar, R. (2017a). Two-warehouse inventory model for non-instantaneous deteriorating items with partial backlogging and permissible delay in payments under inflation. International Journal of Operational Research, 28 (1), 35–69.

Palanivel, M., & Uthayakumar, R. (2017b). An EOQ model for non-instantaneous deteriorating items with partial backlogging and permissible delay in payments under inflation. International Journal of Industrial and Systems Engineering, 26 (1), 63–89.

Pasandideh, S. H. R., Rahbari, M., & Sadati-Keneti, Y. (2023). A Lagrangian relaxation algorithm and hybrid genetic algorithm-black widow optimization for perishable products supply chain with sustainable development goals consideration. Annals of Operations Research, 1–36.

Patel, H., Gor, A. (2018). Salvage value and three variable Weibull deteriorating rate for non-instantaneous deteriorating items.Asian-European Journal of Mathematics, 12(1).

Pérez, F. A., Torres, F., & Mendoza, D. (2019). Stocking and price-reduction decisions for non-instantaneous deteriorating items under time value of money. International Journal of Industrial Engineering Computations, 10 (1), 89–110.

Pundir, A., Devpriya, J., Chakraborty, M., Ganpathy, L. (2019b). Technology integration for improved performance: a case study in digitization of supply chain with integration of internet of things and blockchain technology. In: 2019 IEEE 9th Annual computing and communication workshop and conference (CCWC), Las Vegas, NV, USA, 2019, pp. 0170-0176.

Pundir, A., Jagannath, D., Ganapathy, L. (2019a). Improving supply chain visibility using IoT-Internet of Things. In: IEEE 9th Annual computing and communication workshop and conference (CCWC), Las Vegas, NV, USA, 2019, pp. 0156-0162.

Punj, N., Ahmi, A., Tanwar, A., & Rahim, S. A. (2023). Mapping the field of green manufacturing: A bibliometric review of the literature and research frontiers. Journal of Cleaner Production, 423 , 138729.

Raafat, F. (1991). Survey of literature on continuously deteriorating inventory models. Journal of the Operational Research society, 42 (1), 27–37.

Rabbani, M., Zia, N. P., & Rafiei, H. (2015). Coordinated replenishment and marketing policies for non-instantaneous stock deterioration problem. Computers and Industrial Engineering, 88 , 49–62.

Rabbani, M., Zia, N. P., & Rafiei, H. (2017). Joint optimal inventory, dynamic pricing and advertisement policies for non-instantaneous deteriorating items. RAIRO - Operations Research, 51 (4), 1251–1267.

Rangarajan, K., & Karthikeyan, K. (2017). EOQ models for non-instantaneous/instantaneous deteriorating items with cubic demand rate under inflation and permissible delay in payments. Italian Journal of Pure and Applied Mathematics, 37 , 197–218.

Rapolu, C. N., & Kandpal, D. H. (2020). Joint pricing, advertisement, preservation technology investment and inventory policies for non-instantaneous deteriorating items under trade credit. Opsearch, 57 (2), 274–300. https://doi.org/10.1007/s12597-019-00427-7

Rastogi, M., Kushwah, P., & Singh, S. R. (2018). An inventory model for non-instantaneous deteriorating products having price sensitive demand and partial backlogging of occurring shortages. International Journal of Operations and Quantitative Management, 24 (1), 59–73.

Rezagholifam, M., Sadjadi, S. J., Heydari, M., & Karimi, M. (2022). Optimal pricing and ordering strategy for non-instantaneous deteriorating items with price and stock sensitive demand and capacity constraint. International Journal of Systems Science: Operations and Logistics, 9 (1), 121–132. https://doi.org/10.1080/23302674.2020.1833259

Samanta, A., & Pal, M. (2015). Periodic review inventory policy for non-instantaneous deteriorating items with time dependent deterioration rate. Pakistan Journal of Statistics and Operation Research, 11 (3), 409–419.

Saunders, M. N., & Bezzina, F. (2016). Reflections on conceptions of research methodology among management academics. European Management Journal, 33 (5), 297–304.

Shah, N. H., Soni, H. N., & Patel, K. A. (2013). Optimizing inventory and marketing policy for non-instantaneous deteriorating items with generalized type deterioration and holding cost rates. Omega (united Kingdom), 41 (2), 421–430.

Shaikh, A. A., Cárdenas-Barrón, L. E., & Tiwari, S. (2017). A two-warehouse inventory model for non-instantaneous deteriorating items with interval-valued inventory costs and stock-dependent demand under inflationary conditions. Neural Computing and Applications, 31 , 1–18.

Singh, S. R., & Rathore, H. (2014). An inventory model for deteriorating item with reliability consideration and trade credit. Pakistan Journal of Statistics and Operation Research, 10 (3), 349–360.

Singh, S., Singh, S. R., & Sharma, S. (2017). A partially backlogged EPQ model with demand dependent production and non-instantaneous deterioration. International Journal of Mathematics in Operational Research, 10 (2), 211–228.

Skaf, L., Franzese, P. P., Capone, R., & Buonocore, E. (2021). Unfolding hidden environmental impacts of food waste: An assessment for fifteen countries of the world. Journal of Cleaner Production, 310 , 127523.

Soni, H. N. (2013). Optimal replenishment policies for non-instantaneous deteriorating items with price and stock sensitive demand under permissible delay in payment. International Journal of Production Economics, 146 (1), 259–268.

Soni, H. N., Gor, A. S., & Patel, H. R. (2018). Vendor managed inventory model for non-instantaneous deteriorating product with quadratic demand allowing partial backlogging. Uncertain Supply Chain Management, 6 (3), 321–334.

Soni, H. N., & Patel, K. A. (2012). Optimal pricing and inventory policies for non-instantaneous deteriorating items with permissible delay in payment: Fuzzy expected value model. International Journal of Industrial Engineering Computations, 3 (3), 281–300.

Soni, H. N., & Patel, K. A. (2013). Joint pricing and replenishment policies for non-instantaneous deteriorating items with imprecise deterioration free time and credibility constraint. Computers and Industrial Engineering, 66 (4), 944–951.

Soni, H. N., & Suthar, D. N. (2018). Pricing and inventory decisions for non-instantaneous deteriorating items with price and promotional effort stochastic demand. Journal of Control and Decision . https://doi.org/10.1080/23307706.2018.1478327

Soni, H. N., & Suthar, D. N. (2020). Optimal pricing and replenishment policy for non-instantaneous deteriorating items with varying rate of demand and partial backlogging. Opsearch, 57 (3), 986–1021. https://doi.org/10.1007/s12597-020-00451-y

Sugapriya, C., & Jeyaraman, K. (2008). An EPQ model for non-instantaneous deteriorating item in which holding cost varies with time. Electronic Journal of Applied Statistical Analysis, 1 (1), 16–23.

Sundararajan, R., Palanivel, M., & Uthayakumar, R. (2021). An EOQ model of non-instantaneous deteriorating items with price, time-dependent demand and backlogging. Journal of Control and Decision, 8 (2), 135–154. https://doi.org/10.1080/23307706.2019.1656560

Tashakkor, N., Mirmohammadi, S. H., & Iranpoor, M. (2018). Joint optimization of dynamic pricing and replenishment cycle considering variable non-instantaneous deterioration and stock-dependent demand. Computers and Industrial Engineering, 123 , 232–241.

Tat, R., Taleizadeh, A. A., & Esmaeili, M. (2013). Developing economic order quantity model for non-instantaneous deteriorating items in vendor-managed inventory (VMI) system. International Journal of Systems Science, 46 (7), 1257–1268

Tat, R., Taleizadeh, A. A., & Esmaeili, M. (2015). Developing economic order quantity model for non-instantaneous deteriorating items in vendor-managed inventory (VMI) system. International Journal of Systems Science, 46 (7), 1257–1268.

Tayal, S., Singh, S. R., Sharma, R., & Singh, A. P. (2015). An EPQ model for non-instantaneous deteriorating item with time dependent holding cost and exponential demand rate. International Journal of Operational Research, 23 (2), 145–162.

Tayyab, M., Habib, M. S., Jajja, M. S. S., & Sarkar, B. (2022). Economic assessment of a serial production system with random imperfection and shortages: A step towards sustainability. Computers & Industrial Engineering, 171 , 108398.

Thangam, A., & Uthayakumar, R. (2008). Evaluation of supply chain structures using a delayed product differentiation process. International Journal of Advanced Manufacturing Technology, 39 (3–4), 401–408.

Ting, P.-S., & Chung, K.-J. (1994). Inventory replenishment policy for deteriorating items with a linear trend in demand considering shortages. International Journal of Operations & Production Management, 14 (8), 102–110.

Tiwari, S., Cárdenas-Barrón, L. E., Khanna, A., & Jaggi, C. K. (2016). Impact of trade credit and inflation on retailer’s ordering policies for non-instantaneous deteriorating items in a two-warehouse environment. International Journal of Production Economics, 176 , 154–169.

Tiwari, S., Jaggi, C. K., Bhunia, A. K., Shaikh, A. A., & Goh, M. (2017). Two-warehouse inventory model for non-instantaneous deteriorating items with stock-dependent demand and inflation using particle swarm optimization. Annals of Operations Research, 254 (1–2), 401–423.

Tiwari, S., Jaggi, C. K., Gupta, M., & Cárdenas-Barrón, L. E. (2018). Optimal pricing and lot-sizing policy for supply chain system with deteriorating items under limited storage capacity. International Journal of Production Economics, 200 , 278–290.

Tranfield, D., Denyer, D., & Smart, P. (2006). Towards a methodology for developing evidence-informed management knowledge by means of systematic review. British Journal of Management, 14 , 207–222.

Tripathy, M., Sharma, G., & Sharma, A. K. (2022). An EOQ inventory model for non-instantaneous deteriorating item with constant demand under progressive financial trade credit facility. Opsearch, 59 (4), 1215–1243. https://doi.org/10.1007/s12597-022-00573-5

Tsao, Y.-C. (2016). Joint location, inventory, and preservation decisions for non-instantaneous deterioration items under delay in payments. International Journal of Systems Science, 47 (3), 572–585.

Tsao, Y.-C., Zhang, Q., Fang, H.-P., & Lee, P.-L. (2017). Two-tiered pricing and ordering for non-instantaneous deteriorating items under trade credit. Operational Research, 19 , 1–20.

Tyagi, A. P., Pandey, R. K., & Singh, S. (2014). An optimal replenishment policy for non-instantaneous deteriorating items with stock-dependent demand and variable holding cost. International Journal of Operational Research, 21 (4), 466–488.

Udayakumar, R. (2022). An EOQ model for non-instantaneous deteriorating items with time-dependent demand under partial backlogging. Journal of Management Analytics . https://doi.org/10.1080/23270012.2022.2073571

Udayakumar, R., & Geetha, K. V. (2017). Economic ordering policy for single item inventory model over finite time horizon. International Journal of Systems Assurance Engineering and Management, 8 , 734–757.

Udayakumar, R., & Geetha, K. V. (2018). An EOQ model for non-instantaneous deteriorating items with two levels of storage under trade credit policy. Journal of Industrial Engineering International, 14 (2), 343–365.

UN. (2022). Transforming our world: the 2030 agenda for sustainable development. division for sustainable development goals, New York, NY, USA. https://unstats.un.org/sdgs/report/2022/ .

Uthayakumar, R., & Geetha, K. V. (2009a). Replenishment policy for single item inventory model with money inflation. Opsearch, 46 (3), 345–357.

Uthayakumar, R., & Geetha, K. V. (2009b). A replenishment policy for non-instantaneous deteriorating inventory system with partial backlogging. Tamsui Oxford Journal of Mathematical Sciences, 25 (3), 313–332.

Valliathal, M., & Uthayakumar, R. (2011). Optimal pricing and replenishment policies of an EOQ model for non-instantaneous deteriorating items with shortages. International Journal of Advanced Manufacturing Technology, 54 (1–4), 361–371.

Valliathal, M., & Uthayakumar, R. (2012). Designing retailer’s replenishment policies for non-instantaneous deteriorating items under fuzzy costs. International Journal of Services and Operations Management, 13 (4), 415–440.

Valliathal, M., & Uthayakumar, R. (2013). Designing and computing optimal policies on a production model for non-instantaneous deteriorating items with shortages. International Journal of Production Research, 51 (1), 215–229.

Valliathal, M., & Uthayakumar, R. (2016). Optimal replenishment policies of an EOQ model for non-instantaneous Weibull deteriorating items with ramp-type of demand under shortages. International Journal of Mathematics in Operational Research, 8 (1), 60–86.

VandanaSharma, B. K. (2016). An inventory model for non-instantaneous deteriorating items with quadratic demand rate and shortages under trade credit policy. Journal of Applied Analysis and Computation, 6 (3), 720–737.

Wang, Y., Zhang, J., & Tang, W. (2015). Dynamic pricing for non-instantaneous deteriorating items. Journal of Intelligent Manufacturing, 26 (4), 629–640.

Wu, J., Skouri, K., Teng, J.-T., & Ouyang, L.-Y. (2014). A note on “optimal replenishment policies for non-instantaneous deteriorating items with price and stock sensitive demand under permissible delay in payment.” International Journal of Production Economics, 155 , 324–329.

Wu, K.-S., Ouyang, L.-Y., & Yang, C.-T. (2006). An optimal replenishment policy for non-instantaneous deteriorating items with stock-dependent demand and partial backlogging. International Journal of Production Economics, 101 (2), 369–384.

Wu, K.-S., Ouyang, L.-Y., & Yang, C.-T. (2009). Coordinating replenishment and pricing policies for non-instantaneous deteriorating items with price-sensitive demand. International Journal of Systems Science, 40 (12), 1273–1281.

Xu, T.-T., & Xiao, T.-J. (2013). Coordination of supply chain with a non-instantaneous deteriorating item based on quantity discount policy. Xitong Gongcheng Lilun Yu Shijian/system Engineering Theory and Practice, 33 (7), 1690–1698.

Xu, Z., Peng, J., Qiu, S., Liu, Y., Dong, J., & Zhang, H. (2022). Responses of spatial relationships between ecosystem services and the sustainable development goals to urbanization. Science of the Total Environment, 850 , 157868.

Yang, C.-T., Ouyang, L.-Y., & Wu, H.-H. (2009). Retailer’s optimal pricing and ordering policies for non-instantaneous deteriorating items with price-dependent demand and partial backlogging. Mathematical Problems in Engineering, 2009 , 198305.

Zhang, J., Wang, Y., Lu, L., & Tang, W. (2015). Optimal dynamic pricing and replenishment cycle for non-instantaneous deterioration items with inventory-level-dependent demand. International Journal of Production Economics, 170 , 136–145.

Zhang, Y., Wang, Y., & Gong, B. (2016). Quantity discount coordination for deteriorating items supply chain based on non-instantaneous replenishment. Computer Integrated Manufacturing Systems, CIMS, 22 (12), 2858–2866.

Download references

Author information

Authors and affiliations.

Indian Institute of Management Jammu, Jammu, 180016, India

Pratik Maheshwari

EDHEC Business School, Roubaix, France

Sachin Kamble

Rabat Business School, International University of Rabat, Sale, Morocco

Amine Belhadi

Indian Institute of Management Nagpur, Nagpur, India

Satish Kumar

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Satish Kumar .

Ethics declarations

Conflict of interest.

Author have no conflict of intertest to declare.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix 1: Comprehensive literature review on NIDIs

Appendix 2: Summary of SDG

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Maheshwari, P., Kamble, S., Belhadi, A. et al. UN sustainable development goals and management of non-instantaneous deteriorating items: a literature review and applications framework. Ann Oper Res (2024). https://doi.org/10.1007/s10479-024-06144-9

Download citation

Received : 03 March 2023

Accepted : 02 July 2024

Published : 01 August 2024

DOI : https://doi.org/10.1007/s10479-024-06144-9

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Sustainable development goals
• Inventory management
• Non-instantaneous deteriorating items
• Find a journal
• Publish with us
• Track your research

Sign in to add this item to your wishlist, follow it, or mark it as ignored

In-game purchases, Chance based in-game purchases

Play Morimens

About this game.

【Worldview】

【Game Features】

A message from the development team.

System Requirements

• Requires a 64-bit processor and operating system
• OS: Windows10 64位以及更高版本
• Processor: Intel Core i5-2500K AMD FX-6300
• Memory: 4 GB RAM
• Graphics: NVIDIA GeForce GTX 960 或 AMD Radeon R9 280
• DirectX: Version 11
• Network: Broadband Internet connection
• Storage: 16 GB available space
• VR Support: 不支持
• Processor: Intel Core i7-4770K 或以上 AMD Ryzen 5 1500X 或以上
• Memory: 8 GB RAM
• Graphics: NVIDIA GeForce GTX 1060 6GB 或 AMD Radeon RX 580

More like this

What curators say, customer reviews.

You can use this widget-maker to generate a bit of HTML that can be embedded in your website to easily allow customers to purchase this game on Steam.

Enter up to 375 characters to add a description to your widget:

Copy and paste the HTML below into your website to make the above widget appear

Popular user-defined tags for this product: (?)

Sign in to add your own tags to this product.