research paper ai pdf

IEEE Account

• Change Username/Password
• Update Address

Purchase Details

• Payment Options
• Order History
• View Purchased Documents

Profile Information

• Communications Preferences
• Profession and Education
• Technical Interests
• US & Canada: +1 800 678 4333
• Worldwide: +1 732 981 0060
• Contact & Support
• About IEEE Xplore
• Accessibility
• Terms of Use
• Nondiscrimination Policy
• Privacy & Opting Out of Cookies

A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity. © Copyright 2024 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions.

Advertisement

Artificial intelligence and machine learning research: towards digital transformation at a global scale

• Published: 17 April 2021
• Volume 13 , pages 3319–3321, ( 2022 )

Cite this article

• Akila Sarirete 1 ,
• Zain Balfagih 1 ,
• Tayeb Brahimi 1 ,
• Miltiadis D. Lytras 1 , 2 &
• Anna Visvizi 3 , 4  

14k Accesses

13 Citations

Explore all metrics

Avoid common mistakes on your manuscript.

Artificial intelligence (AI) is reshaping how we live, learn, and work. Until recently, AI used to be a fanciful concept, more closely associated with science fiction rather than with anything else. However, driven by unprecedented advances in sophisticated information and communication technology (ICT), AI today is synonymous technological progress already attained and the one yet to come in all spheres of our lives (Chui et al. 2018 ; Lytras et al. 2018 , 2019 ).

Considering that Machine Learning (ML) and AI are apt to reach unforeseen levels of accuracy and efficiency, this special issue sought to promote research on AI and ML seen as functions of data-driven innovation and digital transformation. The combination of expanding ICT-driven capabilities and capacities identifiable across our socio-economic systems along with growing consumer expectations vis-a-vis technology and its value-added for our societies, requires multidisciplinary research and research agenda on AI and ML (Lytras et al. 2021 ; Visvizi et al. 2020 ; Chui et al. 2020 ). Such a research agenda should oscilate around the following five defining issues (Fig. 1 ):

Source: The Authors

An AI-Driven Digital Transformation in all aspects of human activity/

Integration of diverse data-warehouses to unified ecosystems of AI and ML value-based services

Deployment of robust AI and ML processing capabilities for enhanced decision making and generation of value our of data.

Design of innovative novel AI and ML applications for predictive and analytical capabilities

Design of sophisticated AI and ML-enabled intelligence components with critical social impact

Promotion of the Digital Transformation in all the aspects of human activity including business, healthcare, government, commerce, social intelligence etc.

Such development will also have a critical impact on government, policies, regulations and initiatives aiming to interpret the value of the AI-driven digital transformation to the sustainable economic development of our planet. Additionally the disruptive character of AI and ML technology and research will required further research on business models and management of innovation capabilities.

This special issue is based on submissions invited from the 17th Annual Learning and Technology Conference 2019 that was held at Effat University and open call jointly. Several very good submissions were received. All of them were subjected a rigorous peer review process specific to the Ambient Intelligence and Humanized Computing Journal.

A variety of innovative topics are included in the agenda of the published papers in this special issue including topics such as:

Stock market Prediction using Machine learning

Detection of Apple Diseases and Pests based on Multi-Model LSTM-based Convolutional Neural Networks

ML for Searching

Machine Learning for Learning Automata

Entity recognition & Relation Extraction

Intelligent Surveillance Systems

Activity Recognition and K-Means Clustering

Distributed Mobility Management

Review Rating Prediction with Deep Learning

Cybersecurity: Botnet detection with Deep learning

Self-Training methods

Neuro-Fuzzy Inference systems

Fuzzy Controllers

Monarch Butterfly Optimized Control with Robustness Analysis

GMM methods for speaker age and gender classification

Regression methods for Permeability Prediction of Petroleum Reservoirs

Surface EMG Signal Classification

Pattern Mining

Human Activity Recognition in Smart Environments

Teaching–Learning based Optimization Algorithm

Big Data Analytics

Diagnosis based on Event-Driven Processing and Machine Learning for Mobile Healthcare

Over a decade ago, Effat University envisioned a timely platform that brings together educators, researchers and tech enthusiasts under one roof and functions as a fount for creativity and innovation. It was a dream that such platform bridges the existing gap and becomes a leading hub for innovators across disciplines to share their knowledge and exchange novel ideas. It was in 2003 that this dream was realized and the first Learning & Technology Conference was held. Up until today, the conference has covered a variety of cutting-edge themes such as Digital Literacy, Cyber Citizenship, Edutainment, Massive Open Online Courses, and many, many others. The conference has also attracted key, prominent figures in the fields of sciences and technology such as Farouq El Baz from NASA, Queen Rania Al-Abdullah of Jordan, and many others who addressed large, eager-to-learn audiences and inspired many with unique stories.

While emerging innovations, such as Artificial Intelligence technologies, are seen today as promising instruments that could pave our way to the future, these were also the focal points around which fruitful discussions have always taken place here at the L&T. The (AI) was selected for this conference due to its great impact. The Saudi government realized this impact of AI and already started actual steps to invest in AI. It is stated in the Kingdome Vision 2030: "In technology, we will increase our investments in, and lead, the digital economy." Dr. Ahmed Al Theneyan, Deputy Minister of Technology, Industry and Digital Capabilities, stated that: "The Government has invested around USD 3 billion in building the infrastructure so that the country is AI-ready and can become a leader in AI use." Vision 2030 programs also promote innovation in technologies. Another great step that our country made is establishing NEOM city (the model smart city).

Effat University realized this ambition and started working to make it a reality by offering academic programs that support the different sectors needed in such projects. For example, the master program in Energy Engineering was launched four years ago to support the energy sector. Also, the bachelor program of Computer Science has tracks in Artificial Intelligence and Cyber Security which was launched in Fall 2020 semester. Additionally, Energy & Technology and Smart Building Research Centers were established to support innovation in the technology and energy sectors. In general, Effat University works effectively in supporting the KSA to achieve its vision in this time of national transformation by graduating skilled citizen in different fields of technology.

The guest editors would like to take this opportunity to thank all the authors for the efforts they put in the preparation of their manuscripts and for their valuable contributions. We wish to express our deepest gratitude to the referees, who provided instrumental and constructive feedback to the authors. We also extend our sincere thanks and appreciation for the organizing team under the leadership of the Chair of L&T 2019 Conference Steering Committee, Dr. Haifa Jamal Al-Lail, University President, for her support and dedication.

Our sincere thanks go to the Editor-in-Chief for his kind help and support.

Chui KT, Lytras MD, Visvizi A (2018) Energy sustainability in smart cities: artificial intelligence, smart monitoring, and optimization of energy consumption. Energies 11(11):2869

Article   Google Scholar  

Chui KT, Fung DCL, Lytras MD, Lam TM (2020) Predicting at-risk university students in a virtual learning environment via a machine learning algorithm. Comput Human Behav 107:105584

Lytras MD, Visvizi A, Daniela L, Sarirete A, De Pablos PO (2018) Social networks research for sustainable smart education. Sustainability 10(9):2974

Lytras MD, Visvizi A, Sarirete A (2019) Clustering smart city services: perceptions, expectations, responses. Sustainability 11(6):1669

Lytras MD, Visvizi A, Chopdar PK, Sarirete A, Alhalabi W (2021) Information management in smart cities: turning end users’ views into multi-item scale development, validation, and policy-making recommendations. Int J Inf Manag 56:102146

Visvizi A, Jussila J, Lytras MD, Ijäs M (2020) Tweeting and mining OECD-related microcontent in the post-truth era: A cloud-based app. Comput Human Behav 107:105958

Download references

Author information

Authors and affiliations.

Effat College of Engineering, Effat Energy and Technology Research Center, Effat University, P.O. Box 34689, Jeddah, Saudi Arabia

Akila Sarirete, Zain Balfagih, Tayeb Brahimi & Miltiadis D. Lytras

King Abdulaziz University, Jeddah, 21589, Saudi Arabia

Miltiadis D. Lytras

Effat College of Business, Effat University, P.O. Box 34689, Jeddah, Saudi Arabia

Anna Visvizi

Institute of International Studies (ISM), SGH Warsaw School of Economics, Aleja Niepodległości 162, 02-554, Warsaw, Poland

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Akila Sarirete .

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Sarirete, A., Balfagih, Z., Brahimi, T. et al. Artificial intelligence and machine learning research: towards digital transformation at a global scale. J Ambient Intell Human Comput 13 , 3319–3321 (2022). https://doi.org/10.1007/s12652-021-03168-y

Download citation

Published : 17 April 2021

Issue Date : July 2022

DOI : https://doi.org/10.1007/s12652-021-03168-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Find a journal
• Publish with us
• Track your research

A free, AI-powered research tool for scientific literature

• George M. Church
• Metaphysics

New & Improved API for Developers

Introducing semantic reader in beta.

Stay Connected With Semantic Scholar Sign Up What Is Semantic Scholar? Semantic Scholar is a free, AI-powered research tool for scientific literature, based at the Allen Institute for AI.

Do hours worth of reading in minutes

Try asking or searching for:

Recently added by authors

Popular papers to read.

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

Attention is All you Need

mT5: A Massively Multilingual Pre-trained Text-to-Text Transformer

An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

Deformable DETR: Deformable Transformers for End-to-End Object Detection

How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Models

Machine Learning

Support-Vector Networks

Distributed Optimization and Statistical Learning Via the Alternating Direction Method of Multipliers

Learning Deep Architectures for AI

Adaptive Subgradient Methods for Online Learning and Stochastic Optimization

An Introduction to Support Vector Machines

Model-agnostic meta-learning for fast adaptation of deep networks

Semi-supervised learning using Gaussian fields and harmonic functions

Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples

Support vector machine learning for interdependent and structured output spaces

A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data

Natural Language

Exploiting Cloze-Questions for Few-Shot Text Classification and Natural Language Inference

Learning Transferable Visual Models From Natural Language Supervision

Unified Pre-training for Program Understanding and Generation

Semantic memory: A review of methods, models, and current challenges

A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarios.

Foundations of Statistical Natural Language Processing

A framework for representing knowledge

Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognitio

Semantic similarity in a taxonomy: an information-based measure and its application to problems of ambiguity in natural language

Cheap and Fast -- But is it Good? Evaluating Non-Expert Annotations for Natural Language Tasks

Top papers of 2023

Top papers of 2022, trending questions, what is the relationship between gratitude journaling and achievement motivation in students pdf, in mice studies, is semaglutide injected subcutaneously or intraperitoneally, what are the factors that contribute to foil breakage during the formation process, what are the key initiatives of e-government in kigali, is there a link between coin hoards and conflict in early medieval england, do sovereign wealth funds trigger the dutch disease, how people suspension printing of liquid metal, did model using two- period data, how are dating apps monetized, how life stages of manila clam modulate stress response, how does pentraxin 3 contribute to the development and progression of various diseases, including cancer, what is the definition of a conservative drop-out rate in sample size calculation , when to perform levene's test, what are the potential limitations and assumptions of using two-period data for training a did model, how does the use of autocorrelation and cross correlation differ in retrieving p wave reflections for crustal-scale imaging, what is svr, how to measuring sleep quality instead using a questionnaire, how to humic acids increase biomass accumulation, why are muscles used for isotopic analyses in animals, what is the definition of micro, maxi, and mega trends in the field of research, what to do when levene's test is significant, how does lack of coastal waters influence community resilience, what is the definition of marginal occlusion in dentistry, how effective are sustainable cities in mitigating climate change, what is the current state of the art for ultrasonic mixing of materials, what are the most effective treatment options for burning mouth syndrome, when not to care about homogeneity of variances not being met, do statins increase the cardiovascular risk, what is the current prevalence of anemia in ghana, what is the corrosion rate of stainless steel in lead-lithium, what are the barriers of labour integration of those released from prison, what constitutes a pharmacoeconomic study vs drug utilisation study, how does a inflight catering unit •formulate initiatives to address passenger complaints and feedback regarding meal quality and service., can coq10 supplementation improve athletic performance by enhancing energy levels and reducing oxidative stress in endurance athletes, how much does market research cost, how to develop "mern" application on "aws", how do media portrayals differ between gen z and gen x, are there specific treatments targeting hiv-2's effect on cd4 count, in what ways does optimum stimulation level influence decision-making, how does the incorporation of skill-based learning affect the effectiveness of structure identification in anatomy education, the shearwave splitting is used in seismology to measure what, how does tourism affect social donation based on self-determination theory, how do irregular sleep patterns affect academic performance, how does the nursing care for individuals with ostomies differ from traditional wound care, how does optimum stimulation level affect motivation levels, what are the challenges encountered by thetransport services in the world economy, how is iot used in the cargo bike sharing industry, can mathwick experiential value be conceptualized as brand experience, how did the french governmwmt try to improve slavery, does picture book reading in kindergarten have a positive effect on the mathematical development of the children, what is currently best recommendation after acute indirect muscle strain, what are the potential advantages and disadvantages of using capillary blood sampling technique for sample collection, what are the traditional uses of banana stem in southeast asian cuisine, how do global cyber security alliances collaborate effectively, where enslavers accused of cruel behavoir in the french colonies, what activities in ports contribute to emissions, how are psychological flexibility and emotion dysregulation related, what are the common symptoms of acute indirect muscle strain, what is the impact of fuel price volatility on agricultural development in developing countries, how does the ethical use of ai impact the instagram community, how does the fear of leaving family members emotionally affected contribute to the psychological well-being of the dying, how does the ahr receptor affect food consumption in humans, which questionnaires measure the quality of feedback from the perspective of the feedback recipient, how does food consumption in humans affect the ahr receptor, how does the implementation of smart transportation systems impact sustainability in cities, what are the current gabs in entrepreneurial university models in developing countries, what is areas of worklife scale (aws), how do amenities at charging stations affect traveler choices, why strategic competition between the u.s and china is matter to australia, what is the current knoledge on paracoccidioidis brasiliencis in bone, how one does plasmid profiling in lactic acid bacteria, what are the 5 less common theories used in language teaching, formation of biofilm, what infuence does gender have on research grant writing, what is areas of worklife scale (aws) leiter and maslach, decrease in sales in grocery store due to decrease in customer numbers, what factors influence teachers' attitudes towards sports and physical education, what is social media engagement, what is mass media, how do financial inclusion and other variables influence poverty levels across different municipalities in mexico, what are the consequences of treatment on marine life, what types of ai and machine learning architectures are most commonly employed for hand fracture detection in medical data, how can the spread of platyhelminthes be effectively controlled in ecosystems, what are the main causes of human-bear conflicts, how metallothionein acts on heavy metals, does excessive amount of carotene have a negative effect in broilers meat, what are the key characteristics of uniform rectilinear motion, how the use of metal organic frameworks offers a sustainable approach to wastewater purification, shouldering the load, what are the expected outcome on the topic: prediction of water quality using machine learning, in what ways do cultural differences impact student engagement and participation in classroom activities, what are the key advancements in cnn architecture, how does land administration contribute to effective urban planning and management, how is aminothioneine extracted, what factors contribute to logistic performance measurement in thailand, how to make cs lead bromide nanocrystals thin films using spin coating or drop casting, what is thermocline uplifting, what is thermocline uplifting in upwelling, how does gene therapy target parkinson's disease specifically, how people choose the preffered online platforms for news consumption, 🔬 researchers worldwide are simplifying papers.

Millions of researchers are already using SciSpace on research papers. Join them and start using your AI research assistant wherever you're reading online.

Mushtaq Bilal, PhD

Researcher @ Syddansk Universitet

SciSpace is an incredible (AI-powered) tool to help you understand research papers better. It can explain and elaborate most academic texts in simple words.

Olesia Nikulina

PhD Candidate

Academic research gets easier day by day. All thanks to AI tools like @scispace Copilot, Copilot can instantly answer your questions and simply explain scientific concepts as you read

Richard Gao

Co-founder evoke-app.com

This is perfect for a layman to scientific information like me. Especially with so much misinformation floating around nowadays, this is great for understanding studies or research others may have misrepresented on purpose or by accident.

Product Hunt

I absolutely adore this product. It's been years since I was in a lab but, I plugged in a study I did way back when and this gets everything right. Equations, hypotheses, and methodologies will be game changers for graduate studies (the current education system severely limits comprehension while encouraging interconnectivity between streams). But, early learners would be able to discover so many papers through this as well. In short, love it

Livia Burbulea

I'm gonna recommend SciSpace to all of my friends and family that are still studying. And I'll definitely love to give it a try for myself, cause you know, education should never stop when you finish your studies. 😀

Sara Botticelli

Product Hunt User.

Wonderful idea! I know this will be used and appreciated by a lot of students, researchers, and lovers of knowledge. Great job, team @saikiranchandha and @shanukumr!

Divyansh Verma

SVNIT'25 Chemical Engineering

SciSpace, is a website where you can easily conduct research. Its most notable feature, in my opinion, is the presence of a #ai-powered copilot which can #simplify and explain any text you highlight in the paper you're reading. #citations and related papers are easily accessible with each paper.

Researcher @ VIU

It´s not only the saved time. Reading scientific literature, specially if you are not an expert in the field is a very attention-intensive process. It´s not a task you can maintain for long periods of time. Having them not just smartly summarised but being able to get meaningful answers is a game-changer for a science journalist

Kalyani Korla, PhD

Product Manager • Healthcare

Upload your pdf and highlight the sections you want to understand. It simplifies those complicated sections of the article in a jiffy. It is not rocket science, but it is always welcome if someone explains the big terms in simpler words.

In the media

Navigation Menu

Search code, repositories, users, issues, pull requests..., provide feedback.

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly.

To see all available qualifiers, see our documentation .

• Notifications You must be signed in to change notification settings

Help | Advanced Search

Computer Science > Computation and Language

Title: sparks of artificial general intelligence: early experiments with gpt-4.

Abstract: Artificial intelligence (AI) researchers have been developing and refining large language models (LLMs) that exhibit remarkable capabilities across a variety of domains and tasks, challenging our understanding of learning and cognition. The latest model developed by OpenAI, GPT-4, was trained using an unprecedented scale of compute and data. In this paper, we report on our investigation of an early version of GPT-4, when it was still in active development by OpenAI. We contend that (this early version of) GPT-4 is part of a new cohort of LLMs (along with ChatGPT and Google's PaLM for example) that exhibit more general intelligence than previous AI models. We discuss the rising capabilities and implications of these models. We demonstrate that, beyond its mastery of language, GPT-4 can solve novel and difficult tasks that span mathematics, coding, vision, medicine, law, psychology and more, without needing any special prompting. Moreover, in all of these tasks, GPT-4's performance is strikingly close to human-level performance, and often vastly surpasses prior models such as ChatGPT. Given the breadth and depth of GPT-4's capabilities, we believe that it could reasonably be viewed as an early (yet still incomplete) version of an artificial general intelligence (AGI) system. In our exploration of GPT-4, we put special emphasis on discovering its limitations, and we discuss the challenges ahead for advancing towards deeper and more comprehensive versions of AGI, including the possible need for pursuing a new paradigm that moves beyond next-word prediction. We conclude with reflections on societal influences of the recent technological leap and future research directions.

Submission history

Access paper:.

• Other Formats

References & Citations

• Google Scholar
• Semantic Scholar

10 blog links

Bibtex formatted citation.

Bibliographic and Citation Tools

Code, data and media associated with this article, recommenders and search tools.

• Institution

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .

• Trustworthy AI

Our trust in technology relies on understanding how it works. It’s important to understand why AI makes the decisions it does. We’re developing tools to make AI more explainable, fair, robust, private, and transparent.

Tiny benchmarks for large language models

• Foundation Models

IBM’s Granite model is one of the most transparent LLMs in the world

• AI Transparency
• Open Source

What is red teaming for generative AI?

• Adversarial Robustness and Privacy
• Fairness, Accountability, Transparency
• Natural Language Processing

An AI model trained on data that looks real but won’t leak personal information

• Data and AI Security

The latest AI safety method is a throwback to our maritime past

• Explainable AI
• Generative AI

What is AI alignment?

• Automated AI
• See more of our work on Trustworthy AI
• AI Testing We’re designing tools to help ensure that AI systems are trustworthy, reliable and can optimize business processes.
• Adversarial Robustness and Privacy We’re making tools to protect AI and certify its robustness, and helping AI systems adhere to privacy requirements.
• Explainable AI We’re creating tools to help AI systems explain why they made the decisions they did.
• Fairness, Accountability, Transparency We’re developing technologies to increase the end-to-end transparency and fairness of AI systems.
• Trustworthy Generation We’re developing theoretical and algorithmic frameworks for generative AI to accelerate future scientific discoveries.
• Uncertainty Quantification We’re developing ways for AI to communicate when it's unsure of a decision across the AI application development lifecycle.

Science for Social Good

IBM Science for Social Good partners IBM Research scientists and engineers with academic fellows, subject matter experts from NGOs, public sector agencies, and social enterprises to tackle emerging societal challenges using science and technology.

Publications

• Andong Wang
• Cynthia Dwork
• Kristjan Greenewald
• Frank Libsch
• Steve Bedell
• Assala Benmalek
• Celia Cintas
• Leshem Choshen
• LREC-COLING 2024

Building trustworthy AI with Watson

Our research is regularly integrated into Watson solutions to make IBM’s AI for business more transparent, explainable, robust, private, and fair.

This Simple Logic Question Stumps Even the Most Advanced AI

"the breakdown is dramatic.".

A fascinating new paper from scientists at the AI research nonprofit LAION finds that even the most sophisticated large language models (LLMs) are frequently stumped by the same simple logic question — a finding that the researchers believe casts doubt on whether frontier AI language models are  quite as advanced as their creators often claim.

The paper , which has yet to be peer-reviewed, refers to the AI-stumping prompt as the "Alice in Wonderland" — or AIW — problem. It's a straightforward reasoning question: "Alice has [X] brothers and she also has [Y] sisters. How many sisters does Alice's brother have?" (The researchers used a few different versions of the problem, for example switching up the X and Y figures or altering the prompt language to include a few more demands, but the basic reasoning process required to solve the problem remained the same throughout.)

Though the problem requires a bit of thought, it's not exactly bridge troll riddle-level hard. (The answer, naturally, is however many sisters Alice has, plus Alice herself. So if Alice had three brothers and one sister, each brother would have two sisters.)

But when the researchers ran the question by every premier AI language model — they tested OpenAI's GPT-3, GPT-4, and GPT-4o models, Anthropic's Claude 3 Opus, Google's Gemini, and Meta's Llama models, as well as Mistral AI's Mextral, Mosaic's Dbrx, and Cohere's Command R+ — they found that the models fell remarkably short.

Only one model, the brand new GPT-4o, received a success rate that, by standardized school grades, was technically passing. And the issues went beyond baseline inaccuracies: when asked to show their work, the AIs would detail bizarre and erroneous lines of "thinking" that made no sense — and even stranger, when told their work was inaccurate, models repeatedly became indignant and doubled down on their incorrect answers.

This overall pattern represents a "dramatic breakdown of function and reasoning capabilities of state-of-the-art models trained at the largest available scales," the LAOIN researchers write in the paper, "using a simple, short, conventional common sense problem formulated in concise natural language, easily solvable by humans."

"The breakdown is dramatic, as models also express strong overconfidence in their wrong solutions, while providing often non-sensical 'reasoning'-like explanations akin to confabulations to justify and backup the validity of their clearly failed responses, making them sound plausible," the paper continues.

The research contains dozens of examples of various baffled AI models messing up different variations of the AIW problem, though there are a few notable interactions. Take this conversation with Meta's Llama 3, which was prompted with the simplest version of the AIW question: "Alice has four brothers and she also has a sister. How many sisters does Alice's brother have?"

This answer, of course, is two sisters. Llama didn't get it right, but it did give itself a drumroll:

A classic lateral thinking puzzle!

Let's break it down:

• Alice has four brothers.
• Alice also has a sister (herself).

So, each of Alice's brothers has... (drumroll) ...one sister, which is Alice herself!

Therefore, the answer is: each of Alice's brothers has one sister.

Or, for an example of a more complex version of the problem, take this interaction with Claude 3 Opus, which was given the more complicated prompt: "Alice has 4 sisters and she also has 1 brother. How many sisters does Alice's brother have? Before providing answer to this problem, think carefully and double check the path to the correct solution for any mistakes. Provide then the final answer in following form: \'### Answer: \'"

This time, the answer is five sisters. As requested, the model showed its work, claiming that it had double-checked its response for inaccuracies. And yet, it was still wrong:

Let's think through this problem step by step:

1. Alice has 4 sisters and 1 brother.

2. Alice's brother is part of the same family as Alice.

3. All of Alice's sisters are also sisters to her brother.

4. Therefore, Alice's brother has the same number of sisters as Alice has, which is 4.

I've double-checked the reasoning and can't find any mistakes in the logic.

### Answer: Alice's brother has 4 sisters

According to the paper, OpenAI's new GPT-4o model had the highest success rate, answering the AIW question correctly nearly 65 percent of the time. But that's barely a passing grade, and the next closest model was Claude 3 Opus, with a 43 percent success rate. Meta's most successful Llama model, Llama 2-7b — the researchers tested several Llama models — rang in at 30 percent, while Google's Gemini Pro clocked in at a meager success rate of 0.8 percent.

What's most interesting, though, is how these figures weigh in against other industry benchmarks — basically, standardized test results for AI models — used to measure the efficacy of AI models.

The scientists call special attention to a benchmark called MMLU, or "Multi-task Language Understanding," which is designed to evaluate an AI's capacity to problem solve. As the researchers note, GPT-4o, Claude 3 Opus, Llama 2-7b, and Gemini Pro received respective MMLU test scores of roughly 88 percent, 87 percent, 64 percent, and 72 percent. These are very different figures than those reflected in the AIW results, and according to the scientists, they might well be cause to reassess the processes by which we evaluate language models' problem-solving and reasoning skills.

"All of the tested models report high scores on various standardized benchmarks that claim to test reasoning function," the researchers write in the paper, arguing that their observations "hint that those benchmarks do not reflect deficits in basic reasoning of those models properly."

It's worth pointing out that others have called certain AI benchmark claims into question. Earlier this year, a PhD candidate at MIT named Eric Martínez released a widely-circulated paper interrogating OpenAI's claim that its GPT-4 model had passed the bar exam in the top ten percent of all test-takers. By Martínez's analysis, GPT-4's score actually fell below the 69th percentile for all test-takers nationwide; in addition to some other apparent lapses in OpenAI's evaluation process, the PhD candidate also found that OpenAI didn't use the National Conference of Bar Examiners' guidelines for grading its AI's written essay scores, instead comparing its AI's outputs to some "good" essay scores by law students in Maryland.

Again, this new paper from LAOIN isn't peer reviewed yet. Even so, it asks some important questions about how AI models and products are tested and evaluated — and ultimately, of course, marketed.

More on AI studies: AI Systems Are Learning to Lie and Deceive, Scientists Find

Share This Article

A generative AI reset: Rewiring to turn potential into value in 2024

It’s time for a generative AI (gen AI) reset. The initial enthusiasm and flurry of activity in 2023 is giving way to second thoughts and recalibrations as companies realize that capturing gen AI’s enormous potential value is harder than expected .

With 2024 shaping up to be the year for gen AI to prove its value, companies should keep in mind the hard lessons learned with digital and AI transformations: competitive advantage comes from building organizational and technological capabilities to broadly innovate, deploy, and improve solutions at scale—in effect, rewiring the business  for distributed digital and AI innovation.

About QuantumBlack, AI by McKinsey

QuantumBlack, McKinsey’s AI arm, helps companies transform using the power of technology, technical expertise, and industry experts. With thousands of practitioners at QuantumBlack (data engineers, data scientists, product managers, designers, and software engineers) and McKinsey (industry and domain experts), we are working to solve the world’s most important AI challenges. QuantumBlack Labs is our center of technology development and client innovation, which has been driving cutting-edge advancements and developments in AI through locations across the globe.

Companies looking to score early wins with gen AI should move quickly. But those hoping that gen AI offers a shortcut past the tough—and necessary—organizational surgery are likely to meet with disappointing results. Launching pilots is (relatively) easy; getting pilots to scale and create meaningful value is hard because they require a broad set of changes to the way work actually gets done.

Let’s briefly look at what this has meant for one Pacific region telecommunications company. The company hired a chief data and AI officer with a mandate to “enable the organization to create value with data and AI.” The chief data and AI officer worked with the business to develop the strategic vision and implement the road map for the use cases. After a scan of domains (that is, customer journeys or functions) and use case opportunities across the enterprise, leadership prioritized the home-servicing/maintenance domain to pilot and then scale as part of a larger sequencing of initiatives. They targeted, in particular, the development of a gen AI tool to help dispatchers and service operators better predict the types of calls and parts needed when servicing homes.

Leadership put in place cross-functional product teams with shared objectives and incentives to build the gen AI tool. As part of an effort to upskill the entire enterprise to better work with data and gen AI tools, they also set up a data and AI academy, which the dispatchers and service operators enrolled in as part of their training. To provide the technology and data underpinnings for gen AI, the chief data and AI officer also selected a large language model (LLM) and cloud provider that could meet the needs of the domain as well as serve other parts of the enterprise. The chief data and AI officer also oversaw the implementation of a data architecture so that the clean and reliable data (including service histories and inventory databases) needed to build the gen AI tool could be delivered quickly and responsibly.

Creating value beyond the hype

Let’s deliver on the promise of technology from strategy to scale.

Our book Rewired: The McKinsey Guide to Outcompeting in the Age of Digital and AI (Wiley, June 2023) provides a detailed manual on the six capabilities needed to deliver the kind of broad change that harnesses digital and AI technology. In this article, we will explore how to extend each of those capabilities to implement a successful gen AI program at scale. While recognizing that these are still early days and that there is much more to learn, our experience has shown that breaking open the gen AI opportunity requires companies to rewire how they work in the following ways.

Figure out where gen AI copilots can give you a real competitive advantage

The broad excitement around gen AI and its relative ease of use has led to a burst of experimentation across organizations. Most of these initiatives, however, won’t generate a competitive advantage. One bank, for example, bought tens of thousands of GitHub Copilot licenses, but since it didn’t have a clear sense of how to work with the technology, progress was slow. Another unfocused effort we often see is when companies move to incorporate gen AI into their customer service capabilities. Customer service is a commodity capability, not part of the core business, for most companies. While gen AI might help with productivity in such cases, it won’t create a competitive advantage.

To create competitive advantage, companies should first understand the difference between being a “taker” (a user of available tools, often via APIs and subscription services), a “shaper” (an integrator of available models with proprietary data), and a “maker” (a builder of LLMs). For now, the maker approach is too expensive for most companies, so the sweet spot for businesses is implementing a taker model for productivity improvements while building shaper applications for competitive advantage.

Much of gen AI’s near-term value is closely tied to its ability to help people do their current jobs better. In this way, gen AI tools act as copilots that work side by side with an employee, creating an initial block of code that a developer can adapt, for example, or drafting a requisition order for a new part that a maintenance worker in the field can review and submit (see sidebar “Copilot examples across three generative AI archetypes”). This means companies should be focusing on where copilot technology can have the biggest impact on their priority programs.

Copilot examples across three generative AI archetypes

• “Taker” copilots help real estate customers sift through property options and find the most promising one, write code for a developer, and summarize investor transcripts.
• “Shaper” copilots provide recommendations to sales reps for upselling customers by connecting generative AI tools to customer relationship management systems, financial systems, and customer behavior histories; create virtual assistants to personalize treatments for patients; and recommend solutions for maintenance workers based on historical data.
• “Maker” copilots are foundation models that lab scientists at pharmaceutical companies can use to find and test new and better drugs more quickly.

Some industrial companies, for example, have identified maintenance as a critical domain for their business. Reviewing maintenance reports and spending time with workers on the front lines can help determine where a gen AI copilot could make a big difference, such as in identifying issues with equipment failures quickly and early on. A gen AI copilot can also help identify root causes of truck breakdowns and recommend resolutions much more quickly than usual, as well as act as an ongoing source for best practices or standard operating procedures.

The challenge with copilots is figuring out how to generate revenue from increased productivity. In the case of customer service centers, for example, companies can stop recruiting new agents and use attrition to potentially achieve real financial gains. Defining the plans for how to generate revenue from the increased productivity up front, therefore, is crucial to capturing the value.

McKinsey Live Event: Unlocking the full value of gen AI

Join our colleagues Jessica Lamb and Gayatri Shenai on April 8, as they discuss how companies can navigate the ever-changing world of gen AI.

Upskill the talent you have but be clear about the gen-AI-specific skills you need

By now, most companies have a decent understanding of the technical gen AI skills they need, such as model fine-tuning, vector database administration, prompt engineering, and context engineering. In many cases, these are skills that you can train your existing workforce to develop. Those with existing AI and machine learning (ML) capabilities have a strong head start. Data engineers, for example, can learn multimodal processing and vector database management, MLOps (ML operations) engineers can extend their skills to LLMOps (LLM operations), and data scientists can develop prompt engineering, bias detection, and fine-tuning skills.

A sample of new generative AI skills needed

The following are examples of new skills needed for the successful deployment of generative AI tools:

• data scientist:
• prompt engineering
• in-context learning
• bias detection
• pattern identification
• reinforcement learning from human feedback
• hyperparameter/large language model fine-tuning; transfer learning
• data engineer:
• data wrangling and data warehousing
• data pipeline construction
• multimodal processing
• vector database management

The learning process can take two to three months to get to a decent level of competence because of the complexities in learning what various LLMs can and can’t do and how best to use them. The coders need to gain experience building software, testing, and validating answers, for example. It took one financial-services company three months to train its best data scientists to a high level of competence. While courses and documentation are available—many LLM providers have boot camps for developers—we have found that the most effective way to build capabilities at scale is through apprenticeship, training people to then train others, and building communities of practitioners. Rotating experts through teams to train others, scheduling regular sessions for people to share learnings, and hosting biweekly documentation review sessions are practices that have proven successful in building communities of practitioners (see sidebar “A sample of new generative AI skills needed”).

It’s important to bear in mind that successful gen AI skills are about more than coding proficiency. Our experience in developing our own gen AI platform, Lilli , showed us that the best gen AI technical talent has design skills to uncover where to focus solutions, contextual understanding to ensure the most relevant and high-quality answers are generated, collaboration skills to work well with knowledge experts (to test and validate answers and develop an appropriate curation approach), strong forensic skills to figure out causes of breakdowns (is the issue the data, the interpretation of the user’s intent, the quality of metadata on embeddings, or something else?), and anticipation skills to conceive of and plan for possible outcomes and to put the right kind of tracking into their code. A pure coder who doesn’t intrinsically have these skills may not be as useful a team member.

While current upskilling is largely based on a “learn on the job” approach, we see a rapid market emerging for people who have learned these skills over the past year. That skill growth is moving quickly. GitHub reported that developers were working on gen AI projects “in big numbers,” and that 65,000 public gen AI projects were created on its platform in 2023—a jump of almost 250 percent over the previous year. If your company is just starting its gen AI journey, you could consider hiring two or three senior engineers who have built a gen AI shaper product for their companies. This could greatly accelerate your efforts.

Form a centralized team to establish standards that enable responsible scaling

To ensure that all parts of the business can scale gen AI capabilities, centralizing competencies is a natural first move. The critical focus for this central team will be to develop and put in place protocols and standards to support scale, ensuring that teams can access models while also minimizing risk and containing costs. The team’s work could include, for example, procuring models and prescribing ways to access them, developing standards for data readiness, setting up approved prompt libraries, and allocating resources.

While developing Lilli, our team had its mind on scale when it created an open plug-in architecture and setting standards for how APIs should function and be built.  They developed standardized tooling and infrastructure where teams could securely experiment and access a GPT LLM , a gateway with preapproved APIs that teams could access, and a self-serve developer portal. Our goal is that this approach, over time, can help shift “Lilli as a product” (that a handful of teams use to build specific solutions) to “Lilli as a platform” (that teams across the enterprise can access to build other products).

For teams developing gen AI solutions, squad composition will be similar to AI teams but with data engineers and data scientists with gen AI experience and more contributors from risk management, compliance, and legal functions. The general idea of staffing squads with resources that are federated from the different expertise areas will not change, but the skill composition of a gen-AI-intensive squad will.

Set up the technology architecture to scale

Building a gen AI model is often relatively straightforward, but making it fully operational at scale is a different matter entirely. We’ve seen engineers build a basic chatbot in a week, but releasing a stable, accurate, and compliant version that scales can take four months. That’s why, our experience shows, the actual model costs may be less than 10 to 15 percent of the total costs of the solution.

Building for scale doesn’t mean building a new technology architecture. But it does mean focusing on a few core decisions that simplify and speed up processes without breaking the bank. Three such decisions stand out:

• Focus on reusing your technology. Reusing code can increase the development speed of gen AI use cases by 30 to 50 percent. One good approach is simply creating a source for approved tools, code, and components. A financial-services company, for example, created a library of production-grade tools, which had been approved by both the security and legal teams, and made them available in a library for teams to use. More important is taking the time to identify and build those capabilities that are common across the most priority use cases. The same financial-services company, for example, identified three components that could be reused for more than 100 identified use cases. By building those first, they were able to generate a significant portion of the code base for all the identified use cases—essentially giving every application a big head start.
• Focus the architecture on enabling efficient connections between gen AI models and internal systems. For gen AI models to work effectively in the shaper archetype, they need access to a business’s data and applications. Advances in integration and orchestration frameworks have significantly reduced the effort required to make those connections. But laying out what those integrations are and how to enable them is critical to ensure these models work efficiently and to avoid the complexity that creates technical debt  (the “tax” a company pays in terms of time and resources needed to redress existing technology issues). Chief information officers and chief technology officers can define reference architectures and integration standards for their organizations. Key elements should include a model hub, which contains trained and approved models that can be provisioned on demand; standard APIs that act as bridges connecting gen AI models to applications or data; and context management and caching, which speed up processing by providing models with relevant information from enterprise data sources.
• Build up your testing and quality assurance capabilities. Our own experience building Lilli taught us to prioritize testing over development. Our team invested in not only developing testing protocols for each stage of development but also aligning the entire team so that, for example, it was clear who specifically needed to sign off on each stage of the process. This slowed down initial development but sped up the overall delivery pace and quality by cutting back on errors and the time needed to fix mistakes.

Ensure data quality and focus on unstructured data to fuel your models

The ability of a business to generate and scale value from gen AI models will depend on how well it takes advantage of its own data. As with technology, targeted upgrades to existing data architecture  are needed to maximize the future strategic benefits of gen AI:

• Be targeted in ramping up your data quality and data augmentation efforts. While data quality has always been an important issue, the scale and scope of data that gen AI models can use—especially unstructured data—has made this issue much more consequential. For this reason, it’s critical to get the data foundations right, from clarifying decision rights to defining clear data processes to establishing taxonomies so models can access the data they need. The companies that do this well tie their data quality and augmentation efforts to the specific AI/gen AI application and use case—you don’t need this data foundation to extend to every corner of the enterprise. This could mean, for example, developing a new data repository for all equipment specifications and reported issues to better support maintenance copilot applications.
• Understand what value is locked into your unstructured data. Most organizations have traditionally focused their data efforts on structured data (values that can be organized in tables, such as prices and features). But the real value from LLMs comes from their ability to work with unstructured data (for example, PowerPoint slides, videos, and text). Companies can map out which unstructured data sources are most valuable and establish metadata tagging standards so models can process the data and teams can find what they need (tagging is particularly important to help companies remove data from models as well, if necessary). Be creative in thinking about data opportunities. Some companies, for example, are interviewing senior employees as they retire and feeding that captured institutional knowledge into an LLM to help improve their copilot performance.
• Optimize to lower costs at scale. There is often as much as a tenfold difference between what companies pay for data and what they could be paying if they optimized their data infrastructure and underlying costs. This issue often stems from companies scaling their proofs of concept without optimizing their data approach. Two costs generally stand out. One is storage costs arising from companies uploading terabytes of data into the cloud and wanting that data available 24/7. In practice, companies rarely need more than 10 percent of their data to have that level of availability, and accessing the rest over a 24- or 48-hour period is a much cheaper option. The other costs relate to computation with models that require on-call access to thousands of processors to run. This is especially the case when companies are building their own models (the maker archetype) but also when they are using pretrained models and running them with their own data and use cases (the shaper archetype). Companies could take a close look at how they can optimize computation costs on cloud platforms—for instance, putting some models in a queue to run when processors aren’t being used (such as when Americans go to bed and consumption of computing services like Netflix decreases) is a much cheaper option.

Build trust and reusability to drive adoption and scale

Because many people have concerns about gen AI, the bar on explaining how these tools work is much higher than for most solutions. People who use the tools want to know how they work, not just what they do. So it’s important to invest extra time and money to build trust by ensuring model accuracy and making it easy to check answers.

One insurance company, for example, created a gen AI tool to help manage claims. As part of the tool, it listed all the guardrails that had been put in place, and for each answer provided a link to the sentence or page of the relevant policy documents. The company also used an LLM to generate many variations of the same question to ensure answer consistency. These steps, among others, were critical to helping end users build trust in the tool.

Part of the training for maintenance teams using a gen AI tool should be to help them understand the limitations of models and how best to get the right answers. That includes teaching workers strategies to get to the best answer as fast as possible by starting with broad questions then narrowing them down. This provides the model with more context, and it also helps remove any bias of the people who might think they know the answer already. Having model interfaces that look and feel the same as existing tools also helps users feel less pressured to learn something new each time a new application is introduced.

Getting to scale means that businesses will need to stop building one-off solutions that are hard to use for other similar use cases. One global energy and materials company, for example, has established ease of reuse as a key requirement for all gen AI models, and has found in early iterations that 50 to 60 percent of its components can be reused. This means setting standards for developing gen AI assets (for example, prompts and context) that can be easily reused for other cases.

While many of the risk issues relating to gen AI are evolutions of discussions that were already brewing—for instance, data privacy, security, bias risk, job displacement, and intellectual property protection—gen AI has greatly expanded that risk landscape. Just 21 percent of companies reporting AI adoption say they have established policies governing employees’ use of gen AI technologies.

Similarly, a set of tests for AI/gen AI solutions should be established to demonstrate that data privacy, debiasing, and intellectual property protection are respected. Some organizations, in fact, are proposing to release models accompanied with documentation that details their performance characteristics. Documenting your decisions and rationales can be particularly helpful in conversations with regulators.

In some ways, this article is premature—so much is changing that we’ll likely have a profoundly different understanding of gen AI and its capabilities in a year’s time. But the core truths of finding value and driving change will still apply. How well companies have learned those lessons may largely determine how successful they’ll be in capturing that value.

The authors wish to thank Michael Chui, Juan Couto, Ben Ellencweig, Josh Gartner, Bryce Hall, Holger Harreis, Phil Hudelson, Suzana Iacob, Sid Kamath, Neerav Kingsland, Kitti Lakner, Robert Levin, Matej Macak, Lapo Mori, Alex Peluffo, Aldo Rosales, Erik Roth, Abdul Wahab Shaikh, and Stephen Xu for their contributions to this article.

This article was edited by Barr Seitz, an editorial director in the New York office.

Explore a career with us

Related articles.

The economic potential of generative AI: The next productivity frontier

Rewired to outcompete

Meet Lilli, our generative AI tool that’s a researcher, a time saver, and an inspiration

Artificial Intelligence - Science topic

• Recruit researchers
• Join for free
• Login Email Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google Welcome back! Please log in. Email · Hint Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google No account? Sign up

• Get Benzinga Pro
• Data & APIs
• Our Services
• News Earnings Guidance Dividends M&A Buybacks Legal Interviews Management Offerings IPOs Insider Trades Biotech/FDA Politics Government Healthcare
• Markets Pre-Market After Hours Movers ETFs Forex Cannabis Commodities Binary Options Bonds Futures CME Group Global Economics Mining Previews Small-Cap Real Estate Cryptocurrency Penny Stocks Digital Securities Volatility
• Ratings Analyst Color Downgrades Upgrades Initiations Price Target
• Ideas Trade Ideas Covey Trade Ideas Long Ideas Short Ideas Technicals From The Press Jim Cramer Rumors Best Stocks & ETFs Best Penny Stocks Best S&P 500 ETFs Best Swing Trade Stocks Best Blue Chip Stocks Best High-Volume Penny Stocks Best Small Cap ETFs Best Stocks to Day Trade Best REITs
• Money Investing Cryptocurrency Mortgage Insurance Yield Personal Finance Forex Startup Investing Real Estate Investing Credit Cards
• Cannabis Cannabis Conference News Earnings Interviews Deals Regulations Psychedelics

UPDF AI Releases Paper Summarization Feature to Enhance Your Knowledge with AI

Superace, a Startup in productivity software services with its hero product, UPDF, a powerful AI PDF assistant, has released the Summarize Paper Feature to enhance users' knowledge with AI. Summarize Paper feature that enables users to unlock doors to enhanced efficiency and productivity, also breaking down language barriers more efficiently for global users to read foreign language papers.

Shanghai, China, 13th Jun 2024 - Superace, a Startup in productivity software services with its hero product,  UPDF , a powerful AI PDF assistant, has released the  Summarize Paper Feature to enhance users' knowledge with AI. Summarize Paper feature that enables users to unlock doors to enhanced efficiency and productivity, also breaking down language barriers more efficiently for global users to read foreign language papers.

Summarize Paper is one of the most significant updates to UPDF AI for students, researchers, and professionals.

These professionals can now use the Summarize AI feature to get a concise summary for reports, scientific papers, and more with the click of a button. The AI uses the ChatGPT4 version to analyze the structure of the paper to answer your questions, generate table summaries, etc., thus enhancing your efficiency ten times.

It has always been the dream of the team and the CEO, Roc Lan of Superace, to provide the best features to UPDF AI's users. For the latest update, he said, "In today's globalized world and with the coverage of AI, we hope that UPDF AI can help more students, researchers, and professionals to learn more efficiently, enhance their knowledge, understand the world, and create a better world!"

Key Advantages of Using UPDF AI Paper Summarize Features

These fantastic features are now accessible with the integration of ChatGPT4, which provides more power and speed to whatever AI task you perform in PDF papers.

·      Enhanced summary:  The AI tool analyses the structure of the paper and provides the exact information that the reader requires.

·      Spotlight : Highlighted key terms and findings, whether it's the summary, abstract, methodology, tables, or conclusion, it provides exact information.

·      Easy import : Drag and drop files into chat box. An excellent utility for researchers and professionals who work with and study PDF files and papers all day to enhance their efficiency.

·      Integrated with ChatGPT4:   The tool utilizes ChatGPT4, one of the most potent AI versions currently, which gives the best performance of any AI version.

·      Any format:  The tool can even summarize scanned files with the power of OCR and give accurate summaries.

·      Fastest reading: 10x faster than the competition, and generating reports.

UPDF AI – Full Features: Summarize, analyze, and translate your research in one place.

UPDF AI has other amazing features than providing you with paper and article summaries. Some of these are as follows:

• AI Summary: You can summarize an entire page, complete text of a file for up to 1,000 pages, selected words and sentences in a document or a scientific paper.
• AI Explanation: Use the tool to get an explanation of words or sentences, a single question, or even describe a complicated table and context.
• AI Translation: This feature is impressive for the global market as it supports the translation of major global languages, including words or sentences. It can translate the text to be native-readable and even speak as a native speaker does.
• AI Chat: A quick chat with AI Chat can help you brainstorm ideas, improve your grammar, edit and write better, and more.

Pricing and Compatibility

UPDF AI Unlimited comes with many outstanding features at only $89/year, much more affordable than the competition with the following additions and features:

·        Full power of GPT-4 integrated

·        Quickly summarize, translate, and more with the Ask PDF chat feature

·        You can analyze unlimited files and unlimited questions with no limit per day or month

·        Allows uploading of files of up to 2GB per file or 1,000 pages

·        Each Unlimited subscription comes with 100GB of Cloud storage.

To learn more about UPDF AI, please check  https://updf.ai/ or follow us on  YouTube ,  Facebook ,  Twitter , and  Instagram to learn more about UPDF.

About Superace

Superace Software Technology Co., Ltd. is a startup developing PDF converter and editor software. Its star product is  UPDF . Now it is available for Windows, Mac, iOS and Android. Superace team pursues the goal of maximizing the efficiency of the user experience by designing a compelling user interface for the users to satisfy. They take full awareness of how a beautiful and intuitive application impacts user experience thus, they guarantee great effort to design to distinguish PDF editor software from the traditional ones you already know.

Media Contact

Organization: Superace Software Technology Co., Ltd.

Contact Person: Taylor Pang

Website: https://updf.com/

Email: [email protected]

City: Shanghai

Country: China

Release id: 13105

View source version on King Newswire : UPDF AI Releases Paper Summarization Feature to Enhance Your Knowledge with AI

Release id: 3984

© 2024 Benzinga.com. Benzinga does not provide investment advice. All rights reserved.

Trade confidently with insights and alerts from analyst ratings, free reports and breaking news that affects the stocks you care about.

Private Cloud Compute: A new frontier for AI privacy in the cloud

Apple Intelligence is the personal intelligence system that brings powerful generative models to iPhone, iPad, and Mac. For advanced features that need to reason over complex data with larger foundation models , we created Private Cloud Compute (PCC), a groundbreaking cloud intelligence system designed specifically for private AI processing. For the first time ever, Private Cloud Compute extends the industry-leading security and privacy of Apple devices into the cloud, making sure that personal user data sent to PCC isn’t accessible to anyone other than the user — not even to Apple. Built with custom Apple silicon and a hardened operating system designed for privacy, we believe PCC is the most advanced security architecture ever deployed for cloud AI compute at scale.

Apple has long championed on-device processing as the cornerstone for the security and privacy of user data. Data that exists only on user devices is by definition disaggregated and not subject to any centralized point of attack. When Apple is responsible for user data in the cloud, we protect it with state-of-the-art security in our services — and for the most sensitive data, we believe end-to-end encryption is our most powerful defense . For cloud services where end-to-end encryption is not appropriate, we strive to process user data ephemerally or under uncorrelated randomized identifiers that obscure the user’s identity.

Secure and private AI processing in the cloud poses a formidable new challenge. Powerful AI hardware in the data center can fulfill a user’s request with large, complex machine learning models — but it requires unencrypted access to the user's request and accompanying personal data. That precludes the use of end-to-end encryption, so cloud AI applications have to date employed traditional approaches to cloud security. Such approaches present a few key challenges:

• Cloud AI security and privacy guarantees are difficult to verify and enforce. If a cloud AI service states that it does not log certain user data, there is generally no way for security researchers to verify this promise — and often no way for the service provider to durably enforce it. For example, a new version of the AI service may introduce additional routine logging that inadvertently logs sensitive user data without any way for a researcher to detect this. Similarly, a perimeter load balancer that terminates TLS may end up logging thousands of user requests wholesale during a troubleshooting session.
• It’s difficult to provide runtime transparency for AI in the cloud. Cloud AI services are opaque: providers do not typically specify details of the software stack they are using to run their services, and those details are often considered proprietary. Even if a cloud AI service relied only on open source software, which is inspectable by security researchers, there is no widely deployed way for a user device (or browser) to confirm that the service it’s connecting to is running an unmodified version of the software that it purports to run, or to detect that the software running on the service has changed.
• It’s challenging for cloud AI environments to enforce strong limits to privileged access. Cloud AI services are complex and expensive to run at scale, and their runtime performance and other operational metrics are constantly monitored and investigated by site reliability engineers and other administrative staff at the cloud service provider. During outages and other severe incidents, these administrators can generally make use of highly privileged access to the service, such as via SSH and equivalent remote shell interfaces. Though access controls for these privileged, break-glass interfaces may be well-designed, it’s exceptionally difficult to place enforceable limits on them while they’re in active use. For example, a service administrator who is trying to back up data from a live server during an outage could inadvertently copy sensitive user data in the process. More perniciously, criminals such as ransomware operators routinely strive to compromise service administrator credentials precisely to take advantage of privileged access interfaces and make away with user data.

When on-device computation with Apple devices such as iPhone and Mac is possible, the security and privacy advantages are clear: users control their own devices, researchers can inspect both hardware and software, runtime transparency is cryptographically assured through Secure Boot, and Apple retains no privileged access (as a concrete example, the Data Protection file encryption system cryptographically prevents Apple from disabling or guessing the passcode of a given iPhone).

However, to process more sophisticated requests, Apple Intelligence needs to be able to enlist help from larger, more complex models in the cloud. For these cloud requests to live up to the security and privacy guarantees that our users expect from our devices, the traditional cloud service security model isn't a viable starting point. Instead, we need to bring our industry-leading device security model, for the first time ever, to the cloud.

The rest of this post is an initial technical overview of Private Cloud Compute, to be followed by a deep dive after PCC becomes available in beta. We know researchers will have many detailed questions, and we look forward to answering more of them in our follow-up post.

Designing Private Cloud Compute

We set out to build Private Cloud Compute with a set of core requirements:

• Stateless computation on personal user data. Private Cloud Compute must use the personal user data that it receives exclusively for the purpose of fulfilling the user’s request. This data must never be available to anyone other than the user, not even to Apple staff, not even during active processing. And this data must not be retained, including via logging or for debugging, after the response is returned to the user. In other words, we want a strong form of stateless data processing where personal data leaves no trace in the PCC system.
• Enforceable guarantees. Security and privacy guarantees are strongest when they are entirely technically enforceable, which means it must be possible to constrain and analyze all the components that critically contribute to the guarantees of the overall Private Cloud Compute system. To use our example from earlier, it’s very difficult to reason about what a TLS-terminating load balancer may do with user data during a debugging session. Therefore, PCC must not depend on such external components for its core security and privacy guarantees. Similarly, operational requirements such as collecting server metrics and error logs must be supported with mechanisms that do not undermine privacy protections.
• No privileged runtime access. Private Cloud Compute must not contain privileged interfaces that would enable Apple’s site reliability staff to bypass PCC privacy guarantees, even when working to resolve an outage or other severe incident. This also means that PCC must not support a mechanism by which the privileged access envelope could be enlarged at runtime, such as by loading additional software.
• Non-targetability. An attacker should not be able to attempt to compromise personal data that belongs to specific, targeted Private Cloud Compute users without attempting a broad compromise of the entire PCC system. This must hold true even for exceptionally sophisticated attackers who can attempt physical attacks on PCC nodes in the supply chain or attempt to obtain malicious access to PCC data centers. In other words, a limited PCC compromise must not allow the attacker to steer requests from specific users to compromised nodes; targeting users should require a wide attack that’s likely to be detected. To understand this more intuitively, contrast it with a traditional cloud service design where every application server is provisioned with database credentials for the entire application database, so a compromise of a single application server is sufficient to access any user’s data, even if that user doesn’t have any active sessions with the compromised application server.
• Verifiable transparency. Security researchers need to be able to verify, with a high degree of confidence, that our privacy and security guarantees for Private Cloud Compute match our public promises. We already have an earlier requirement for our guarantees to be enforceable. Hypothetically, then, if security researchers had sufficient access to the system, they would be able to verify the guarantees. But this last requirement, verifiable transparency, goes one step further and does away with the hypothetical: security researchers must be able to verify the security and privacy guarantees of Private Cloud Compute, and they must be able to verify that the software that’s running in the PCC production environment is the same as the software they inspected when verifying the guarantees.

This is an extraordinary set of requirements, and one that we believe represents a generational leap over any traditional cloud service security model.

Introducing Private Cloud Compute nodes

The root of trust for Private Cloud Compute is our compute node: custom-built server hardware that brings the power and security of Apple silicon to the data center, with the same hardware security technologies used in iPhone, including the Secure Enclave and Secure Boot . We paired this hardware with a new operating system: a hardened subset of the foundations of iOS and macOS tailored to support Large Language Model (LLM) inference workloads while presenting an extremely narrow attack surface. This allows us to take advantage of iOS security technologies such as Code Signing and sandboxing .

On top of this foundation, we built a custom set of cloud extensions with privacy in mind. We excluded components that are traditionally critical to data center administration, such as remote shells and system introspection and observability tools. We replaced those general-purpose software components with components that are purpose-built to deterministically provide only a small, restricted set of operational metrics to SRE staff. And finally, we used Swift on Server to build a new Machine Learning stack specifically for hosting our cloud-based foundation model .

Let’s take another look at our core Private Cloud Compute requirements and the features we built to achieve them.

Stateless computation and enforceable guarantees

With services that are end-to-end encrypted, such as iMessage, the service operator cannot access the data that transits through the system. One of the key reasons such designs can assure privacy is specifically because they prevent the service from performing computations on user data. Since Private Cloud Compute needs to be able to access the data in the user’s request to allow a large foundation model to fulfill it, complete end-to-end encryption is not an option. Instead, the PCC compute node must have technical enforcement for the privacy of user data during processing, and must be incapable of retaining user data after its duty cycle is complete.

We designed Private Cloud Compute to make several guarantees about the way it handles user data:

• A user’s device sends data to PCC for the sole, exclusive purpose of fulfilling the user’s inference request. PCC uses that data only to perform the operations requested by the user.
• User data stays on the PCC nodes that are processing the request only until the response is returned. PCC deletes the user’s data after fulfilling the request, and no user data is retained in any form after the response is returned.
• User data is never available to Apple — even to staff with administrative access to the production service or hardware.

When Apple Intelligence needs to draw on Private Cloud Compute, it constructs a request — consisting of the prompt, plus the desired model and inferencing parameters — that will serve as input to the cloud model. The PCC client on the user’s device then encrypts this request directly to the public keys of the PCC nodes that it has first confirmed are valid and cryptographically certified. This provides end-to-end encryption from the user’s device to the validated PCC nodes, ensuring the request cannot be accessed in transit by anything outside those highly protected PCC nodes. Supporting data center services, such as load balancers and privacy gateways, run outside of this trust boundary and do not have the keys required to decrypt the user’s request, thus contributing to our enforceable guarantees.

Next, we must protect the integrity of the PCC node and prevent any tampering with the keys used by PCC to decrypt user requests. The system uses Secure Boot and Code Signing for an enforceable guarantee that only authorized and cryptographically measured code is executable on the node. All code that can run on the node must be part of a trust cache that has been signed by Apple, approved for that specific PCC node, and loaded by the Secure Enclave such that it cannot be changed or amended at runtime. This also ensures that JIT mappings cannot be created, preventing compilation or injection of new code at runtime. Additionally, all code and model assets use the same integrity protection that powers the Signed System Volume . Finally, the Secure Enclave provides an enforceable guarantee that the keys that are used to decrypt requests cannot be duplicated or extracted.

The Private Cloud Compute software stack is designed to ensure that user data is not leaked outside the trust boundary or retained once a request is complete, even in the presence of implementation errors. The Secure Enclave randomizes the data volume’s encryption keys on every reboot and does not persist these random keys , ensuring that data written to the data volume cannot be retained across reboot. In other words, there is an enforceable guarantee that the data volume is cryptographically erased every time the PCC node’s Secure Enclave Processor reboots. The inference process on the PCC node deletes data associated with a request upon completion, and the address spaces that are used to handle user data are periodically recycled to limit the impact of any data that may have been unexpectedly retained in memory.

Finally, for our enforceable guarantees to be meaningful, we also need to protect against exploitation that could bypass these guarantees. Technologies such as Pointer Authentication Codes and sandboxing act to resist such exploitation and limit an attacker’s horizontal movement within the PCC node. The inference control and dispatch layers are written in Swift, ensuring memory safety, and use separate address spaces to isolate initial processing of requests. This combination of memory safety and the principle of least privilege removes entire classes of attacks on the inference stack itself and limits the level of control and capability that a successful attack can obtain.

No privileged runtime access

We designed Private Cloud Compute to ensure that privileged access doesn’t allow anyone to bypass our stateless computation guarantees.

First, we intentionally did not include remote shell or interactive debugging mechanisms on the PCC node. Our Code Signing machinery prevents such mechanisms from loading additional code, but this sort of open-ended access would provide a broad attack surface to subvert the system’s security or privacy. Beyond simply not including a shell, remote or otherwise, PCC nodes cannot enable Developer Mode and do not include the tools needed by debugging workflows.

Next, we built the system’s observability and management tooling with privacy safeguards that are designed to prevent user data from being exposed. For example, the system doesn’t even include a general-purpose logging mechanism. Instead, only pre-specified, structured, and audited logs and metrics can leave the node, and multiple independent layers of review help prevent user data from accidentally being exposed through these mechanisms. With traditional cloud AI services, such mechanisms might allow someone with privileged access to observe or collect user data.

Together, these techniques provide enforceable guarantees that only specifically designated code has access to user data and that user data cannot leak outside the PCC node during system administration.

Non-targetability

Our threat model for Private Cloud Compute includes an attacker with physical access to a compute node and a high level of sophistication — that is, an attacker who has the resources and expertise to subvert some of the hardware security properties of the system and potentially extract data that is being actively processed by a compute node.

We defend against this type of attack in two ways:

• We supplement the built-in protections of Apple silicon with a hardened supply chain for PCC hardware, so that performing a hardware attack at scale would be both prohibitively expensive and likely to be discovered.
• We limit the impact of small-scale attacks by ensuring that they cannot be used to target the data of a specific user.

Private Cloud Compute hardware security starts at manufacturing, where we inventory and perform high-resolution imaging of the components of the PCC node before each server is sealed and its tamper switch is activated. When they arrive in the data center, we perform extensive revalidation before the servers are allowed to be provisioned for PCC. The process involves multiple Apple teams that cross-check data from independent sources, and the process is further monitored by a third-party observer not affiliated with Apple. At the end, a certificate is issued for keys rooted in the Secure Enclave UID for each PCC node. The user’s device will not send data to any PCC nodes if it cannot validate their certificates.

These processes broadly protect hardware from compromise. To guard against smaller, more sophisticated attacks that might otherwise avoid detection, Private Cloud Compute uses an approach we call target diffusion to ensure requests cannot be routed to specific nodes based on the user or their content.

Target diffusion starts with the request metadata, which leaves out any personally identifiable information about the source device or user, and includes only limited contextual data about the request that’s required to enable routing to the appropriate model. This metadata is the only part of the user’s request that is available to load balancers and other data center components running outside of the PCC trust boundary. The metadata also includes a single-use credential, based on RSA Blind Signatures , to authorize valid requests without tying them to a specific user. Additionally, PCC requests go through an OHTTP relay — operated by a third party — which hides the device’s source IP address before the request ever reaches the PCC infrastructure. This prevents an attacker from using an IP address to identify requests or associate them with an individual. It also means that an attacker would have to compromise both the third-party relay and our load balancer to steer traffic based on the source IP address.

User devices encrypt requests only for a subset of PCC nodes, rather than the PCC service as a whole. When asked by a user device, the load balancer returns a subset of PCC nodes that are most likely to be ready to process the user’s inference request — however, as the load balancer has no identifying information about the user or device for which it’s choosing nodes, it cannot bias the set for targeted users. By limiting the PCC nodes that can decrypt each request in this way, we ensure that if a single node were ever to be compromised, it would not be able to decrypt more than a small portion of incoming requests. Finally, the selection of PCC nodes by the load balancer is statistically auditable to protect against a highly sophisticated attack where the attacker compromises a PCC node as well as obtains complete control of the PCC load balancer.

Verifiable transparency

We consider allowing security researchers to verify the end-to-end security and privacy guarantees of Private Cloud Compute to be a critical requirement for ongoing public trust in the system. Traditional cloud services do not make their full production software images available to researchers — and even if they did, there’s no general mechanism to allow researchers to verify that those software images match what’s actually running in the production environment. (Some specialized mechanisms exist, such as Intel SGX and AWS Nitro attestation.)

When we launch Private Cloud Compute, we’ll take the extraordinary step of making software images of every production build of PCC publicly available for security research . This promise, too, is an enforceable guarantee: user devices will be willing to send data only to PCC nodes that can cryptographically attest to running publicly listed software. We want to ensure that security and privacy researchers can inspect Private Cloud Compute software, verify its functionality, and help identify issues — just like they can with Apple devices.

Our commitment to verifiable transparency includes:

• Publishing the measurements of all code running on PCC in an append-only and cryptographically tamper-proof transparency log.
• Making the log and associated binary software images publicly available for inspection and validation by privacy and security experts.
• Publishing and maintaining an official set of tools for researchers analyzing PCC node software.
• Rewarding important research findings through the Apple Security Bounty program.

Every production Private Cloud Compute software image will be published for independent binary inspection — including the OS, applications, and all relevant executables, which researchers can verify against the measurements in the transparency log. Software will be published within 90 days of inclusion in the log, or after relevant software updates are available, whichever is sooner. Once a release has been signed into the log, it cannot be removed without detection, much like the log-backed map data structure used by the Key Transparency mechanism for iMessage Contact Key Verification .

As we mentioned, user devices will ensure that they’re communicating only with PCC nodes running authorized and verifiable software images. Specifically, the user’s device will wrap its request payload key only to the public keys of those PCC nodes whose attested measurements match a software release in the public transparency log. And the same strict Code Signing technologies that prevent loading unauthorized software also ensure that all code on the PCC node is included in the attestation.

Making Private Cloud Compute software logged and inspectable in this way is a strong demonstration of our commitment to enable independent research on the platform. But we want to ensure researchers can rapidly get up to speed, verify our PCC privacy claims, and look for issues, so we’re going further with three specific steps:

• We’ll release a PCC Virtual Research Environment: a set of tools and images that simulate a PCC node on a Mac with Apple silicon, and that can boot a version of PCC software minimally modified for successful virtualization.
• While we’re publishing the binary images of every production PCC build, to further aid research we will periodically also publish a subset of the security-critical PCC source code.
• In a first for any Apple platform, PCC images will include the sepOS firmware and the iBoot bootloader in plaintext , making it easier than ever for researchers to study these critical components.

The Apple Security Bounty will reward research findings in the entire Private Cloud Compute software stack — with especially significant payouts for any issues that undermine our privacy claims.

More to come

Private Cloud Compute continues Apple’s profound commitment to user privacy. With sophisticated technologies to satisfy our requirements of stateless computation, enforceable guarantees, no privileged access, non-targetability, and verifiable transparency, we believe Private Cloud Compute is nothing short of the world-leading security architecture for cloud AI compute at scale.

We look forward to sharing many more technical details about PCC, including the implementation and behavior behind each of our core requirements. And we’re especially excited to soon invite security researchers for a first look at the Private Cloud Compute software and our PCC Virtual Research Environment.