critical thinking skills evaluation

• Social Learning Theory Explained
• Indirect Instruction Methods: A Comprehensive Overview

Problem-solving Skills: A Comprehensive Overview

• Physical Science Lesson Plans for 9-12 Learners
• Classroom Management
• Behavior management techniques
• Classroom rules
• Classroom routines
• Classroom organization
• Assessment Strategies
• Summative assessment techniques
• Formative assessment techniques
• Portfolio assessment
• Performance-based assessment
• Teaching Strategies
• Active learning
• Inquiry-based learning
• Differentiated instruction
• Project-based learning
• Learning Theories
• Behaviorism
• Social Learning Theory
• Cognitivism
• Constructivism
• Critical Thinking Skills
• Analysis skills
• Creative thinking skills
• Problem-solving skills
• Evaluation skills
• Metacognition
• Metacognitive strategies
• Self-reflection and metacognition
• Goal setting and metacognition
• Teaching Methods and Techniques
• Direct instruction methods
• Indirect instruction methods
• Lesson Planning Strategies
• Lesson sequencing strategies
• Unit planning strategies
• Differentiated Instruction Strategies
• Differentiated instruction for English language learners
• Differentiated instruction for gifted students
• Standards and Benchmarks
• State science standards and benchmarks
• National science standards and benchmarks
• Curriculum Design
• Course design and alignment
• Backward design principles
• Curriculum mapping
• Instructional Materials
• Textbooks and digital resources
• Instructional software and apps
• Engaging Activities and Games
• Hands-on activities and experiments
• Cooperative learning games
• Learning Environment Design
• Classroom technology integration
• Classroom layout and design
• Instructional Strategies
• Collaborative learning strategies
• Problem-based learning strategies
• 9-12 Science Lesson Plans
• Life science lesson plans for 9-12 learners
• Earth science lesson plans for 9-12 learners
• Physical science lesson plans for 9-12 learners
• K-5 Science Lesson Plans
• Earth science lesson plans for K-5 learners
• Life science lesson plans for K-5 learners
• Physical science lesson plans for K-5 learners
• 6-8 Science Lesson Plans
• Earth science lesson plans for 6-8 learners
• Life science lesson plans for 6-8 learners
• Physical science lesson plans for 6-8 learners
• Science Learning
• Evaluation Skills: A Comprehensive Overview

This article provides an overview of evaluation skills, including what they are, why they are important, and how to develop them. It is written for anyone interested in improving their science learning and critical thinking skills.

Evaluation skills are a vital part of any student's education and are essential for success in the modern world. From determining the quality of evidence to assessing the validity of an argument, evaluation skills are essential for critical thinking and problem-solving. In this comprehensive overview, we'll explore the importance of evaluation skills and how they can be developed and applied in all areas of life. From formulating hypotheses to understanding the different types of evidence and the criteria used to assess them, this article will provide an in-depth look at the evaluation process. We'll discuss why evaluation skills are so important, explore how they can be developed, and provide examples of how they can be used in everyday life.

For those looking for more guidance, Spires online BMAT tutors can provide expert advice on developing and applying evaluation skills. By the end of this article, readers will have a better understanding of the importance of evaluation skills, as well as a clear picture of how to develop and apply them in everyday life. Evaluation skills consist of a set of abilities that enable us to analyze information accurately and come to sound conclusions. These skills include critical thinking , problem-solving, data interpretation, and logical reasoning. They are important for science learning and critical thinking because they allow us to evaluate the validity of scientific theories and hypotheses. Developing evaluation skills requires practice and dedication.

It involves developing the ability to ask the right questions, observe carefully, think objectively, and draw accurate conclusions. To develop these skills, it is important to practice analyzing information from different sources and coming to sound conclusions. It is also important to practice questioning assumptions and challenging accepted beliefs. Additionally, it is helpful to read books and articles on science and critical thinking to better understand how to evaluate information accurately. Another important part of developing evaluation skills is understanding the importance of data accuracy and precision.

Data accuracy refers to how closely a measurement or estimate reflects the true value of a phenomenon, while data precision refers to how precise the measurement is. Understanding these concepts helps us evaluate data more accurately. Finally, developing evaluation skills requires honing our communication skills. Being able to effectively communicate our thoughts and opinions helps us share our ideas with others and develop our understanding of the material. Communicating clearly also helps us evaluate information more accurately by making sure that everyone is on the same page when discussing complex topics. In conclusion, evaluation skills are essential for effective learning in any subject.

How Can We Develop Evaluation Skills?

The importance of communication.

Communication allows us to share our ideas with others, which can then be discussed, debated, and further developed. Communication also allows us to develop a deeper understanding of the material by being able to ask questions and get feedback from peers or experts. In order to develop our communication skills, we need to learn how to express ourselves clearly and concisely. We must also learn how to actively listen to others and truly understand their perspectives. This will allow us to better engage in dialogue and form meaningful connections with our peers.

Why Are Evaluation Skills Important?

Understanding data accuracy & precision.

For example, if a measurement is off by 10% from the true value, then it has an accuracy of 90%. On the other hand, data precision is determined by the number of significant figures in a measurement. The more significant figures a measurement has, the more precise it is. In order to evaluate data accurately, it is important to understand how these two concepts interact. For example, if a measurement has high accuracy but low precision, then it may still be inaccurate because of the amount of error in the measurement.

Similarly, a measurement with high precision but low accuracy can still be inaccurate if the true value is significantly different from the measurement. By understanding the importance of both accuracy and precision in data evaluation, we can make more informed decisions about the validity of our conclusions. This is especially important when making decisions based on scientific evidence, as inaccurate or imprecise data can lead to incorrect conclusions and inaccurate results. In conclusion, evaluation skills are an essential part of science learning and critical thinking. Developing these skills requires practice and dedication, as well as an understanding of data accuracy and precision. Finally, effective communication is a key component of mastering these skills, as it allows us to share our ideas with others.

Shahid Lakha

Shahid Lakha is a seasoned educational consultant with a rich history in the independent education sector and EdTech. With a solid background in Physics, Shahid has cultivated a career that spans tutoring, consulting, and entrepreneurship. As an Educational Consultant at Spires Online Tutoring since October 2016, he has been instrumental in fostering educational excellence in the online tutoring space. Shahid is also the founder and director of Specialist Science Tutors, a tutoring agency based in West London, where he has successfully managed various facets of the business, including marketing, web design, and client relationships. His dedication to education is further evidenced by his role as a self-employed tutor, where he has been teaching Maths, Physics, and Engineering to students up to university level since September 2011. Shahid holds a Master of Science in Photon Science from the University of Manchester and a Bachelor of Science in Physics from the University of Bath.

New Articles

• Inquiry-Based Learning: An Introduction to Teaching Strategies

Learn about inquiry-based learning, a teaching approach that encourages students to ask questions, find answers, and make meaningful connections. Discover key strategies for implementing inquiry-based learning in your classroom.

• Earth Science Lesson Plans for 9-12 Learners

This article provides an overview of Earth Science lesson plans for 9-12 learners, and includes tips for making the lessons engaging and informative.

• Creative Thinking Skills

Learn all about Creative Thinking Skills, how to develop them, and why they are important for Science Learning & Critical Thinking.

This guide provides an in-depth overview of indirect instruction methods, including their applications and benefits.

Leave Reply

Your email address will not be published. Required fields are marked *

I agree that spam comments wont´t be published

• Behavior Management Techniques
• Behaviorism: A Comprehensive Overview
• Summative Assessment Techniques: An Overview
• Formative Assessment Techniques
• Active Learning: A Comprehensive Overview
• Understanding Cognitivism: A Learning Theory
• Analysis Skills: Understanding Critical Thinking and Science Learning
• Constructivism: Exploring the Theory of Learning
• Classroom Rules - A Comprehensive Overview
• Exploring Portfolio Assessment: An Introduction
• Differentiated Instruction: A Comprehensive Overview
• Classroom Routines: A Comprehensive Overview
• Effective Classroom Organization Strategies for Science Teaching
• Project-Based Learning: An In-Depth Look
• Performance-Based Assessment: A Comprehensive Overview
• Understanding Direct Instruction Methods
• State Science Standards and Benchmarks
• Course Design and Alignment
• The Advantages of Textbooks and Digital Resources
• Engaging Hands-on Activities and Experiments
• An Overview of Metacognitive Strategies
• Backward Design Principles: Understanding Curriculum Design
• Engaging Cooperative Learning Games
• Integrating Technology into the Classroom
• Understanding Classroom Layout and Design
• Lesson Sequencing Strategies: A Comprehensive Overview
• Instructional Software and Apps: A Comprehensive Overview
• Understanding Curriculum Mapping

Collaborative Learning Strategies

• Understanding National Science Standards and Benchmarks
• Exploring Problem-Based Learning Strategies
• Unit Planning Strategies
• Exploring Self-Reflection and Metacognition
• Exploring Goal Setting and Metacognition
• Life Science Lesson Plans for 9-12 Learners
• Earth Science Lesson Plans for K-5 Learners
• Differentiated Instruction for English Language Learners

Life Science Lesson Plans for K-5 Learners

• Earth Science Lesson Plans for 6-8 Learners
• Life Science Lesson Plans for 6-8 Learners
• Physical Science Lesson Plans for K-5 Learners
• Physical Science Lesson Plans for 6-8 Learners
• Differentiated Instruction Strategies for Gifted Students

Recent Posts

Which cookies do you want to accept?

Critical Thinking: 25 Performance Review Phrases Examples

By Status.net Editorial Team on July 15, 2023 — 8 minutes to read

Critical thinking skills are an essential aspect of an employee’s evaluation: the ability to solve problems, analyze situations, and make informed decisions is crucial for the success of any organization.

Questions that can help you determine an employee’s rating for critical thinking:

• Does the employee consistently analyze data and information to identify patterns and trends?
• Does the employee proactively identify potential problems and develop solutions to mitigate them?
• Has the employee demonstrated the ability to think creatively and come up with innovative ideas or approaches?
• Does the employee actively seek out feedback and input from others to inform their decision-making process?
• Has the employee demonstrated the ability to make sound decisions based on available information and data?

Performance Review Phrases and Paragraphs Examples For Critical Thinking

5 – outstanding.

Employees with outstanding critical thinking skills are exceptional at identifying patterns, making connections, and using past experiences to inform their decisions.

Phrases Examples

• Consistently demonstrates exceptional critical thinking abilities
• Always finds creative and innovative solutions to complex problems
• Skilfully analyzes information and data to make well-informed decisions
• Frequently provides valuable insights and perspectives that benefit the team
• Continuously seeks out new learning opportunities to sharpen their critical thinking skills
• Demonstrates exceptional ability to identify and analyze complex issues
• Consistently develops innovative solutions to problems
• Skillfully connects disparate ideas to create coherent arguments
• Effectively communicates well-reasoned conclusions
• Exceptional ability to recognize trends in data
• Expertly applies existing knowledge to new situations
• Consistently anticipates potential challenges and develops solution

Paragraph Example 1

“Jane consistently demonstrates outstanding critical thinking skills in her role. She not only engages in deep analysis of complex information, but she also presents unique solutions to problems that have a significant positive impact on the team’s performance. Her ability to make well-informed decisions and offer valuable insights has led to numerous successes for the organization. Moreover, Jane’s dedication to improvement and learning demonstrates her commitment to personal and professional growth in the area of critical thinking.”

Paragraph Example 2

“Jessica consistently displays outstanding critical thinking skills. She is able to identify and analyze complex issues with ease and has demonstrated her ability to develop innovative solutions. Her skill in connecting disparate ideas to create coherent arguments is impressive, and she excels at communicating her well-reasoned conclusions to the team.”

Paragraph Example 3

“Melanie consistently demonstrates an exceptional ability to recognize patterns and trends in data, which has significantly contributed to the success of our projects. Her critical thinking skills allow her to apply her extensive knowledge and experience in creative and innovative ways, proactively addressing potential challenges and developing effective solutions.”

4 – Exceeds Expectations

Employees exceeding expectations in critical thinking skills are adept at analyzing information, making sound decisions, and providing thoughtful recommendations. They are also effective at adapting their knowledge to novel situations and displaying confidence in their abilities.

• Excellent analytical capabilities
• Provides well-reasoned recommendations
• Demonstrates a solid understanding of complex concepts
• Regularly demonstrates the ability to think analytically and critically
• Effectively identifies and addresses complex problems with well-thought-out solutions
• Shows exceptional skill in generating innovative ideas and solutions
• Exhibits a consistently high level of decision-making based on sound reasoning
• Proactively seeks out new information to improve critical thinking skills
• Routinely identifies potential challenges and provides solutions
• Typically recognizes and prioritizes the most relevant information
• Logical thinking is evident in daily decision-making
• Often weighs the pros and cons of multiple options before selecting a course of action

“Eric’s critical thinking skills have consistently exceeded expectations throughout his tenure at the company. He is skilled at reviewing and analyzing complex information, leading him to provide well-reasoned recommendations and insights. Eric regularly demonstrates a deep understanding of complicated concepts, which allows him to excel in his role.”

“In this evaluation period, Jane has consistently demonstrated an exceptional ability to think critically and analytically. She has repeatedly shown skill in identifying complex issues while working on projects and has provided well-thought-out and effective solutions. Her innovative ideas have contributed significantly to the success of several key initiatives. Moreover, Jane’s decision-making skills are built on sound reasoning, which has led to positive outcomes for the team and organization. Additionally, she actively seeks opportunities to acquire new information and apply it to her work, further strengthening her critical thinking capabilities.”

“John consistently exceeds expectations in his critical thinking abilities. He routinely identifies potential challenges and provides thoughtful solutions. He is skilled at recognizing and prioritizing the most relevant information to make well-informed decisions. John regularly weighs the pros and cons of various options and selects the best course of action based on logic.”

3 – Meets Expectations

Employees meeting expectations in critical thinking skills demonstrate an ability to analyze information and draw logical conclusions. They are effective at problem-solving and can make informed decisions with minimal supervision.

• Capable of processing information and making informed decisions
• Displays problem-solving skills
• Demonstrates logical thinking and reasoning
• Consistently demonstrates the ability to analyze problems and find possible solutions.
• Actively engages in group discussions and contributes valuable ideas.
• Demonstrates the ability to draw conclusions based on logical analysis of information.
• Shows willingness to consider alternative perspectives when making decisions.
• Weighs the pros and cons of a situation before reaching a decision.
• Usually identifies relevant factors when faced with complex situations
• Demonstrates an understanding of cause and effect relationships
• Generally uses sound reasoning to make decisions
• Listens to and considers different perspectives

“Sarah consistently meets expectations in her critical thinking skills, successfully processing information and making informed decisions. She has shown her ability to solve problems effectively and displays logical reasoning when approaching new challenges. Sarah continues to be a valuable team member thanks to these critical thinking skills.”

“Jane is a team member who consistently meets expectations in regards to her critical thinking skills. She demonstrates an aptitude for analyzing problems within the workplace and actively seeks out potential solutions by collaborating with her colleagues. Jane is open-minded and makes an effort to consider alternative perspectives during decision-making processes. She carefully weighs the pros and cons of the situations she encounters, which helps her make informed choices that align with the company’s objectives.”

“David meets expectations in his critical thinking skills. He can usually identify the relevant factors when dealing with complex situations and demonstrates an understanding of cause and effect relationships. David’s decision-making is generally based on sound reasoning, and he listens to and considers different perspectives before reaching a conclusion.”

2 – Needs Improvement

Employees in need of improvement in critical thinking skills may struggle with processing information and making logical conclusions. They may require additional guidance when making decisions or solving problems.

• Struggles with analyzing complex information
• Requires guidance when working through challenges
• Difficulty applying past experiences to new situations
• With some guidance, Jane is able to think critically, but she struggles to do so independently.
• John tends to jump to conclusions without analyzing a situation fully.
• Sarah’s problem-solving skills need improvement, as she often overlooks important information when making decisions.
• David’s critical thinking skills are limited and need further development to enhance his overall work performance.
• Occasionally struggles to identify and analyze problems effectively
• Inconsistently uses logic to make decisions
• Often overlooks important information or perspectives
• Requires guidance in weighing options and making judgments

“Bob’s critical thinking skills could benefit from further development and improvement. He often struggles when analyzing complex information and tends to need additional guidance when working through challenges. Enhancing Bob’s ability to apply his past experiences to new situations would lead to a notable improvement in his overall performance.”

“Jenny is a valuable team member, but her critical thinking skills need improvement before she will be able to reach her full potential. In many instances, Jenny makes decisions based on her first impressions without questioning the validity of her assumptions or considering alternative perspectives. Her tendency to overlook key details has led to several instances in which her solutions are ineffective or only partly beneficial. With focused guidance and support, Jenny has the potential to develop her critical thinking skills and make more informed decisions in the future.”

“Tom’s critical thinking skills require improvement. He occasionally struggles to identify and analyze problems effectively, and his decision-making is inconsistent in its use of logic. Tom often overlooks important information or perspectives and may require guidance in weighing options and making judgments.”

1 – Unacceptable

Employees with unacceptable critical thinking skills lack the ability to analyze information effectively, struggle with decision-making, and fail to solve problems without extensive support from others.

• Fails to draw logical conclusions from information
• Incapable of making informed decisions
• Unable to solve problems without extensive assistance
• Fails to analyze potential problems before making decisions
• Struggles to think critically and ask relevant questions
• Cannot effectively identify alternative solutions
• Lacks the ability to apply logic and reason in problem-solving situations
• Does not consistently seek input from others or gather information before making a decision
• Regularly fails to recognize or address important issues
• Makes hasty decisions without considering potential consequences
• Lacks objectivity and often relies on personal biases
• Resistant to alternative viewpoints and constructive feedback

“Unfortunately, Sue’s critical thinking skills have been consistently unacceptable. She fails to draw logical conclusions from available information and is incapable of making informed decisions. Sue has also shown that she is unable to solve problems without extensive assistance from others, which significantly impacts her performance and the team’s productivity.”

“Jane’s performance in critical thinking has been unacceptable. She often fails to analyze potential problems before making decisions and struggles to think critically and ask relevant questions. Jane’s inability to effectively identify alternative solutions and apply logic and reason in problem-solving situations has negatively impacted her work. Furthermore, she does not consistently seek input from others or gather information before making a decision. It is crucial for Jane to improve her critical thinking skills to become a more effective and valuable team member.”

“Susan’s critical thinking skills are unacceptable. She regularly fails to recognize and address important issues, and her decision-making is often hasty and without considering potential consequences. Susan frequently lacks objectivity and tends to rely on personal biases. She is resistant to alternative viewpoints and constructive feedback, which negatively affects her work performance.”

• Strategic Thinking: 25 Performance Review Phrases Examples
• Flexibility: 25 Performance Review Phrases Examples
• Initiative: 25 Performance Review Phrases Examples
• Productivity: 25 Performance Review Phrases Examples
• Listening Skills: 25 Performance Review Phrases Examples
• Conflict Resolution: 25 Performance Review Phrases Examples

The Peak Performance Center

The pursuit of performance excellence, critical thinking.

Critical thinking refers to the process of actively analyzing, assessing, synthesizing, evaluating and reflecting on information gathered from observation, experience, or communication. It is thinking in a clear, logical, reasoned, and reflective manner to solve problems or make decisions. Basically, critical thinking is taking a hard look at something to understand what it really means.

Critical Thinkers

Critical thinkers do not simply accept all ideas, theories, and conclusions as facts. They have a mindset of questioning ideas and conclusions. They make reasoned judgments that are logical and well thought out by assessing the evidence that supports a specific theory or conclusion.

When presented with a new piece of new information, critical thinkers may ask questions such as;

“What information supports that?”

“How was this information obtained?”

“Who obtained the information?”

“How do we know the information is valid?”

“Why is it that way?”

“What makes it do that?”

“How do we know that?”

“Are there other possibilities?”

Combination of Analytical and Creative Thinking

Many people perceive critical thinking just as analytical thinking. However, critical thinking incorporates both analytical thinking and creative thinking. Critical thinking does involve breaking down information into parts and analyzing the parts in a logical, step-by-step manner. However, it also involves challenging consensus to formulate new creative ideas and generate innovative solutions. It is critical thinking that helps to evaluate and improve your creative ideas.

Elements of Critical Thinking

Critical thinking involves:

• Gathering relevant information
• Evaluating information
• Asking questions
• Assessing bias or unsubstantiated assumptions
• Making inferences from the information and filling in gaps
• Using abstract ideas to interpret information
• Formulating ideas
• Weighing opinions
• Reaching well-reasoned conclusions
• Considering alternative possibilities
• Testing conclusions
• Verifying if evidence/argument support the conclusions

Developing Critical Thinking Skills

Critical thinking is considered a higher order thinking skills, such as analysis, synthesis, deduction, inference, reason, and evaluation. In order to demonstrate critical thinking, you would need to develop skills in;

Interpreting : understanding the significance or meaning of information

Analyzing : breaking information down into its parts

Connecting : making connections between related items or pieces of information.

Integrating : connecting and combining information to better understand the relationship between the information.

Evaluating : judging the value, credibility, or strength of something

Reasoning : creating an argument through logical steps

Deducing : forming a logical opinion about something based on the information or evidence that is available

Inferring : figuring something out through reasoning based on assumptions and ideas

Generating : producing new information, ideas, products, or ways of viewing things.

Blooms Taxonomy

Bloom’s Taxonomy Revised

Mind Mapping

Chunking Information

Brainstorming

Copyright © 2024 | WordPress Theme by MH Themes

How to Evaluate a Job Candidate’s Critical Thinking Skills in an Interview

by Christopher Frank , Paul Magnone and Oded Netzer

Summary .   

Hiring is one of the most challenging competencies to master, yet it is one of the most strategic and impactful managerial functions. A McKinsey study quantified that superior talent is up to eight times more productive, showing that the relationship between talent quality and business performance is dramatic. Organizations seeking growth or simply survival during difficult times must successfully recruit A-list talent, thought leaders, and subject matter experts. This is often done under time constraints as you must quickly fill a key position. Essentially you are committing to a long-term relationship after a few very short dates.

Partner Center

ORIGINAL RESEARCH article

Performance assessment of critical thinking: conceptualization, design, and implementation.

• 1 Lynch School of Education and Human Development, Boston College, Chestnut Hill, MA, United States
• 2 Graduate School of Education, Stanford University, Stanford, CA, United States
• 3 Department of Business and Economics Education, Johannes Gutenberg University, Mainz, Germany

Enhancing students’ critical thinking (CT) skills is an essential goal of higher education. This article presents a systematic approach to conceptualizing and measuring CT. CT generally comprises the following mental processes: identifying, evaluating, and analyzing a problem; interpreting information; synthesizing evidence; and reporting a conclusion. We further posit that CT also involves dealing with dilemmas involving ambiguity or conflicts among principles and contradictory information. We argue that performance assessment provides the most realistic—and most credible—approach to measuring CT. From this conceptualization and construct definition, we describe one possible framework for building performance assessments of CT with attention to extended performance tasks within the assessment system. The framework is a product of an ongoing, collaborative effort, the International Performance Assessment of Learning (iPAL). The framework comprises four main aspects: (1) The storyline describes a carefully curated version of a complex, real-world situation. (2) The challenge frames the task to be accomplished (3). A portfolio of documents in a range of formats is drawn from multiple sources chosen to have specific characteristics. (4) The scoring rubric comprises a set of scales each linked to a facet of the construct. We discuss a number of use cases, as well as the challenges that arise with the use and valid interpretation of performance assessments. The final section presents elements of the iPAL research program that involve various refinements and extensions of the assessment framework, a number of empirical studies, along with linkages to current work in online reading and information processing.

Introduction

In their mission statements, most colleges declare that a principal goal is to develop students’ higher-order cognitive skills such as critical thinking (CT) and reasoning (e.g., Shavelson, 2010 ; Hyytinen et al., 2019 ). The importance of CT is echoed by business leaders ( Association of American Colleges and Universities [AACU], 2018 ), as well as by college faculty (for curricular analyses in Germany, see e.g., Zlatkin-Troitschanskaia et al., 2018 ). Indeed, in the 2019 administration of the Faculty Survey of Student Engagement (FSSE), 93% of faculty reported that they “very much” or “quite a bit” structure their courses to support student development with respect to thinking critically and analytically. In a listing of 21st century skills, CT was the most highly ranked among FSSE respondents ( Indiana University, 2019 ). Nevertheless, there is considerable evidence that many college students do not develop these skills to a satisfactory standard ( Arum and Roksa, 2011 ; Shavelson et al., 2019 ; Zlatkin-Troitschanskaia et al., 2019 ). This state of affairs represents a serious challenge to higher education – and to society at large.

In view of the importance of CT, as well as evidence of substantial variation in its development during college, its proper measurement is essential to tracking progress in skill development and to providing useful feedback to both teachers and learners. Feedback can help focus students’ attention on key skill areas in need of improvement, and provide insight to teachers on choices of pedagogical strategies and time allocation. Moreover, comparative studies at the program and institutional level can inform higher education leaders and policy makers.

The conceptualization and definition of CT presented here is closely related to models of information processing and online reasoning, the skills that are the focus of this special issue. These two skills are especially germane to the learning environments that college students experience today when much of their academic work is done online. Ideally, students should be capable of more than naïve Internet search, followed by copy-and-paste (e.g., McGrew et al., 2017 ); rather, for example, they should be able to critically evaluate both sources of evidence and the quality of the evidence itself in light of a given purpose ( Leu et al., 2020 ).

In this paper, we present a systematic approach to conceptualizing CT. From that conceptualization and construct definition, we present one possible framework for building performance assessments of CT with particular attention to extended performance tasks within the test environment. The penultimate section discusses some of the challenges that arise with the use and valid interpretation of performance assessment scores. We conclude the paper with a section on future perspectives in an emerging field of research – the iPAL program.

Conceptual Foundations, Definition and Measurement of Critical Thinking

In this section, we briefly review the concept of CT and its definition. In accordance with the principles of evidence-centered design (ECD; Mislevy et al., 2003 ), the conceptualization drives the measurement of the construct; that is, implementation of ECD directly links aspects of the assessment framework to specific facets of the construct. We then argue that performance assessments designed in accordance with such an assessment framework provide the most realistic—and most credible—approach to measuring CT. The section concludes with a sketch of an approach to CT measurement grounded in performance assessment .

Concept and Definition of Critical Thinking

Taxonomies of 21st century skills ( Pellegrino and Hilton, 2012 ) abound, and it is neither surprising that CT appears in most taxonomies of learning, nor that there are many different approaches to defining and operationalizing the construct of CT. There is, however, general agreement that CT is a multifaceted construct ( Liu et al., 2014 ). Liu et al. (2014) identified five key facets of CT: (i) evaluating evidence and the use of evidence; (ii) analyzing arguments; (iii) understanding implications and consequences; (iv) developing sound arguments; and (v) understanding causation and explanation.

There is empirical support for these facets from college faculty. A 2016–2017 survey conducted by the Higher Education Research Institute (HERI) at the University of California, Los Angeles found that a substantial majority of faculty respondents “frequently” encouraged students to: (i) evaluate the quality or reliability of the information they receive; (ii) recognize biases that affect their thinking; (iii) analyze multiple sources of information before coming to a conclusion; and (iv) support their opinions with a logical argument ( Stolzenberg et al., 2019 ).

There is general agreement that CT involves the following mental processes: identifying, evaluating, and analyzing a problem; interpreting information; synthesizing evidence; and reporting a conclusion (e.g., Erwin and Sebrell, 2003 ; Kosslyn and Nelson, 2017 ; Shavelson et al., 2018 ). We further suggest that CT includes dealing with dilemmas of ambiguity or conflict among principles and contradictory information ( Oser and Biedermann, 2020 ).

Importantly, Oser and Biedermann (2020) posit that CT can be manifested at three levels. The first level, Critical Analysis , is the most complex of the three levels. Critical Analysis requires both knowledge in a specific discipline (conceptual) and procedural analytical (deduction, inclusion, etc.) knowledge. The second level is Critical Reflection , which involves more generic skills “… necessary for every responsible member of a society” (p. 90). It is “a basic attitude that must be taken into consideration if (new) information is questioned to be true or false, reliable or not reliable, moral or immoral etc.” (p. 90). To engage in Critical Reflection, one needs not only apply analytic reasoning, but also adopt a reflective stance toward the political, social, and other consequences of choosing a course of action. It also involves analyzing the potential motives of various actors involved in the dilemma of interest. The third level, Critical Alertness , involves questioning one’s own or others’ thinking from a skeptical point of view.

Wheeler and Haertel (1993) categorized higher-order skills, such as CT, into two types: (i) when solving problems and making decisions in professional and everyday life, for instance, related to civic affairs and the environment; and (ii) in situations where various mental processes (e.g., comparing, evaluating, and justifying) are developed through formal instruction, usually in a discipline. Hence, in both settings, individuals must confront situations that typically involve a problematic event, contradictory information, and possibly conflicting principles. Indeed, there is an ongoing debate concerning whether CT should be evaluated using generic or discipline-based assessments ( Nagel et al., 2020 ). Whether CT skills are conceptualized as generic or discipline-specific has implications for how they are assessed and how they are incorporated into the classroom.

In the iPAL project, CT is characterized as a multifaceted construct that comprises conceptualizing, analyzing, drawing inferences or synthesizing information, evaluating claims, and applying the results of these reasoning processes to various purposes (e.g., solve a problem, decide on a course of action, find an answer to a given question or reach a conclusion) ( Shavelson et al., 2019 ). In the course of carrying out a CT task, an individual typically engages in activities such as specifying or clarifying a problem; deciding what information is relevant to the problem; evaluating the trustworthiness of information; avoiding judgmental errors based on “fast thinking”; avoiding biases and stereotypes; recognizing different perspectives and how they can reframe a situation; considering the consequences of alternative courses of actions; and communicating clearly and concisely decisions and actions. The order in which activities are carried out can vary among individuals and the processes can be non-linear and reciprocal.

In this article, we focus on generic CT skills. The importance of these skills derives not only from their utility in academic and professional settings, but also the many situations involving challenging moral and ethical issues – often framed in terms of conflicting principles and/or interests – to which individuals have to apply these skills ( Kegan, 1994 ; Tessier-Lavigne, 2020 ). Conflicts and dilemmas are ubiquitous in the contexts in which adults find themselves: work, family, civil society. Moreover, to remain viable in the global economic environment – one characterized by increased competition and advances in second generation artificial intelligence (AI) – today’s college students will need to continually develop and leverage their CT skills. Ideally, colleges offer a supportive environment in which students can develop and practice effective approaches to reasoning about and acting in learning, professional and everyday situations.

Measurement of Critical Thinking

Critical thinking is a multifaceted construct that poses many challenges to those who would develop relevant and valid assessments. For those interested in current approaches to the measurement of CT that are not the focus of this paper, consult Zlatkin-Troitschanskaia et al. (2018) .

In this paper, we have singled out performance assessment as it offers important advantages to measuring CT. Extant tests of CT typically employ response formats such as forced-choice or short-answer, and scenario-based tasks (for an overview, see Liu et al., 2014 ). They all suffer from moderate to severe construct underrepresentation; that is, they fail to capture important facets of the CT construct such as perspective taking and communication. High fidelity performance tasks are viewed as more authentic in that they provide a problem context and require responses that are more similar to what individuals confront in the real world than what is offered by traditional multiple-choice items ( Messick, 1994 ; Braun, 2019 ). This greater verisimilitude promises higher levels of construct representation and lower levels of construct-irrelevant variance. Such performance tasks have the capacity to measure facets of CT that are imperfectly assessed, if at all, using traditional assessments ( Lane and Stone, 2006 ; Braun, 2019 ; Shavelson et al., 2019 ). However, these assertions must be empirically validated, and the measures should be subjected to psychometric analyses. Evidence of the reliability, validity, and interpretative challenges of performance assessment (PA) are extensively detailed in Davey et al. (2015) .

We adopt the following definition of performance assessment:

A performance assessment (sometimes called a work sample when assessing job performance) … is an activity or set of activities that requires test takers, either individually or in groups, to generate products or performances in response to a complex, most often real-world task. These products and performances provide observable evidence bearing on test takers’ knowledge, skills, and abilities—their competencies—in completing the assessment ( Davey et al., 2015 , p. 10).

A performance assessment typically includes an extended performance task and short constructed-response and selected-response (i.e., multiple-choice) tasks (for examples, see Zlatkin-Troitschanskaia and Shavelson, 2019 ). In this paper, we refer to both individual performance- and constructed-response tasks as performance tasks (PT) (For an example, see Table 1 in section “iPAL Assessment Framework”).

Table 1. The iPAL assessment framework.

An Approach to Performance Assessment of Critical Thinking: The iPAL Program

The approach to CT presented here is the result of ongoing work undertaken by the International Performance Assessment of Learning collaborative (iPAL 1 ). iPAL is an international consortium of volunteers, primarily from academia, who have come together to address the dearth in higher education of research and practice in measuring CT with performance tasks ( Shavelson et al., 2018 ). In this section, we present iPAL’s assessment framework as the basis of measuring CT, with examples along the way.

iPAL Background

The iPAL assessment framework builds on the Council of Aid to Education’s Collegiate Learning Assessment (CLA). The CLA was designed to measure cross-disciplinary, generic competencies, such as CT, analytic reasoning, problem solving, and written communication ( Klein et al., 2007 ; Shavelson, 2010 ). Ideally, each PA contained an extended PT (e.g., examining a range of evidential materials related to the crash of an aircraft) and two short PT’s: one in which students either critique an argument or provide a solution in response to a real-world societal issue.

Motivated by considerations of adequate reliability, in 2012, the CLA was later modified to create the CLA+. The CLA+ includes two subtests: a PT and a 25-item Selected Response Question (SRQ) section. The PT presents a document or problem statement and an assignment based on that document which elicits an open-ended response. The CLA+ added the SRQ section (which is not linked substantively to the PT scenario) to increase the number of student responses to obtain more reliable estimates of performance at the student-level than could be achieved with a single PT ( Zahner, 2013 ; Davey et al., 2015 ).

iPAL Assessment Framework

Methodological foundations.

The iPAL framework evolved from the Collegiate Learning Assessment developed by Klein et al. (2007) . It was also informed by the results from the AHELO pilot study ( Organisation for Economic Co-operation and Development [OECD], 2012 , 2013 ), as well as the KoKoHs research program in Germany (for an overview see, Zlatkin-Troitschanskaia et al., 2017 , 2020 ). The ongoing refinement of the iPAL framework has been guided in part by the principles of Evidence Centered Design (ECD) ( Mislevy et al., 2003 ; Mislevy and Haertel, 2006 ; Haertel and Fujii, 2017 ).

In educational measurement, an assessment framework plays a critical intermediary role between the theoretical formulation of the construct and the development of the assessment instrument containing tasks (or items) intended to elicit evidence with respect to that construct ( Mislevy et al., 2003 ). Builders of the assessment framework draw on the construct theory and operationalize it in a way that provides explicit guidance to PT’s developers. Thus, the framework should reflect the relevant facets of the construct, where relevance is determined by substantive theory or an appropriate alternative such as behavioral samples from real-world situations of interest (criterion-sampling; McClelland, 1973 ), as well as the intended use(s) (for an example, see Shavelson et al., 2019 ). By following the requirements and guidelines embodied in the framework, instrument developers strengthen the claim of construct validity for the instrument ( Messick, 1994 ).

An assessment framework can be specified at different levels of granularity: an assessment battery (“omnibus” assessment, for an example see below), a single performance task, or a specific component of an assessment ( Shavelson, 2010 ; Davey et al., 2015 ). In the iPAL program, a performance assessment comprises one or more extended performance tasks and additional selected-response and short constructed-response items. The focus of the framework specified below is on a single PT intended to elicit evidence with respect to some facets of CT, such as the evaluation of the trustworthiness of the documents provided and the capacity to address conflicts of principles.

From the ECD perspective, an assessment is an instrument for generating information to support an evidentiary argument and, therefore, the intended inferences (claims) must guide each stage of the design process. The construct of interest is operationalized through the Student Model , which represents the target knowledge, skills, and abilities, as well as the relationships among them. The student model should also make explicit the assumptions regarding student competencies in foundational skills or content knowledge. The Task Model specifies the features of the problems or items posed to the respondent, with the goal of eliciting the evidence desired. The assessment framework also describes the collection of task models comprising the instrument, with considerations of construct validity, various psychometric characteristics (e.g., reliability) and practical constraints (e.g., testing time and cost). The student model provides grounds for evidence of validity, especially cognitive validity; namely, that the students are thinking critically in responding to the task(s).

In the present context, the target construct (CT) is the competence of individuals to think critically, which entails solving complex, real-world problems, and clearly communicating their conclusions or recommendations for action based on trustworthy, relevant and unbiased information. The situations, drawn from actual events, are challenging and may arise in many possible settings. In contrast to more reductionist approaches to assessment development, the iPAL approach and framework rests on the assumption that properly addressing these situational demands requires the application of a constellation of CT skills appropriate to the particular task presented (e.g., Shavelson, 2010 , 2013 ). For a PT, the assessment framework must also specify the rubric by which the responses will be evaluated. The rubric must be properly linked to the target construct so that the resulting score profile constitutes evidence that is both relevant and interpretable in terms of the student model (for an example, see Zlatkin-Troitschanskaia et al., 2019 ).

iPAL Task Framework

The iPAL ‘omnibus’ framework comprises four main aspects: A storyline , a challenge , a document library , and a scoring rubric . Table 1 displays these aspects, brief descriptions of each, and the corresponding examples drawn from an iPAL performance assessment (Version adapted from original in Hyytinen and Toom, 2019 ). Storylines are drawn from various domains; for example, the worlds of business, public policy, civics, medicine, and family. They often involve moral and/or ethical considerations. Deriving an appropriate storyline from a real-world situation requires careful consideration of which features are to be kept in toto , which adapted for purposes of the assessment, and which to be discarded. Framing the challenge demands care in wording so that there is minimal ambiguity in what is required of the respondent. The difficulty of the challenge depends, in large part, on the nature and extent of the information provided in the document library , the amount of scaffolding included, as well as the scope of the required response. The amount of information and the scope of the challenge should be commensurate with the amount of time available. As is evident from the table, the characteristics of the documents in the library are intended to elicit responses related to facets of CT. For example, with regard to bias, the information provided is intended to play to judgmental errors due to fast thinking and/or motivational reasoning. Ideally, the situation should accommodate multiple solutions of varying degrees of merit.

The dimensions of the scoring rubric are derived from the Task Model and Student Model ( Mislevy et al., 2003 ) and signal which features are to be extracted from the response and indicate how they are to be evaluated. There should be a direct link between the evaluation of the evidence and the claims that are made with respect to the key features of the task model and student model . More specifically, the task model specifies the various manipulations embodied in the PA and so informs scoring, while the student model specifies the capacities students employ in more or less effectively responding to the tasks. The score scales for each of the five facets of CT (see section “Concept and Definition of Critical Thinking”) can be specified using appropriate behavioral anchors (for examples, see Zlatkin-Troitschanskaia and Shavelson, 2019 ). Of particular importance is the evaluation of the response with respect to the last dimension of the scoring rubric; namely, the overall coherence and persuasiveness of the argument, building on the explicit or implicit characteristics related to the first five dimensions. The scoring process must be monitored carefully to ensure that (trained) raters are judging each response based on the same types of features and evaluation criteria ( Braun, 2019 ) as indicated by interrater agreement coefficients.

The scoring rubric of the iPAL omnibus framework can be modified for specific tasks ( Lane and Stone, 2006 ). This generic rubric helps ensure consistency across rubrics for different storylines. For example, Zlatkin-Troitschanskaia et al. (2019 , p. 473) used the following scoring scheme:

Based on our construct definition of CT and its four dimensions: (D1-Info) recognizing and evaluating information, (D2-Decision) recognizing and evaluating arguments and making decisions, (D3-Conseq) recognizing and evaluating the consequences of decisions, and (D4-Writing), we developed a corresponding analytic dimensional scoring … The students’ performance is evaluated along the four dimensions, which in turn are subdivided into a total of 23 indicators as (sub)categories of CT … For each dimension, we sought detailed evidence in students’ responses for the indicators and scored them on a six-point Likert-type scale. In order to reduce judgment distortions, an elaborate procedure of ‘behaviorally anchored rating scales’ (Smith and Kendall, 1963) was applied by assigning concrete behavioral expectations to certain scale points (Bernardin et al., 1976). To this end, we defined the scale levels by short descriptions of typical behavior and anchored them with concrete examples. … We trained four raters in 1 day using a specially developed training course to evaluate students’ performance along the 23 indicators clustered into four dimensions (for a description of the rater training, see Klotzer, 2018).

Shavelson et al. (2019) examined the interrater agreement of the scoring scheme developed by Zlatkin-Troitschanskaia et al. (2019) and “found that with 23 items and 2 raters the generalizability (“reliability”) coefficient for total scores to be 0.74 (with 4 raters, 0.84)” ( Shavelson et al., 2019 , p. 15). In the study by Zlatkin-Troitschanskaia et al. (2019 , p. 478) three score profiles were identified (low-, middle-, and high-performer) for students. Proper interpretation of such profiles requires care. For example, there may be multiple possible explanations for low scores such as poor CT skills, a lack of a disposition to engage with the challenge, or the two attributes jointly. These alternative explanations for student performance can potentially pose a threat to the evidentiary argument. In this case, auxiliary information may be available to aid in resolving the ambiguity. For example, student responses to selected- and short-constructed-response items in the PA can provide relevant information about the levels of the different skills possessed by the student. When sufficient data are available, the scores can be modeled statistically and/or qualitatively in such a way as to bring them to bear on the technical quality or interpretability of the claims of the assessment: reliability, validity, and utility evidence ( Davey et al., 2015 ; Zlatkin-Troitschanskaia et al., 2019 ). These kinds of concerns are less critical when PT’s are used in classroom settings. The instructor can draw on other sources of evidence, including direct discussion with the student.

Use of iPAL Performance Assessments in Educational Practice: Evidence From Preliminary Validation Studies

The assessment framework described here supports the development of a PT in a general setting. Many modifications are possible and, indeed, desirable. If the PT is to be more deeply embedded in a certain discipline (e.g., economics, law, or medicine), for example, then the framework must specify characteristics of the narrative and the complementary documents as to the breadth and depth of disciplinary knowledge that is represented.

At present, preliminary field trials employing the omnibus framework (i.e., a full set of documents) indicated that 60 min was generally an inadequate amount of time for students to engage with the full set of complementary documents and to craft a complete response to the challenge (for an example, see Shavelson et al., 2019 ). Accordingly, it would be helpful to develop modified frameworks for PT’s that require substantially less time. For an example, see a short performance assessment of civic online reasoning, requiring response times from 10 to 50 min ( Wineburg et al., 2016 ). Such assessment frameworks could be derived from the omnibus framework by focusing on a reduced number of facets of CT, and specifying the characteristics of the complementary documents to be included – or, perhaps, choices among sets of documents. In principle, one could build a ‘family’ of PT’s, each using the same (or nearly the same) storyline and a subset of the full collection of complementary documents.

Paul and Elder (2007) argue that the goal of CT assessments should be to provide faculty with important information about how well their instruction supports the development of students’ CT. In that spirit, the full family of PT’s could represent all facets of the construct while affording instructors and students more specific insights on strengths and weaknesses with respect to particular facets of CT. Moreover, the framework should be expanded to include the design of a set of short answer and/or multiple choice items to accompany the PT. Ideally, these additional items would be based on the same narrative as the PT to collect more nuanced information on students’ precursor skills such as reading comprehension, while enhancing the overall reliability of the assessment. Areas where students are under-prepared could be addressed before, or even in parallel with the development of the focal CT skills. The parallel approach follows the co-requisite model of developmental education. In other settings (e.g., for summative assessment), these complementary items would be administered after the PT to augment the evidence in relation to the various claims. The full PT taking 90 min or more could serve as a capstone assessment.

As we transition from simply delivering paper-based assessments by computer to taking full advantage of the affordances of a digital platform, we should learn from the hard-won lessons of the past so that we can make swifter progress with fewer missteps. In that regard, we must take validity as the touchstone – assessment design, development and deployment must all be tightly linked to the operational definition of the CT construct. Considerations of reliability and practicality come into play with various use cases that highlight different purposes for the assessment (for future perspectives, see next section).

The iPAL assessment framework represents a feasible compromise between commercial, standardized assessments of CT (e.g., Liu et al., 2014 ), on the one hand, and, on the other, freedom for individual faculty to develop assessment tasks according to idiosyncratic models. It imposes a degree of standardization on both task development and scoring, while still allowing some flexibility for faculty to tailor the assessment to meet their unique needs. In so doing, it addresses a key weakness of the AAC&U’s VALUE initiative 2 (retrieved 5/7/2020) that has achieved wide acceptance among United States colleges.

The VALUE initiative has produced generic scoring rubrics for 15 domains including CT, problem-solving and written communication. A rubric for a particular skill domain (e.g., critical thinking) has five to six dimensions with four ordered performance levels for each dimension (1 = lowest, 4 = highest). The performance levels are accompanied by language that is intended to clearly differentiate among levels. 3 Faculty are asked to submit student work products from a senior level course that is intended to yield evidence with respect to student learning outcomes in a particular domain and that, they believe, can elicit performances at the highest level. The collection of work products is then graded by faculty from other institutions who have been trained to apply the rubrics.

A principal difficulty is that there is neither a common framework to guide the design of the challenge, nor any control on task complexity and difficulty. Consequently, there is substantial heterogeneity in the quality and evidential value of the submitted responses. This also causes difficulties with task scoring and inter-rater reliability. Shavelson et al. (2009) discuss some of the problems arising with non-standardized collections of student work.

In this context, one advantage of the iPAL framework is that it can provide valuable guidance and an explicit structure for faculty in developing performance tasks for both instruction and formative assessment. When faculty design assessments, their focus is typically on content coverage rather than other potentially important characteristics, such as the degree of construct representation and the adequacy of their scoring procedures ( Braun, 2019 ).

Concluding Reflections

Challenges to interpretation and implementation.

Performance tasks such as those generated by iPAL are attractive instruments for assessing CT skills (e.g., Shavelson, 2010 ; Shavelson et al., 2019 ). The attraction mainly rests on the assumption that elaborated PT’s are more authentic (direct) and more completely capture facets of the target construct (i.e., possess greater construct representation) than the widely used selected-response tests. However, as Messick (1994) noted authenticity is a “promissory note” that must be redeemed with empirical research. In practice, there are trade-offs among authenticity, construct validity, and psychometric quality such as reliability ( Davey et al., 2015 ).

One reason for Messick (1994) caution is that authenticity does not guarantee construct validity. The latter must be established by drawing on multiple sources of evidence ( American Educational Research Association et al., 2014 ). Following the ECD principles in designing and developing the PT, as well as the associated scoring rubrics, constitutes an important type of evidence. Further, as Leighton (2019) argues, response process data (“cognitive validity”) is needed to validate claims regarding the cognitive complexity of PT’s. Relevant data can be obtained through cognitive laboratory studies involving methods such as think aloud protocols or eye-tracking. Although time-consuming and expensive, such studies can yield not only evidence of validity, but also valuable information to guide refinements of the PT.

Going forward, iPAL PT’s must be subjected to validation studies as recommended in the Standards for Psychological and Educational Testing by American Educational Research Association et al. (2014) . With a particular focus on the criterion “relationships to other variables,” a framework should include assumptions about the theoretically expected relationships among the indicators assessed by the PT, as well as the indicators’ relationships to external variables such as intelligence or prior (task-relevant) knowledge.

Complementing the necessity of evaluating construct validity, there is the need to consider potential sources of construct-irrelevant variance (CIV). One pertains to student motivation, which is typically greater when the stakes are higher. If students are not motivated, then their performance is likely to be impacted by factors unrelated to their (construct-relevant) ability ( Lane and Stone, 2006 ; Braun et al., 2011 ; Shavelson, 2013 ). Differential motivation across groups can also bias comparisons. Student motivation might be enhanced if the PT is administered in the context of a course with the promise of generating useful feedback on students’ skill profiles.

Construct-irrelevant variance can also occur when students are not equally prepared for the format of the PT or fully appreciate the response requirements. This source of CIV could be alleviated by providing students with practice PT’s. Finally, the use of novel forms of documentation, such as those from the Internet, can potentially introduce CIV due to differential familiarity with forms of representation or contents. Interestingly, this suggests that there may be a conflict between enhancing construct representation and reducing CIV.

Another potential source of CIV is related to response evaluation. Even with training, human raters can vary in accuracy and usage of the full score range. In addition, raters may attend to features of responses that are unrelated to the target construct, such as the length of the students’ responses or the frequency of grammatical errors ( Lane and Stone, 2006 ). Some of these sources of variance could be addressed in an online environment, where word processing software could alert students to potential grammatical and spelling errors before they submit their final work product.

Performance tasks generally take longer to administer and are more costly than traditional assessments, making it more difficult to reliably measure student performance ( Messick, 1994 ; Davey et al., 2015 ). Indeed, it is well known that more than one performance task is needed to obtain high reliability ( Shavelson, 2013 ). This is due to both student-task interactions and variability in scoring. Sources of student-task interactions are differential familiarity with the topic ( Hyytinen and Toom, 2019 ) and differential motivation to engage with the task. The level of reliability required, however, depends on the context of use. For use in formative assessment as part of an instructional program, reliability can be lower than use for summative purposes. In the former case, other types of evidence are generally available to support interpretation and guide pedagogical decisions. Further studies are needed to obtain estimates of reliability in typical instructional settings.

With sufficient data, more sophisticated psychometric analyses become possible. One challenge is that the assumption of unidimensionality required for many psychometric models might be untenable for performance tasks ( Davey et al., 2015 ). Davey et al. (2015) provide the example of a mathematics assessment that requires students to demonstrate not only their mathematics skills but also their written communication skills. Although the iPAL framework does not explicitly address students’ reading comprehension and organization skills, students will likely need to call on these abilities to accomplish the task. Moreover, as the operational definition of CT makes evident, the student must not only deploy several skills in responding to the challenge of the PT, but also carry out component tasks in sequence. The former requirement strongly indicates the need for a multi-dimensional IRT model, while the latter suggests that the usual assumption of local item independence may well be problematic ( Lane and Stone, 2006 ). At the same time, the analytic scoring rubric should facilitate the use of latent class analysis to partition data from large groups into meaningful categories ( Zlatkin-Troitschanskaia et al., 2019 ).

Future Perspectives

Although the iPAL consortium has made substantial progress in the assessment of CT, much remains to be done. Further refinement of existing PT’s and their adaptation to different languages and cultures must continue. To this point, there are a number of examples: The refugee crisis PT (cited in Table 1 ) was translated and adapted from Finnish to US English and then to Colombian Spanish. A PT concerning kidney transplants was translated and adapted from German to US English. Finally, two PT’s based on ‘legacy admissions’ to US colleges were translated and adapted to Colombian Spanish.

With respect to data collection, there is a need for sufficient data to support psychometric analysis of student responses, especially the relationships among the different components of the scoring rubric, as this would inform both task development and response evaluation ( Zlatkin-Troitschanskaia et al., 2019 ). In addition, more intensive study of response processes through cognitive laboratories and the like are needed to strengthen the evidential argument for construct validity ( Leighton, 2019 ). We are currently conducting empirical studies, collecting data on both iPAL PT’s and other measures of CT. These studies will provide evidence of convergent and discriminant validity.

At the same time, efforts should be directed at further development to support different ways CT PT’s might be used—i.e., use cases—especially those that call for formative use of PT’s. Incorporating formative assessment into courses can plausibly be expected to improve students’ competency acquisition ( Zlatkin-Troitschanskaia et al., 2017 ). With suitable choices of storylines, appropriate combinations of (modified) PT’s, supplemented by short-answer and multiple-choice items, could be interwoven into ordinary classroom activities. The supplementary items may be completely separate from the PT’s (as is the case with the CLA+), loosely coupled with the PT’s (as in drawing on the same storyline), or tightly linked to the PT’s (as in requiring elaboration of certain components of the response to the PT).

As an alternative to such integration, stand-alone modules could be embedded in courses to yield evidence of students’ generic CT skills. Core curriculum courses or general education courses offer ideal settings for embedding performance assessments. If these assessments were administered to a representative sample of students in each cohort over their years in college, the results would yield important information on the development of CT skills at a population level. For another example, these PA’s could be used to assess the competence profiles of students entering Bachelor’s or graduate-level programs as a basis for more targeted instructional support.

Thus, in considering different use cases for the assessment of CT, it is evident that several modifications of the iPAL omnibus assessment framework are needed. As noted earlier, assessments built according to this framework are demanding with respect to the extensive preliminary work required by a task and the time required to properly complete it. Thus, it would be helpful to have modified versions of the framework, focusing on one or two facets of the CT construct and calling for a smaller number of supplementary documents. The challenge to the student should be suitably reduced.

Some members of the iPAL collaborative have developed PT’s that are embedded in disciplines such as engineering, law and education ( Crump et al., 2019 ; for teacher education examples, see Jeschke et al., 2019 ). These are proving to be of great interest to various stakeholders and further development is likely. Consequently, it is essential that an appropriate assessment framework be established and implemented. It is both a conceptual and an empirical question as to whether a single framework can guide development in different domains.

Performance Assessment in Online Learning Environment

Over the last 15 years, increasing amounts of time in both college and work are spent using computers and other electronic devices. This has led to formulation of models for the new literacies that attempt to capture some key characteristics of these activities. A prominent example is a model proposed by Leu et al. (2020) . The model frames online reading as a process of problem-based inquiry that calls on five practices to occur during online research and comprehension:

1. Reading to identify important questions,

2. Reading to locate information,

3. Reading to critically evaluate information,

4. Reading to synthesize online information, and

5. Reading and writing to communicate online information.

The parallels with the iPAL definition of CT are evident and suggest there may be benefits to closer links between these two lines of research. For example, a report by Leu et al. (2014) describes empirical studies comparing assessments of online reading using either open-ended or multiple-choice response formats.

The iPAL consortium has begun to take advantage of the affordances of the online environment (for examples, see Schmidt et al. and Nagel et al. in this special issue). Most obviously, Supplementary Materials can now include archival photographs, audio recordings, or videos. Additional tasks might include the online search for relevant documents, though this would add considerably to the time demands. This online search could occur within a simulated Internet environment, as is the case for the IEA’s ePIRLS assessment ( Mullis et al., 2017 ).

The prospect of having access to a wealth of materials that can add to task authenticity is exciting. Yet it can also add ambiguity and information overload. Increased authenticity, then, should be weighed against validity concerns and the time required to absorb the content in these materials. Modifications of the design framework and extensive empirical testing will be required to decide on appropriate trade-offs. A related possibility is to employ some of these materials in short-answer (or even selected-response) items that supplement the main PT. Response formats could include highlighting text or using a drag-and-drop menu to construct a response. Students’ responses could be automatically scored, thereby containing costs. With automated scoring, feedback to students and faculty, including suggestions for next steps in strengthening CT skills, could also be provided without adding to faculty workload. Therefore, taking advantage of the online environment to incorporate new types of supplementary documents should be a high priority and, perhaps, to introduce new response formats as well. Finally, further investigation of the overlap between this formulation of CT and the characterization of online reading promulgated by Leu et al. (2020) is a promising direction to pursue.

Data Availability Statement

All datasets generated for this study are included in the article/supplementary material.

Author Contributions

HB wrote the article. RS, OZ-T, and KB were involved in the preparation and revision of the article and co-wrote the manuscript. All authors contributed to the article and approved the submitted version.

This study was funded in part by the Spencer Foundation (Grant No. #201700123).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

We would like to thank all the researchers who have participated in the iPAL program.

• ^ https://www.ipal-rd.com/
• ^ https://www.aacu.org/value
• ^ When test results are reported by means of substantively defined categories, the scoring is termed “criterion-referenced”. This is, in contrast to results, reported as percentiles; such scoring is termed “norm-referenced”.

American Educational Research Association, American Psychological Association, and National Council on Measurement in Education (2014). Standards for Educational and Psychological Testing. Washington, D.C: American Educational Research Association.

Google Scholar

Arum, R., and Roksa, J. (2011). Academically Adrift: Limited Learning on College Campuses. Chicago, IL: University of Chicago Press.

Association of American Colleges and Universities (n.d.). VALUE: What is value?. Available online at:: https://www.aacu.org/value (accessed May 7, 2020).

Association of American Colleges and Universities [AACU] (2018). Fulfilling the American Dream: Liberal Education and the Future of Work. Available online at:: https://www.aacu.org/research/2018-future-of-work (accessed May 1, 2020).

Braun, H. (2019). Performance assessment and standardization in higher education: a problematic conjunction? Br. J. Educ. Psychol. 89, 429–440. doi: 10.1111/bjep.12274

PubMed Abstract | CrossRef Full Text | Google Scholar

Braun, H. I., Kirsch, I., and Yamoto, K. (2011). An experimental study of the effects of monetary incentives on performance on the 12th grade NAEP reading assessment. Teach. Coll. Rec. 113, 2309–2344.

Crump, N., Sepulveda, C., Fajardo, A., and Aguilera, A. (2019). Systematization of performance tests in critical thinking: an interdisciplinary construction experience. Rev. Estud. Educ. 2, 17–47.

Davey, T., Ferrara, S., Shavelson, R., Holland, P., Webb, N., and Wise, L. (2015). Psychometric Considerations for the Next Generation of Performance Assessment. Washington, DC: Center for K-12 Assessment & Performance Management, Educational Testing Service.

Erwin, T. D., and Sebrell, K. W. (2003). Assessment of critical thinking: ETS’s tasks in critical thinking. J. Gen. Educ. 52, 50–70. doi: 10.1353/jge.2003.0019

CrossRef Full Text | Google Scholar

Haertel, G. D., and Fujii, R. (2017). “Evidence-centered design and postsecondary assessment,” in Handbook on Measurement, Assessment, and Evaluation in Higher Education , 2nd Edn, eds C. Secolsky and D. B. Denison (Abingdon: Routledge), 313–339. doi: 10.4324/9781315709307-26

Hyytinen, H., and Toom, A. (2019). Developing a performance assessment task in the Finnish higher education context: conceptual and empirical insights. Br. J. Educ. Psychol. 89, 551–563. doi: 10.1111/bjep.12283

Hyytinen, H., Toom, A., and Shavelson, R. J. (2019). “Enhancing scientific thinking through the development of critical thinking in higher education,” in Redefining Scientific Thinking for Higher Education: Higher-Order Thinking, Evidence-Based Reasoning and Research Skills , eds M. Murtonen and K. Balloo (London: Palgrave MacMillan).

Indiana University (2019). FSSE 2019 Frequencies: FSSE 2019 Aggregate. Available online at:: http://fsse.indiana.edu/pdf/FSSE_IR_2019/summary_tables/FSSE19_Frequencies_(FSSE_2019).pdf (accessed May 1, 2020).

Jeschke, C., Kuhn, C., Lindmeier, A., Zlatkin-Troitschanskaia, O., Saas, H., and Heinze, A. (2019). Performance assessment to investigate the domain specificity of instructional skills among pre-service and in-service teachers of mathematics and economics. Br. J. Educ. Psychol. 89, 538–550. doi: 10.1111/bjep.12277

Kegan, R. (1994). In Over Our Heads: The Mental Demands of Modern Life. Cambridge, MA: Harvard University Press.

Klein, S., Benjamin, R., Shavelson, R., and Bolus, R. (2007). The collegiate learning assessment: facts and fantasies. Eval. Rev. 31, 415–439. doi: 10.1177/0193841x07303318

Kosslyn, S. M., and Nelson, B. (2017). Building the Intentional University: Minerva and the Future of Higher Education. Cambridge, MAL: The MIT Press.

Lane, S., and Stone, C. A. (2006). “Performance assessment,” in Educational Measurement , 4th Edn, ed. R. L. Brennan (Lanham, MA: Rowman & Littlefield Publishers), 387–432.

Leighton, J. P. (2019). The risk–return trade-off: performance assessments and cognitive validation of inferences. Br. J. Educ. Psychol. 89, 441–455. doi: 10.1111/bjep.12271

Leu, D. J., Kiili, C., Forzani, E., Zawilinski, L., McVerry, J. G., and O’Byrne, W. I. (2020). “The new literacies of online research and comprehension,” in The Concise Encyclopedia of Applied Linguistics , ed. C. A. Chapelle (Oxford: Wiley-Blackwell), 844–852.

Leu, D. J., Kulikowich, J. M., Kennedy, C., and Maykel, C. (2014). “The ORCA Project: designing technology-based assessments for online research,” in Paper Presented at the American Educational Research Annual Meeting , Philadelphia, PA.

Liu, O. L., Frankel, L., and Roohr, K. C. (2014). Assessing critical thinking in higher education: current state and directions for next-generation assessments. ETS Res. Rep. Ser. 1, 1–23. doi: 10.1002/ets2.12009

McClelland, D. C. (1973). Testing for competence rather than for “intelligence.”. Am. Psychol. 28, 1–14. doi: 10.1037/h0034092

McGrew, S., Ortega, T., Breakstone, J., and Wineburg, S. (2017). The challenge that’s bigger than fake news: civic reasoning in a social media environment. Am. Educ. 4, 4-9, 39.

Mejía, A., Mariño, J. P., and Molina, A. (2019). Incorporating perspective analysis into critical thinking performance assessments. Br. J. Educ. Psychol. 89, 456–467. doi: 10.1111/bjep.12297

Messick, S. (1994). The interplay of evidence and consequences in the validation of performance assessments. Educ. Res. 23, 13–23. doi: 10.3102/0013189x023002013

Mislevy, R. J., Almond, R. G., and Lukas, J. F. (2003). A brief introduction to evidence-centered design. ETS Res. Rep. Ser. 2003, i–29. doi: 10.1002/j.2333-8504.2003.tb01908.x

Mislevy, R. J., and Haertel, G. D. (2006). Implications of evidence-centered design for educational testing. Educ. Meas. Issues Pract. 25, 6–20. doi: 10.1111/j.1745-3992.2006.00075.x

Mullis, I. V. S., Martin, M. O., Foy, P., and Hooper, M. (2017). ePIRLS 2016 International Results in Online Informational Reading. Available online at:: http://timssandpirls.bc.edu/pirls2016/international-results/ (accessed May 1, 2020).

Nagel, M.-T., Zlatkin-Troitschanskaia, O., Schmidt, S., and Beck, K. (2020). “Performance assessment of generic and domain-specific skills in higher education economics,” in Student Learning in German Higher Education , eds O. Zlatkin-Troitschanskaia, H. A. Pant, M. Toepper, and C. Lautenbach (Berlin: Springer), 281–299. doi: 10.1007/978-3-658-27886-1_14

Organisation for Economic Co-operation and Development [OECD] (2012). AHELO: Feasibility Study Report , Vol. 1. Paris: OECD. Design and implementation.

Organisation for Economic Co-operation and Development [OECD] (2013). AHELO: Feasibility Study Report , Vol. 2. Paris: OECD. Data analysis and national experiences.

Oser, F. K., and Biedermann, H. (2020). “A three-level model for critical thinking: critical alertness, critical reflection, and critical analysis,” in Frontiers and Advances in Positive Learning in the Age of Information (PLATO) , ed. O. Zlatkin-Troitschanskaia (Cham: Springer), 89–106. doi: 10.1007/978-3-030-26578-6_7

Paul, R., and Elder, L. (2007). Consequential validity: using assessment to drive instruction. Found. Crit. Think. 29, 31–40.

Pellegrino, J. W., and Hilton, M. L. (eds) (2012). Education for life and work: Developing Transferable Knowledge and Skills in the 21st Century. Washington DC: National Academies Press.

Shavelson, R. (2010). Measuring College Learning Responsibly: Accountability in a New Era. Redwood City, CA: Stanford University Press.

Shavelson, R. J. (2013). On an approach to testing and modeling competence. Educ. Psychol. 48, 73–86. doi: 10.1080/00461520.2013.779483

Shavelson, R. J., Zlatkin-Troitschanskaia, O., Beck, K., Schmidt, S., and Marino, J. P. (2019). Assessment of university students’ critical thinking: next generation performance assessment. Int. J. Test. 19, 337–362. doi: 10.1080/15305058.2018.1543309

Shavelson, R. J., Zlatkin-Troitschanskaia, O., and Marino, J. P. (2018). “International performance assessment of learning in higher education (iPAL): research and development,” in Assessment of Learning Outcomes in Higher Education: Cross-National Comparisons and Perspectives , eds O. Zlatkin-Troitschanskaia, M. Toepper, H. A. Pant, C. Lautenbach, and C. Kuhn (Berlin: Springer), 193–214. doi: 10.1007/978-3-319-74338-7_10

Shavelson, R. J., Klein, S., and Benjamin, R. (2009). The limitations of portfolios. Inside Higher Educ. Available online at: https://www.insidehighered.com/views/2009/10/16/limitations-portfolios

Stolzenberg, E. B., Eagan, M. K., Zimmerman, H. B., Berdan Lozano, J., Cesar-Davis, N. M., Aragon, M. C., et al. (2019). Undergraduate Teaching Faculty: The HERI Faculty Survey 2016–2017. Los Angeles, CA: UCLA.

Tessier-Lavigne, M. (2020). Putting Ethics at the Heart of Innovation. Stanford, CA: Stanford Magazine.

Wheeler, P., and Haertel, G. D. (1993). Resource Handbook on Performance Assessment and Measurement: A Tool for Students, Practitioners, and Policymakers. Palm Coast, FL: Owl Press.

Wineburg, S., McGrew, S., Breakstone, J., and Ortega, T. (2016). Evaluating Information: The Cornerstone of Civic Online Reasoning. Executive Summary. Stanford, CA: Stanford History Education Group.

Zahner, D. (2013). Reliability and Validity–CLA+. Council for Aid to Education. Available online at:: https://pdfs.semanticscholar.org/91ae/8edfac44bce3bed37d8c9091da01d6db3776.pdf .

Zlatkin-Troitschanskaia, O., and Shavelson, R. J. (2019). Performance assessment of student learning in higher education [Special issue]. Br. J. Educ. Psychol. 89, i–iv, 413–563.

Zlatkin-Troitschanskaia, O., Pant, H. A., Lautenbach, C., Molerov, D., Toepper, M., and Brückner, S. (2017). Modeling and Measuring Competencies in Higher Education: Approaches to Challenges in Higher Education Policy and Practice. Berlin: Springer VS.

Zlatkin-Troitschanskaia, O., Pant, H. A., Toepper, M., and Lautenbach, C. (eds) (2020). Student Learning in German Higher Education: Innovative Measurement Approaches and Research Results. Wiesbaden: Springer.

Zlatkin-Troitschanskaia, O., Shavelson, R. J., and Pant, H. A. (2018). “Assessment of learning outcomes in higher education: international comparisons and perspectives,” in Handbook on Measurement, Assessment, and Evaluation in Higher Education , 2nd Edn, eds C. Secolsky and D. B. Denison (Abingdon: Routledge), 686–697.

Zlatkin-Troitschanskaia, O., Shavelson, R. J., Schmidt, S., and Beck, K. (2019). On the complementarity of holistic and analytic approaches to performance assessment scoring. Br. J. Educ. Psychol. 89, 468–484. doi: 10.1111/bjep.12286

Keywords : critical thinking, performance assessment, assessment framework, scoring rubric, evidence-centered design, 21st century skills, higher education

Citation: Braun HI, Shavelson RJ, Zlatkin-Troitschanskaia O and Borowiec K (2020) Performance Assessment of Critical Thinking: Conceptualization, Design, and Implementation. Front. Educ. 5:156. doi: 10.3389/feduc.2020.00156

Received: 30 May 2020; Accepted: 04 August 2020; Published: 08 September 2020.

Reviewed by:

Copyright © 2020 Braun, Shavelson, Zlatkin-Troitschanskaia and Borowiec. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Henry I. Braun, [email protected]

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.

• Our Team of Presenters
• Fellows of the Foundation
• Dr. Richard Paul
• Dr. Linda Elder
• Dr. Gerald Nosich
• Contact Us - Office Information
• Permission to Use Our Work
• Create a CriticalThinking.Org Account
• Contributions to the Foundation for Critical Thinking
• Testimonials
• Center for Critical Thinking
• The National Council for Excellence in Critical Thinking
• International Center for the Assessment of Higher Order Thinking
• Library of Critical Thinking Resources
• Professional Development
• Inservice Information Request Form
• Certification Online Course
• The State of Critical Thinking Today
• Higher Education
• K-12 Instruction
• Customized Webinars and Online Courses for Faculty
• Business & Professional Groups
• The Center for Critical Thinking Community Online
• Certification in the Paul-Elder Approach to Critical Thinking
• Professional Development Model - College and University
• Professional Development Model for K-12
• Workshop Descriptions
• Online Courses in Critical Thinking
• Critical Thinking Training for Law Enforcement
• Consulting for Leaders and Key Personnel at Your Organization
• Critical Thinking Therapy
• Conferences & Events
• Upcoming Learning Opportunities
• 2024 Fall Academy on Critical Thinking
• Daily Schedule
• Transportation, Lodging, and Social Functions
• Critical Thinking Therapy Release & Book Signing
• Academy Presuppositions
• 45th Annual Conference: Dates
• Presuppositions of the Conference
• Call for Proposals
• Conference Archives
• 44th Annual International Conference on Critical Thinking
• Focal Session Descriptions
• Guest Presentation Program
• Presuppositions of the 44th Annual International Conference on Critical Thinking
• Recommended Reading
• 43rd Annual International Conference on Critical Thinking
• Register as an Ambassador
• Testimonials from Past Attendees
• Thank You to Our Donors
• 42nd Annual International Conference on Critical Thinking
• Overview of Sessions (Flyer)
• Presuppositions of the Annual International Conference
• Testimonials from Past Conferences
• 41st Annual International Conference on Critical Thinking
• Recommended Publications
• Dedication to Our Donors
• 40th Annual International Conference on Critical Thinking
• Session Descriptions
• Testimonials from Prior Conferences
• International Critical Thinking Manifesto
• Scholarships Available
• 39th Annual International Conference on Critical Thinking
• Travel and Lodging Info
• FAQ & General Announcements
• Focal and Plenary Session Descriptions
• Program and Proceedings of the 39th Annual International Conference on Critical Thinking
• The Venue: KU Leuven
• Call for Critical Thinking Ambassadors
• Conference Background Information
• 38th Annual International Conference on Critical Thinking
• Call for Ambassadors for Critical Thinking
• Conference Focal Session Descriptions
• Conference Concurrent Session Descriptions
• Conference Roundtable Discussions
• Conference Announcements and FAQ
• Conference Program and Proceedings
• Conference Daily Schedule
• Conference Hotel Information
• Conference Academic Credit
• Conference Presuppositions
• What Participants Have Said About the Conference
• 37th Annual International Conference on Critical Thinking
• Registration & Fees
• FAQ and Announcements
• Conference Presenters
• 37th Conference Flyer
• Program and Proceedings of the 37th Conference
• 36th International Conference
• Conference Sessions
• Conference Flyer
• Program and Proceedings
• Academic Credit
• 35th International Conference
• Conference Session Descriptions
• Available Online Sessions
• Bertrand Russell Distinguished Scholar - Daniel Ellsberg
• 35th International Conference Program
• Concurrent Sessions
• Posthumous Bertrand Russell Scholar
• Hotel Information
• Conference FAQs
• Visiting UC Berkeley
• 34th INTERNATIONAL CONFERENCE
• Bertrand Russell Distinguished Scholar - Ralph Nader
• Conference Concurrent Presenters
• Conference Program
• Conference Theme
• Roundtable Discussions
• Flyer for Bulletin Boards
• 33rd INTERNATIONAL CONFERENCE
• 33rd International Conference Program
• 33rd International Conference Sessions
• 33rd International Conference Presenters
• The Bertrand Russell Distinguished Scholars Critical Thinking Conversations
• 33rd International Conference - Fees & Registration
• 33rd International Conference Concurrent Presenters
• 33rd International Conference - Hotel Information
• 33rd International Conference Flyer
• 32nd INTERNATIONAL CONFERENCE
• 32nd Annual Conference Sessions
• 32nd Annual Conference Presenter Information
• 32nd Conference Program
• The Bertrand Russell Distinguished Scholars Critical Thinking Lecture Series
• 32nd Annual Conference Concurrent Presenters
• 32nd Annual Conference Academic Credit
• 31st INTERNATIONAL CONFERENCE
• 31st Conference Sessions
• Comments about previous conferences
• Conference Hotel (2011)
• 31st Concurrent Presenters
• Registration Fees
• 31st International Conference
• 30th INTERNATIONAL CONFERENCE ON CRITICAL THINKING
• 30th International Conference Theme
• 30th Conference Sessions
• PreConference Sessions
• 30th Concurrent Presenters
• 30th Conference Presuppositions
• Hilton Garden Inn
• 29th International Conference
• 29th Conference Theme
• 29th Conference Sessions
• 29th Preconference Sessions
• 29th Conference Concurrent Sessions
• 2008 International Conference on Critical Thinking
• 2008 Preconference Sessions (28th Intl. Conference)
• 2007 Conference on Critical Thinking (Main Page)
• 2007 Conference Theme and sessions
• 2007 Pre-Conference Workshops
• 2006 Annual International Conference (archived)
• 2006 International Conference Theme
• 2005 International Conference (archived)
• Prior Conference Programs (Pre 2000)
• Workshop Archives
• Spring 2022 Online Workshops
• 2021 Online Workshops for Winter & Spring
• 2019 Seminar for Military and Intelligence Trainers and Instructors
• Transportation, Lodging, and Recreation
• Seminar Flyer
• 2013 Spring Workshops
• Our Presenters
• 2013 Spring Workshops - Hotel Information
• 2013 Spring Workshops Flyer
• 2013 Spring Workshops - Schedule
• Spring Workshop 2012
• 2012 Spring Workshop Strands
• 2012 Spring Workshop Flier
• 2011 Spring Workshop
• Spring 2010 Workshop Strands
• 2009 Spring Workshops on Critical Thinking
• 2008 SPRING Workshops and Seminars on Critical Thinking
• 2008 Ethical Reasoning Workshop
• 2008 - On Richard Paul's Teaching Design
• 2008 Engineering Reasoning Workshop
• 2008 Academia sobre Formulando Preguntas Esenciales
• Fellows Academy Archives
• 2017 Fall International Fellows Academy
• 4th International Fellows Academy - 2016
• 3rd International Fellows Academy
• 2nd International Fellows Academy
• 1st International Fellows Academy
• Academy Archives
• October 2019 Critical Thinking Academy for Educators and Administrators
• Transportation, Lodging, and Leisure
• Advanced Seminar: Oxford Tutorial
• Recreational Group Activities
• Limited Scholarships Available
• September 2019 Critical Thinking Educators and Administrators Academy
• 2019 Critical Thinking Training for Trainers and Advanced Academy
• Academy Flyer
• Seattle, WA 2017 Spring Academy
• San Diego, CA 2017 Spring Academy
• 2016 Spring Academy -- Washington D.C.
• 2016 Spring Academy -- Houston, TX
• The 2nd International Academy on Critical Thinking (Oxford 2008)
• 2007 National Academy on Critical Thinking Testing and Assessment
• 2006 Cambridge Academy (archived)
• 2006 Cambridge Academy Theme
• 2006 Cambridge Academy Sessions
• Accommodations at St. John's College
• Assessment & Testing
• A Model for the National Assessment of Higher Order Thinking
• International Critical Thinking Essay Test
• Online Critical Thinking Basic Concepts Test
• Online Critical Thinking Basic Concepts Sample Test
• Consequential Validity: Using Assessment to Drive Instruction
• News & Announcements
• Newest Pages Added to CriticalThinking.Org
• Online Learning
• Critical Thinking Online Courses
• Critical Thinking Blog
• 2019 Blog Entries
• 2020 Blog Entries
• 2021 Blog Entries
• 2022 Blog Entries
• 2023 Blog Entries
• Online Courses for Your Students
• 2023 Webinar Archives
• 2022 Webinar Archives
• 2021 Webinar Archive
• 2020 Webinar Archive
• Guided Study Groups
• Critical Thinking Channel on YouTube
• CT800: Fall 2024

Translate this page from English...

*Machine translated pages not guaranteed for accuracy. Click Here for our professional translations.

Critical Thinking: Where to Begin

• For College and University Faculty
• For College and University Students
• For High School Teachers
• For Jr. High School Teachers
• For Elementary Teachers (Grades 4-6)
• For Elementary Teachers (Kindergarten - 3rd Grade)
• For Science and Engineering Instruction
• For Business and Professional Development
• For Nursing and Health Care
• For Home Schooling and Home Study

If you are new to critical thinking or wish to deepen your conception of it, we recommend you review the content below and bookmark this page for future reference.

Our Conception of Critical Thinking...

"Critical thinking is the intellectually disciplined process of actively and skillfully conceptualizing, applying, analyzing, synthesizing, and/or evaluating information gathered from, or generated by, observation, experience, reflection, reasoning, or communication, as a guide to belief and action. In its exemplary form, it is based on universal intellectual values that transcend subject matter divisions: clarity, accuracy, precision, consistency, relevance, sound evidence, good reasons, depth, breadth, and fairness..."

"Critical thinking is self-guided, self-disciplined thinking which attempts to reason at the highest level of quality in a fairminded way. People who think critically attempt, with consistent and conscious effort, to live rationally, reasonably, and empathically. They are keenly aware of the inherently flawed nature of human thinking when left unchecked. They strive to diminish the power of their egocentric and sociocentric tendencies. They use the intellectual tools that critical thinking offers – concepts and principles that enable them to analyze, assess, and improve thinking. They work diligently to develop the intellectual virtues of intellectual integrity, intellectual humility, intellectual civility, intellectual empathy, intellectual sense of justice and confidence in reason. They realize that no matter how skilled they are as thinkers, they can always improve their reasoning abilities and they will at times fall prey to mistakes in reasoning, human irrationality, prejudices, biases, distortions, uncritically accepted social rules and taboos, self-interest, and vested interest.

They strive to improve the world in whatever ways they can and contribute to a more rational, civilized society. At the same time, they recognize the complexities often inherent in doing so. They strive never to think simplistically about complicated issues and always to consider the rights and needs of relevant others. They recognize the complexities in developing as thinkers, and commit themselves to life-long practice toward self-improvement. They embody the Socratic principle: The unexamined life is not worth living , because they realize that many unexamined lives together result in an uncritical, unjust, dangerous world."

Why Critical Thinking?

The Problem:

Everyone thinks; it is our nature to do so. But much of our thinking, left to itself, is biased, distorted, partial, uninformed, or down-right prejudiced. Yet the quality of our lives and that of what we produce, make, or build depends precisely on the quality of our thought. Shoddy thinking is costly, both in money and in quality of life. Excellence in thought, however, must be systematically cultivated.

A Brief Definition:

Critical thinking is the art of analyzing and evaluating thinking with a view to improving it. The Result: 

  A well-cultivated critical thinker:

• raises vital questions and problems, formulating them clearly and precisely;
• gathers and assesses relevant information, using abstract ideas to interpret it effectively;
• comes to well-reasoned conclusions and solutions, testing them against relevant criteria and standards;
• thinks openmindedly within alternative systems of thought, recognizing and assessing, as need be, their assumptions, implications, and practical consequences; and
• communicates effectively with others in figuring out solutions to complex problems.

Critical thinking is, in short, self-directed, self-disciplined, self-monitored, and self-corrective thinking. It requires rigorous standards of excellence and mindful command of their use. It entails effective communication and problem-solving abilities, and a commitment to overcoming our native egocentrism and sociocentrism. Read more about our concept of critical thinking .

The Essential Dimensions of Critical Thinking

Our conception of critical thinking is based on the substantive approach developed by Dr. Richard Paul and his colleagues at the Center and Foundation for Critical Thinking over multiple decades. It is relevant to every subject, discipline, and profession, and to reasoning through the problems of everyday life. It entails five essential dimensions of critical thinking:

At the left is an overview of the first three dimensions. In sum, the elements or structures of thought enable us to "take our thinking apart" and analyze it. The intellectual standards are used to assess and evaluate the elements. The intellectual traits are dispositions of mind embodied by the fairminded critical thinker. To cultivate the mind, we need command of these essential dimensions, and we need to consistently apply them as we think through the many problems and issues in our lives.

The Elements of Reasoning and Intellectual Standards

To learn more about the elements of thought and how to apply the intellectual standards, check out our interactive model. Simply click on the link below, scroll to the bottom of the page, and explore the model with your mouse.

Why the Analysis of Thinking Is Important If you want to think well, you must understand at least the rudiments of thought, the most basic structures out of which all thinking is made. You must learn how to take thinking apart. Analyzing the Logic of a Subject When we understand the elements of reasoning, we realize that all subjects, all disciplines, have a fundamental logic defined by the structures of thought embedded within them. Therefore, to lay bare a subject’s most fundamental logic, we should begin with these questions:

Going Deeper...

The Critical Thinking Bookstore  

Our online bookstore houses numerous books and teacher's manuals , Thinker's Guides , videos , and other educational materials .  

Learn From Our Fellows and Scholars

Watch our Event Calendar , which provides an overview of all upcoming conferences and academies hosted by the Foundation for Critical Thinking. Clicking an entry on the Event Calendar will bring up that event's details, and the option to register. For those interested in online learning, the Foundation offers accredited online courses in critical thinking for both educators and the general public, as well as an online test for evaluating basic comprehension of critical thinking concepts . We are in the process of developing more online learning tools and tests to offer the community.  

Utilizing this Website

This website contains large amounts research and an online library of articles , both of which are freely available to the public. We also invite you to become a member of the Critical Thinking Community , where you will gain access to more tools and materials.  If you cannot locate a resource on a specific topic or concept, try searching for it using our Search Tool . The Search Tool is at the upper-right of every page on the website.