analysis of competing hypothesis

• Search Menu

Sign in through your institution

• Browse content in Arts and Humanities
• Browse content in Archaeology
• Archaeological Methodology and Techniques
• Historical Archaeology
• Prehistoric Archaeology
• Browse content in Art
• History of Art
• Browse content in Classical Studies
• Classical History
• Classical Philosophy
• Classical Literature
• Greek and Roman Archaeology
• Late Antiquity
• Religion in the Ancient World
• Browse content in History
• Colonialism and Imperialism
• Diplomatic History
• Environmental History
• History by Period
• History of Gender and Sexuality
• Intellectual History
• Legal and Constitutional History
• Maritime History
• Military History
• Political History
• Regional and National History
• Slavery and Abolition of Slavery
• Social and Cultural History
• Theory, Methods, and Historiography
• Language Teaching and Learning
• Browse content in Linguistics
• Cognitive Linguistics
• Computational Linguistics
• Forensic Linguistics
• Grammar, Syntax and Morphology
• Historical and Diachronic Linguistics
• History of English
• Language Acquisition
• Language Variation
• Language Families
• Linguistic Theories
• Linguistic Anthropology
• Phonetics and Phonology
• Psycholinguistics
• Sociolinguistics
• Writing Systems
• Browse content in Literature
• Children's Literature Studies
• Literary Studies (European)
• Literary Studies (Romanticism)
• Literary Studies (American)
• Literary Studies - World
• Literary Studies (1500 to 1800)
• Literary Studies (19th Century)
• Literary Studies (20th Century onwards)
• Literary Studies (African American Literature)
• Literary Studies (British and Irish)
• Literary Studies (Early and Medieval)
• Literary Studies (Fiction, Novelists, and Prose Writers)
• Literary Studies (Gender Studies)
• Literary Studies (History of the Book)
• Literary Studies (Plays and Playwrights)
• Literary Studies (Poetry and Poets)
• Literary Studies (Postcolonial Literature)
• Literary Studies (Travel Literature)
• Literary Studies (War Literature)
• Literary Studies (Women's Writing)
• Literary Theory and Cultural Studies
• Shakespeare Studies and Criticism
• Browse content in Media Studies
• Browse content in Philosophy
• Aesthetics and Philosophy of Art
• Epistemology
• Feminist Philosophy
• History of Western Philosophy
• Metaphysics
• Moral Philosophy
• Philosophy of Science
• Philosophy of Law
• Philosophy of Religion
• Philosophy of Language
• Philosophy of Mind
• Philosophy of Mathematics and Logic
• Practical Ethics
• Social and Political Philosophy
• Browse content in Religion
• Christianity
• History of Religion
• Judaism and Jewish Studies
• Religion and Politics
• Religion and Art, Literature, and Music
• Religious Studies
• Browse content in Society and Culture
• Cultural Studies
• Customs and Traditions
• Ethical Issues and Debates
• Natural world, Country Life, and Pets
• Popular Beliefs and Controversial Knowledge
• Sports and Outdoor Recreation
• Browse content in Law
• Company and Commercial Law
• Browse content in Comparative Law
• Systems of Law
• Constitutional and Administrative Law
• Criminal Law
• History of Law
• Human Rights and Immigration
• Browse content in International Law
• Private International Law and Conflict of Laws
• Public International Law
• Jurisprudence and Philosophy of Law
• Legal System and Practice
• Medical and Healthcare Law
• Browse content in Trusts Law
• Wills and Probate or Succession
• Browse content in Medicine and Health
• History of Medicine
• Browse content in Science and Mathematics
• History of Science and Technology
• Browse content in Social Sciences
• Browse content in Anthropology
• Regional Anthropology
• Social and Cultural Anthropology
• Browse content in Economics
• Economic History
• Browse content in Education
• Educational Strategies and Policy
• Higher and Further Education
• Browse content in Politics
• Asian Politics
• Comparative Politics
• Conflict Politics
• European Union
• Human Rights and Politics
• International Relations
• Middle Eastern Politics
• Political Theory
• Political Behaviour
• Political Economy
• Political Institutions
• Public Policy
• Russian Politics
• Security Studies
• UK Politics
• US Politics
• Browse content in Regional and Area Studies
• African Studies
• Middle Eastern Studies
• Scottish Studies
• Browse content in Sociology
• Gender and Sexuality
• Sociology of Religion
• Reviews and Awards
• Journals on Oxford Academic
• Books on Oxford Academic

Cognitive Bias in Intelligence Analysis: Testing the Analysis of Competing Hypotheses Method

• Cite Icon Cite

Belief, Bias and Intelligence outlines an approach for reducing the risk of cognitive biases impacting intelligence analysis that draws from experimental research in the social sciences. It critiques the reliance of Western intelligence agencies on the use of a method for intelligence analysis developed by the CIA in the 1990’s, the Analysis of Competing Hypotheses (ACH). The book shows that the theoretical basis of the ACH method is significantly flawed, and that there is no empirical basis for the use of ACH in mitigating cognitive biases. It puts ACH to the test in an experimental setting against two key cognitive biases with unique empirical research facilitated by UK’s Professional Heads of Intelligence Analysis unit at the Cabinet Office, includes meta-analysis into which analytical factors increase and reduce the risk of cognitive bias and recommends an alternative approach to risk mitigation for intelligence communities. Finally, it proposes alternative models for explaining the underlying causes of cognitive biases, challenging current leading theories in the social sciences.

Personal account

• Sign in with email/username & password
• Get email alerts
• Save searches
• Purchase content
• Activate your purchase/trial code
• Add your ORCID iD

Institutional access

Sign in with a library card.

• Sign in with username/password
• Recommend to your librarian
• Institutional account management
• Get help with access

Access to content on Oxford Academic is often provided through institutional subscriptions and purchases. If you are a member of an institution with an active account, you may be able to access content in one of the following ways:

IP based access

Typically, access is provided across an institutional network to a range of IP addresses. This authentication occurs automatically, and it is not possible to sign out of an IP authenticated account.

Choose this option to get remote access when outside your institution. Shibboleth/Open Athens technology is used to provide single sign-on between your institution’s website and Oxford Academic.

• Click Sign in through your institution.
• Select your institution from the list provided, which will take you to your institution's website to sign in.
• When on the institution site, please use the credentials provided by your institution. Do not use an Oxford Academic personal account.
• Following successful sign in, you will be returned to Oxford Academic.

If your institution is not listed or you cannot sign in to your institution’s website, please contact your librarian or administrator.

Enter your library card number to sign in. If you cannot sign in, please contact your librarian.

Society Members

Society member access to a journal is achieved in one of the following ways:

Sign in through society site

Many societies offer single sign-on between the society website and Oxford Academic. If you see ‘Sign in through society site’ in the sign in pane within a journal:

• Click Sign in through society site.
• When on the society site, please use the credentials provided by that society. Do not use an Oxford Academic personal account.

If you do not have a society account or have forgotten your username or password, please contact your society.

Sign in using a personal account

Some societies use Oxford Academic personal accounts to provide access to their members. See below.

A personal account can be used to get email alerts, save searches, purchase content, and activate subscriptions.

Some societies use Oxford Academic personal accounts to provide access to their members.

Viewing your signed in accounts

Click the account icon in the top right to:

• View your signed in personal account and access account management features.
• View the institutional accounts that are providing access.

Signed in but can't access content

Oxford Academic is home to a wide variety of products. The institutional subscription may not cover the content that you are trying to access. If you believe you should have access to that content, please contact your librarian.

For librarians and administrators, your personal account also provides access to institutional account management. Here you will find options to view and activate subscriptions, manage institutional settings and access options, access usage statistics, and more.

Our books are available by subscription or purchase to libraries and institutions.

• About Oxford Academic
• Publish journals with us
• University press partners
• What we publish
• New features  
• Open access
• Rights and permissions
• Accessibility
• Advertising
• Media enquiries
• Oxford University Press
• Oxford Languages
• University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

• Copyright © 2024 Oxford University Press
• Cookie settings
• Cookie policy
• Privacy policy
• Legal notice

This Feature Is Available To Subscribers Only

Sign In or Create an Account

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

Academia.edu no longer supports Internet Explorer.

To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to  upgrade your browser .

Enter the email address you signed up with and we'll email you a reset link.

• We're Hiring!
• Help Center

The ‘Analysis of Competing Hypotheses’ in Intelligence Analysis

Applied Cognitive Psychology

Related Papers

Peter Pirolli

Human Factors: The Journal of the Human Factors and Ergonomics Society

Harvey Smallman

vii Author’s Preface This volume pulls together and republishes, with some editing, updating, and additions, articles written during 1978–86 for internal use within the CIA Directorate of Intelligence. Four of the articles also appeared in the Intelligence Community journal Studies in Intelligence during that time frame. The information is relatively timeless and still relevant to the never-ending quest for better analysis. The articles are based on reviewing cognitive psychology literature concerning how people process information to make judgments on in - complete and ambiguous information. I selected the experiments and findings that seem most relevant to intelligence analysis and most in need of communication to intelligence analysts. I then translated the techni - cal reports into language that intelligence analysts can understand and interpreted the relevance of these findings to the problems intelligence analysts face. The result is a compromise that may not be wholly satisfactory to either research psychologists or intelligence analysts. Cognitive psychol - ogists and decision analysts may complain of oversimplification, while the non-psychologist reader may have to absorb some new terminology. Unfortunately, mental processes are so complex that discussion of them does require some specialized vocabulary. Intelligence analysts who have read and thought seriously about the nature of their craft should have no difficulty with this book. Those who are plowing virgin ground may require serious effort. I wish to thank all those who contributed comments and suggestions on the draft of this book: Jack Davis (who also wrote the Introduction); four former Directorate of Intelligence (DI) analysts whose names cannot be cited here; my current colleague, Prof. Theodore Sarbin; and my edi - tor at the CIA’s Center for the Study of Intelligence, Hank Appelbaum. All made many substantive and editorial suggestions that helped greatly to make this a better book.

PsycEXTRA Dataset

Bezpečnostní teorie a praxe 1/2019 přehledový článek

Alp Cenk Arslan

20th and 21st centuries’ intelligence failures put an emphasis on the requirement of evaluating the influence of psychological biases on intelligence analysis. The aim of this study is to provide a discussion on how psychological biases affect intelligence analysis. Accordingly, conceptual discussion upon the notions of analysis and psychological bias is presented, and the biases that could have impacts on the intelligence analysis are defined in the study. By doing this, it is aimed to raise a clear understanding on the article topic through analyzing prominent sources and official reports in the literature. United Kingdom’s failure to foresee Falkland’s crisis in 1980s and United States’ intelligence analysis onWeapons of Mass Destruction in Iraq in the beginning of 2000s is selected as case studies in order to discuss how psychological biases affected intelligence analysis and accordingly caused intelligence failure. In conclusion, recommendations are presented in order to overcome the impact of psychological biases in intelligence analysis. Keywords: Intelligence analysis, psychological bias, Falkland’s crisis, weapons of mass destruction

Marvin Cohen

Emily Patterson

Pavla Vrbkova

Military Psychology

Mandeep Dhami

Analytic performance may be assessed by the nature of the process applied to intelligence tasks and analysts are expected to use a “critical” or deliberative mindset. However, there is little research on how analysts do their work. We report the findings of a quantitative survey of 113 intelligence analysts who were asked to report how often they would apply strategies involving more or less critical thinking when performing representative tasks along the analytic workflow. Analysts reported using “deliberative” strategies significantly more often than “intuitive” ones when capturing customer requirements, processing data, and communicating conclusions. Years of experience working in the intelligence community, skill level, analytic thinking training, and time spent working collaboratively (opposed to individually) were largely unrelated to reported strategy use. We discuss the implications of these findings for both improving intelligence analysis and developing an evidence-based a...

Jason Proctor

Loading Preview

Sorry, preview is currently unavailable. You can download the paper by clicking the button above.

RELATED PAPERS

Journal of Cognitive Engineering and Decision Making

Routledge Handbook of Bounded Rationality

natia jolokhava

Barbara Mellers

Mihai Boicu

American Intelligence Journal

Stephen Marrin

Ilmārs Lejnieks

Psychology, 14, 264-294

María Dolores Valiña

Rajesh Sharma

Matylda Gerber

Jeff Corkill

Dalene Duvenage (DSyRM)

B.Kristin Degirmenci

Computing in Science & Engineering

Gheorghe Tecuci

Pablo Pablo

Procedia - Social and Behavioral Sciences

Rene Diekstra

Mary Manjikian

Proceedings of the 2005 International Conference on Intelligence Analysis

Jean Scholtz

RELATED TOPICS

•   We're Hiring!
•   Help Center
• Find new research papers in:
• Health Sciences
• Earth Sciences
• Cognitive Science
• Mathematics
• Computer Science
• Academia ©2024

BUS610: Business Intelligence and Analytics

Analysis of competing hypotheses.

ACH is one of the best for ensuring a complete audit trail and keeping analysts "honest" by ensuring they include all evidence without bias. A country stability report is one of the most basic studies new intelligence studies students complete. They use ACH to determine whether an assigned country will likely be stable in the next 18 months. Yes or No are the only options. This does not help a decision-maker trying to ensure a region of the world remains stable; they need more information. Structured ACH allows the analyst to repeat the ACH exercise, drilling deeper into the analysis until the available evidence has been exhausted. For instance, the nation of Diania is expected to become unstable in the next 18 months. So what? The analyst can repeat the analysis with hypotheses positing that instability will be caused by H1: Inflation or by H2: Unemployment. H2 is disproven because the country has had high unemployment for the past ten years, and the informal economy now functions well enough that the official numbers no longer matter.

H1, however, is of concern. Diania is in a tense standoff with neighboring Ruania over their shared main river access, with Diania starting to build a hydroelectric dam for a self-sufficient energy source. The dam will not be operational for five years. In the meantime, Ruania is threatening to double the price of oil it now sells to Duania, and winter is coming. People are likely to be unable to afford fuel to heat their homes. In this case, the long-term energy independence strategy will have little value if people freeze to death next month due to inflated oil prices. NOW the decision-maker has something to work with beyond "likely to be unstable in 18 months". They can either support the dam building, provide a subsidy for the increased cost of oil, or offer to broker an agreement between the two nations. Adding structure helps ACH to provide decision-makers with far more actionable intelligence.

Have you used ACH? Can you think of a recent decision at work or home in which using a SACH matrix might have helped you decide? Use an ACH matrix to decide between two open positions you have saved on your favorite job announcement app. Does ACH help you keep your bias from the process and make it more objective? Or did it only show you that there is one more attractive position for reasons you had not articulated before you undertook your analysis? Now you know what they are, and even if your ACH shoots them down, you may still want them... Even if ACH does not eliminate bias, when applied to personal decisions, it can at least reveal what they are. Acknowledgment is half the battle against irrational decision-making.

The analysis of competing hypotheses (ACH) is a methodology for evaluating multiple competing hypotheses for observed data. It was developed by Richards (Dick) J. Heuer, Jr., a 45-year veteran of the Central Intelligence Agency, in the 1970s for use by the Agency. ACH is used by analysts in various fields who make judgments that entail a high risk of error in reasoning. ACH aims to help an analyst overcome, or at least minimize, some of the cognitive limitations that make prescient intelligence analysis so difficult to achieve. ACH was a step forward in intelligence analysis methodology, but it was first described in relatively informal terms. Producing the best available information from uncertain data remains the goal of researchers, tool-builders, and analysts in industry, academia and government. Their domains include data mining, cognitive psychology and visualization, probability and statistics, etc. Abductive reasoning is an earlier concept with similarities to ACH.

Heuer outlines the ACH process in considerable depth in his book, Psychology of Intelligence Analysis. It consists of the following steps:

• Hypothesis – The first step of the process is to identify all potential hypotheses, preferably using a group of analysts with different perspectives to brainstorm the possibilities. The process discourages the analyst from choosing one "likely" hypothesis and using evidence to prove its accuracy. Cognitive bias is minimized when all possible hypotheses are considered.
• Evidence – The analyst then lists evidence and arguments (including assumptions and logical deductions) for and against each hypothesis.
• Diagnostics – Using a matrix, the analyst applies evidence against each hypothesis in an attempt to disprove as many theories as possible. Some evidence will have greater "diagnosticity" than other evidence - that is, some will be more helpful in judging the relative likelihood of alternative hypotheses. This step is the most important, according to Heuer. Instead of looking at one hypothesis and all the evidence ("working down" the matrix), the analyst is encouraged to consider one piece of evidence at a time, and examine it against all possible hypotheses ("working across" the matrix).
• Refinement – The analyst reviews the findings, identifies any gaps, and collects any additional evidence needed to refute as many of the remaining hypotheses as possible.
• Inconsistency – The analyst then seeks to draw tentative conclusions about the relative likelihood of each hypothesis. Less consistency implies a lower likelihood. The least consistent hypotheses are eliminated. While the matrix generates a definitive mathematical total for each hypothesis, the analyst must use their judgment to make the final conclusion. The result of the ACH analysis itself must not overrule analysts' own judgments.
• Sensitivity – The analyst tests the conclusions using sensitivity analysis, which weighs how the conclusion would be affected if key evidence or arguments were wrong, misleading, or subject to different interpretations. The validity of key evidence and the consistency of important arguments are double-checked to assure the soundness of the conclusion's linchpins and drivers.
• Conclusions and evaluation – Finally, the analyst provides the decisionmaker with his or her conclusions, as well as a summary of alternatives that were considered and why they were rejected. The analyst also identifies milestones in the process that can serve as indicators in future analyses.

Some benefits of doing an ACH matrix are:

• It is auditable.
• It is widely believed to help overcome cognitive biases, though there is a lack of strong empirical evidence to support this belief.
• Since the ACH requires the analyst to construct a matrix, the evidence and hypotheses can be backtracked. This allows the decisionmaker or other analysts to see the sequence of rules and data that led to the conclusion.

Weaknesses of doing an ACH matrix include:

• The process to create an ACH is time consuming.
• The ACH matrix can be problematic when analyzing a complex project.
• It can be cumbersome for an analyst to manage a large database with multiple pieces of evidence.
• Evidence also presents a problem if it is unreliable.
• The evidence used in the matrix is static and therefore it can be a snapshot in time.

Especially in intelligence, both governmental and business, analysts must always be aware that the opponent(s) is intelligent and may be generating information intended to deceive. Since deception often is the result of a cognitive trap, Elsaesser and Stech use state-based hierarchical plan recognition (see abductive reasoning) to generate causal explanations of observations. The resulting hypotheses are converted to a dynamic Bayesian network and value of information analysis is employed to isolate assumptions implicit in the evaluation of paths in, or conclusions of, particular hypotheses. As evidence in the form of observations of states or assumptions is observed, they can become the subject of separate validation. Should an assumption or necessary state be negated, hypotheses depending on it are rejected. This is a form of root cause analysis. According to social constructivist critics, ACH also fails to stress sufficiently (or to address as a method) the problematic nature of the initial formation of the hypotheses used to create its grid. There is considerable evidence, for example, that in addition to any bureaucratic, psychological, or political biases that may affect hypothesis generation, there are also factors of culture and identity at work. These socially constructed factors may restrict or pre-screen which hypotheses end up being considered, and then reinforce confirmation bias in those selected. Philosopher and argumentation theorist Tim van Gelder has made the following criticisms:

• ACH demands that the analyst makes too many discrete judgments, a great many of which contribute little if anything to discerning the best hypothesis
• ACH misconceives the nature of the relationship between items of evidence and hypotheses by supposing that items of evidence are, on their own, consistent or inconsistent with hypotheses.
• ACH treats the hypothesis set as "flat", i.e. a mere list, and so is unable to relate evidence to hypotheses at the appropriate levels of abstraction
• ACH cannot represent subordinate argumentation, i.e. the argumentation bearing up on a piece of evidence.
• ACH activities at realistic scales leave analysts disoriented or confused.

Van Gelder proposed hypothesis mapping (similar to argument mapping) as an alternative to ACH.

Structured analysis of competing hypotheses

The structured analysis of competing hypotheses offers analysts an improvement over the limitations of the original ACH. The SACH maximizes the possible hypotheses by allowing the analyst to split one hypothesis into two complex ones. For example, two tested hypotheses could be that Iraq has WMD or Iraq does not have WMD. If the evidence showed that it is more likely there are WMDs in Iraq then two new hypotheses could be formulated: WMD are in Baghdad or WMD are in Mosul. Or perhaps, the analyst may need to know what type of WMD Iraq has; the new hypotheses could be that Iraq has biological WMD, Iraq has chemical WMD and Iraq has nuclear WMD. By giving the ACH structure, the analyst is able to give a nuanced estimate.

Other approaches to formalism

One method, by Valtorta and colleagues uses probabilistic methods, adds Bayesian analysis to ACH. A generalization of this concept to a distributed community of analysts lead to the development of CACHE (the Collaborative ACH Environment), which introduced the concept of a Bayes (or Bayesian) community. The work by Akram and Wang applies paradigms from graph theory.

Other work focuses less on probabilistic methods and more on cognitive and visualization extensions to ACH, as discussed by Madsen and Hicks. DECIDE, discussed under automation is visualization-oriented. Work by Pope and Jøsang uses subjective logic, a formal mathematical methodology that explicitly deals with uncertainty. This methodology forms the basis of the Sheba technology that is used in Veriluma's intelligence assessment software.

A few online and downloadable software tools help automate the ACH process. These programs leave a visual trail of evidence and allow the analyst to weigh evidence.

• PARC ACH 2.0 was developed by Palo Alto Research Center (PARC) in collaboration with Richards J. Heuer, Jr. It is a standard ACH program that allows analysts to enter evidence and rate its credibility and relevance.
• Decision Command software was developed by Willard Zangwill.
• DECIDE was developed by the analytic research firm SSS Research, Inc. DECIDE not only allows analysts to manipulate ACH, but it provides multiple visualization products.
• Analysis of Competing Hypotheses (ACH) is an open-source ACH implementation.
• ACH Template is an Excel sheet that implements the scoring and weighting methodology of ACH, more specifically the weighted inconsistency counting algorithm.

• Integrations
• Case Studies
• State of Competitive Intelligence
• Partner Directory
• In the News
• Crayon Academy

Crayon Competitive Intelligence blog

Your competitor is up to something—but what use analysis of competing hypotheses (ach) to crack the case.

Imagine you lead the intelligence function at a CRM software company. One day, you notice that your closest competitor has recently published a couple blog posts about contracts and document management—which is out of the ordinary for them.

The first explanation that comes to your mind is, “They’re building an esignature solution.” And that could absolutely be the case! That’s a perfectly plausible explanation.

But it’s not the only explanation, is it?

Given what you know at this point, it’s equally valid to conclude that your competitor is not building a solution, but rather acquiring one. Or maybe they’re getting ready to announce an integration or partnership. Or maybe they’re simply trying to grow their website traffic by ranking for a new group of keywords.

So—you’ve got four different explanations of what’s going on. Each of them is valid. And each of them demands a different proactive response from your company. And the clock is ticking.

Where do you go from here? How do you determine which explanation is most likely to be the truth? One option is to use a technique called analysis of competing hypotheses .

How to use analysis of competing hypotheses (ACH) to predict & prepare for your competitor’s next big move

At the core of ACH is a simple matrix like the one you see below, where each column is one of your hypotheses and each row is a relevant observation or piece of intelligence.

For the sake of the example, let’s assume that after you came up with the four hypotheses—more on this process later—you went out and found two more relevant pieces of intel in addition to the blog content: (1) six months ago, your competitor raised $200 million in Series E funding; (2) not long after the announcement of their Series E round, they hired a strategist with experience leading build vs. buy initiatives.

To use the matrix, start with the first piece of intel. Ask yourself: If we knew for a fact that our competitor was building an esignature solution, would we expect them to publish content about document management? The answer is yes, so we’ll put a plus sign in the corresponding cell:

Next: If we knew for a fact that our competitor was acquiring an esignature solution, would we expect them to publish content about document management? Yes, we would. Add a plus sign:

How about if they were creating an integration or partnership? Yes—in that case, we’d expect them to publish this content. And the same would be true if they were simply trying to grow their website traffic. Add two more plus signs:

So it turns out that your first observation, the blog content, supports all four hypotheses. I know we already knew this—that this observation supports all four hypotheses is precisely why you’re doing this exercise in the first place—but I wanted to make sure you saw the visual of a single row with nothing but plus signs.

When you use ACH for real and see a single row with nothing but plus signs, that tells you that this particular piece of intel is not going to get you any closer to the truth—so it’s time to move on to the next observation.

Ask yourself: If we knew for a fact that our competitor was building an esignature solution, would we expect them to use financial capital? Yes—you need money to do R&D, hire engineers, and so on. Add a plus sign:

Obviously, if you knew for a fact that your competitor was making an acquisition, you would expect them to use financial capital. Add another plus sign:

Here’s where things get interesting. If you knew for a fact that your competitor was creating an integration or partnership, would you expect them to use financial capital? Not necessarily. An initiative like that certainly requires time and energy, but not the kind of money you need to build a product from scratch or acquire a company. Therefore, the $200M Series E is not applicable to the integration/partnership hypothesis:

Using the same logic, it’s not applicable to the website traffic hypothesis either:

One more observation to assess! If your competitor were building something that they could just as easily buy, would you expect them to hire someone with experience leading build vs. buy initiatives? Absolutely. And the same would be true if they were buying something that they could just as easily build . Add two plus signs:

Because the hiring of this strategist seems to outright undermine our third and fourth hypotheses, we’re going to fill in the last two cells with minus signs:

Well done! Using ACH, you’ve narrowed it down to the two most plausible hypotheses: Given what you know at this point, it seems likely that your competitor is either building or acquiring an esignature solution.

Creating early warning systems for your remaining hypotheses

From here, your next step is to ask yourself a slightly different version of the question you’ve been asking yourself this whole time: If it’s true that your competitor is building a solution, what else can you expect to observe going forward? Alternatively, if it’s true that your competitor is acquiring a solution, what else can you expect to observe going forward?

I suggest answering these questions using a T diagram:

This diagram, simple though it may be, is a very powerful tool known as an early warning system . If one or more of the items listed on the left-hand side actually occurs, you can safely assume that your first hypothesis is correct: Your competitor is building a solution. If one or more of the items on the right-hand side actually occurs, go with your second hypothesis.

Your next step: Take your ACH matrix and T diagram to your senior stakeholders in sales, marketing, product, and customer success and bring them up to speed. Show them how you narrowed it down to the two most likely scenarios and tell them what you’re looking out for moving forward. Then, start working with them on contingency plans.

Until you catch wind of a silver bullet piece of intel—like a beta offering (hypothesis 1) or a change manager job req (hypothesis 2)—you’ll need to prepare your internal teams for both the build scenario and the acquisition scenario. Worst case, you learn the truth along with everyone else in the world when the big event actually happens, but as far as worse case scenarios go, that’s a pretty good one—because no matter what, you’ve already done the prep work.

So that, in a nutshell, is analysis of competing hypotheses—a simple yet powerful way to:

• Keep yourself from jumping to the first conclusion that comes to mind,
• Identify gaps in your intelligence and create an early warning system, and
• Ensure that when you go to your stakeholders with a prediction (or two competing predictions), you’re able to show your thought process and defend your work.

Now, there’s one last question we need to address.

What if you hadn’t considered the acquisition hypothesis?

For the sake of our ACH example, we assumed that you had thought of every plausible explanation—but what if you hadn’t considered the acquisition hypothesis? In that case, the ACH matrix would’ve led you to the conclusion that your competitor was building a solution.

And what if that had been the wrong conclusion? You would’ve worked with your senior stakeholders to prepare for a product launch, only to get blindsided by an acquisition.

It’s true: For ACH to work, you need to consider every explanation for what you’re observing. The best way to do that is to loop in your most trusted colleagues as soon as you realize that your competitor is up to something . Brainstorm hypotheses with them until you’re confident that you’ve collectively thought of everything. From there, you should have what you need to start looking for relevant pieces of intel and filling out the left-hand part of your matrix.

To recap, here’s the step-by-step process you should follow as soon as you realize that your competitor is up to something:

• Brainstorm hypotheses with your most trusted colleagues.
• With your hypotheses in mind, search for relevant pieces of information. Focus your search by asking, “If X were true, what would I expect this competitor to do?”
• Fill out your matrix using the process outlined above.
• If Step 3 does not lead you to a single plausible hypothesis, look at each of your remaining hypotheses and ask yourself, “If this is true, what else can we expect to observe?” Create an early warning diagram.
• Use your matrix and your early warning diagram to bring your senior stakeholders up to speed. Work with them on contingency plans.
• Keep your eye out for the silver bullets outlined in your T diagram. And remember: Even if you don’t see any of them, you’ve already put your company in an excellent position.

Happy analyzing!

Seeing is believing! Check out Crayon for yourself.

Related Blog Posts

Popular Posts

• Architecture and Design
• Asian and Pacific Studies
• Business and Economics
• Classical and Ancient Near Eastern Studies
• Computer Sciences
• Cultural Studies
• Engineering
• General Interest
• Geosciences
• Industrial Chemistry
• Islamic and Middle Eastern Studies
• Jewish Studies
• Library and Information Science, Book Studies
• Life Sciences
• Linguistics and Semiotics
• Literary Studies
• Materials Sciences
• Mathematics
• Social Sciences
• Sports and Recreation
• Theology and Religion
• Publish your article
• The role of authors
• Promoting your article
• Abstracting & indexing
• Publishing Ethics
• Why publish with De Gruyter
• How to publish with De Gruyter
• Our book series
• Our subject areas
• Your digital product at De Gruyter
• Contribute to our reference works
• Product information
• Tools & resources
• Product Information
• Promotional Materials
• Orders and Inquiries
• FAQ for Library Suppliers and Book Sellers
• Repository Policy
• Free access policy
• Open Access agreements
• Database portals
• For Authors
• Customer service
• People + Culture
• Journal Management
• How to join us
• Working at De Gruyter
• Mission & Vision
• De Gruyter Foundation
• De Gruyter Ebound
• Our Responsibility
• Partner publishers

Your purchase has been completed. Your documents are now available to view.

Cognitive Bias in Intelligence Analysis

Testing the analysis of competing hypotheses method.

• Martha Whitesmith
• X / Twitter

Please login or register with De Gruyter to order this product.

• Language: English
• Publisher: Edinburgh University Press
• Copyright year: 2020
• Audience: College/higher education;
• Main content: 304
• Keywords: Politics
• Published: March 24, 2022
• ISBN: 9781474466363

• About EclecticIQ

Get the most out of your EclecticIQ cybersecurity solutions.

Unlock the full potential of our Threat Intelligence Platform with expert guidance.

Explore our world-class partners – or learn about our partner program.

TIP for CTI

Power your CTI practice with analyst-centric threat intelligence solutions.

TIP for SOC

Go beyond the IOC to augment your SOC in defense of your organization.

Intelligence Center

Threat scout, curated feeds.

Learn how EclecticIQ can help you address your specific challenges – by team and by need – and improve your overall security posture.

For CTI Teams

Provide your CTI team with the automation, performance, flexibility, and integrations needed to supercharge their CTI operations with our range of analyst-centric products and services.

For SOC Teams

Enable your SOC team to better operationalize threat intelligence for more effective and efficient incident response with our range of analyst-centric management products and services.

For Situational Awareness

Improve your situational awareness and mitigate risk with our collection of analyst-centric threat intelligence products and services.

For Collaboration & Dissemination

Operationalize threat intelligence for more effective and efficient incident response with our range of analyst-centric management products and services.

Our Ecosystem

An ecosystem supporting our customers' intelligence-led proactive cybersecurity needs with collaborative partner programs delivering world-class joint solutions. 

Partner Program

Partner with EclecticIQ to bring valuable and innovative security solutions and services to end users. Open to all partner types, including technology developers, service providers, resellers, and community.

Our Partnerships

We partner with the world's premier technology and solution providers to support all phases of your cybersecurity needs. Explore all our partners' solutions and offerings to build and extend your cyber defense ecosystem.

EclecticIQ Resources

We are committed to increasing the knowledge and capabilities of the cybersecurity community through our research & analysis efforts and open source projects.

Browse Resources

Learn more about our technology, solutions and services, and stay updated on the cyber threat landscape with our research reports, webinars and other information.

Open Source Projects

We are proud to be an active member in the open source community and to help develop and advance progress of security technology. Learn more about contributions or go directly to our GitHub page.

How to Structure Analysis of Competing Hypotheses (ACH)

Welcome to the first post in a three-part series of blogs which will address:

• How to structure Analysis of Competing Hypotheses
• Moving past STIX 2.1 Opinion Object
• Introducing the Hypothesis Object

In this first blog post we will look at what Analysis of Competing Hypotheses (ACH) means and how it can support security analysts.

An introduction to ACH

A good security analyst uses a portfolio of known structured techniques, methods and skills that support them in their job. This allows them to work with and make sense of large amounts of data.

One of these is techniques is Analysis of Competing Hypotheses (ACH). Put simply, ACH is a method analysts can use to evaluate hypotheses against a given range of evidence. Intelligence vendors, producers and consumers use ACH to evaluate a threat based on the available evidence.

During an investigation, analysts may need to identify inconsistencies across a set of hypotheses. Using ACH improves an analyst’s ability to assess and validate an issue with an assertion that has been tested for confidence.

How ACH is Currently Applied

Cybersecurity intelligence provider Digital Shadows gave a real-world example of ACH in action in May 2018, when it modeled and published a report about multiple competing hypotheses surrounding the WannaCry ransomware incident. The malware impacted enterprise networks and organizations across the globe. Hours after WannaCry occurred, many public and private companies within the intelligence community were attempting to identify and attribute the attack, these analysts hoped for a greater understanding of how and why the attacks took place.

Digital Shadows outlined four possible hypotheses about potential attribution and tested them against the set of evidence that became available during and after the incident occurred:

H1 - A sophisticated financially-motivated cybercriminal actor

H2 - An unsophisticated financially-motivated cybercriminal actor

H3 - A nation state or state-affiliated actor conducting a disruptive operation

H4 - A nation state or state-affiliated actor aiming to discredit the National Security Agency (NSA)

The evidence in this case are data points that the community learned or observed about the incident:

• Use of Eternal Blue Equation Group exploit
• Targeted globally-diverse victims
• Installed DOUBLEPULSAR backdoor
• Code similarities to North Korean malware
• No evidence of phishing vector
• Kill switch as anti-analysis feature
• Only three Bitcoin wallets produced

Taking Digital Shadows’ plausible scenarios about WannaCry origin, there are limitations in STIX (Structured Threat Information Expression) that make it difficult to structure the process of conducting and structuring ACH.

In the image above, the groupings show the evidence and the various hypotheses, but one can see that there is no way in STIX for a producer/consumer/collaborator to structure and convey the results of having tested multiple hypotheses at once. In its current form, STIX allows us to see a confirmed reality (for example: Threat Actors à Campaign; Indicators à TTPs; Incident à Targeted Victims). There is currently no entity that will represent an alternative view that would let consumers see competing hypotheses in a structured way.

At the time of Digital Shadows’ analysis, it was identified that H2 – an unsophisticated financially-motivated cybercriminal actor – was the strongest-scoring hypothesis from the evidence that was available. After using the evidence to test a set of hypotheses, there is still no way to structure which hypothesis scored strongest.

Identifying the Problem and Wrapping Things Up

While ACH brings many benefits, the main problem with using the approach is that there is currently no way to structure the process of testing evidence against a set of hypotheses (H1, H2, H3, etc.). As a result, producers of intelligence often create multiple competing hypotheses around a given threat hoping to identify the strongest hypothesis, i.e. the one most supported by the available evidence. If the process of testing evidence can be structured more effectively, the benefits of ACH will become even more apparent.

Part two of this series will address STIX in more detail and some of the limitations in structuring ACH. Make sure to check our blog for the rest of this series. 

We hope you enjoyed this post. Follow us here for more interesting reads on Cyber Threat Intelligence or check out our resource section for whitepapers, threat analysis reports and more.

Receive all our latest updates

Explore all topics.

• Threats and Vulnerabilities
• Intelligence Research
• Threat Intelligence
• Cybercriminal
• Threat Actor APT
• Threat Intelligence Platform
• STIX and TAXII
• EclecticIQ Intelligence Center
• Critical Infrastructure
• Nation-State