Words of Estimative Probability, Analytic Confidences, and Structured Analytic Techniques

In line with Intelligence Community Directives 203 and 206, MS-ISAC Cyber Threat Intelligence (CTI) products leverage specific verbiage to express uncertainties associated with analytical assessments. These products account for such uncertainties, particularly related to forecasts and predictions, using words of estimative probability (WEPs) and analytic confidences.

1. The chart below outlines the WEPs that MS-ISAC CTI commonly uses in member engagements to communicate likelihoods and probabilities. Note: The words “High” and “Break” are often exchanged for “Very” and “Roughly,” respectively.



Almost no chance	Very unlikely	Unlikely	Roughly even chance	Likely	Very likely	Almost certain(ly)
Remote	Highly improbably	Improbable (improbably)	Roughly even odds	Probable (probably)	Highly probable	Nearly certain
1-5%	5-20%	20-45%	45-55%	55-80%	80-95%	95-99%



2. Disseminated products with a formal assessment also explicitly state analytic confidence based on the number, variety, and reliability of sources. The following confidence levels articulate the MS-ISAC CTI team’s assessment of the quality and quantity of source information supporting judgments within a given product:
1. High Confidence generally indicates that CTI’s judgments are based on high-quality information from multiple sources, most or all of which are considered trustworthy, with minimal to no conflict among sources. High confidence in source information does not imply that the assessment is a fact or a certainty; there is always a chance that an assessment might be wrong.
2. Moderate Confidence generally means that the information is credibly sourced and interpreted to be plausible but not of sufficient quality or corroborated sufficiently to warrant a higher level of confidence. For example, multiple sources may have opposing or alternative views, and while the CTI team or partners may have evidence to support one assessment over an alternative, it may not be sufficient enough to claim high confidence
3. Low Confidence generally means that the source information’s credibility or plausibility is uncertain. Here, source information is scant, questionable, fragmented, or poorly corroborated to the point where it is difficult to make solid analytic inferences. Low confidence could also indicate that the CTI team has concerns with the reliability of the source data.

In addition to WEPs and analytic confidences, the MS-ISAC CTI team regularly leverages Structured Analytic Techniques (SATs) to enrich, guide, and formalize U.S. State, Local, Tribal, and Territorial (SLTT) focused threat assessments. SATs are methodical analytic procedures that analysts can apply to hone the accuracy and relevancy of assessments. The SATs are generally categorized under three umbrella techniques: Diagnostic, Contrarian, and Imaginative.

• Diagnostic
• Key Assumptions Check
• Quality of Information Check
• Indicators or Signposts of Change
• Analysis of Competing Hypothesis (ACH)

• Contrarian
  • Devil’s Advocacy
  • Team A/Team B
  • High-Impact/Low-Probability Analysis
  • “What If?” Analysis

• Imaginative
• Brainstorming
• Outside-In Thinking
• Red Team Analysis
• Alternative Futures Analysis

All three leveraged analysis components (WEPs, Analytic Confidences, and SATs) can be found within the body of MS-ISAC CTI team products or summarized near the end of the product under the “Analytic Confidence” header. The MS-ISAC CTI team can be reached at [email protected].

Resources

ICD 203

ICD 206

SATs