New mathematical logic to help intelligence services cope with uncertainty

Certainty
Let us assume that an intelligence agent estimates the probability of two different events to be one half. Even though the probability is the same, their certainty and uncertainty may vary. If you flip a coin, the probability of heads is one half. There is a very high certainty that this probability estimate is correct. Unless the coin is biased or manipulated, this certainty is one hundred per cent.

The probability that Oswald shot Kennedy in 1962 may also be estimated to one half, but this probability is fraught with a large margin of uncertainty. Even though the probability is set at one half here as well, there is a large uncertainty that the probability estimate might be totally wrong.

Quantifying uncertainty
Intelligence people struggle precisely with such things. They need to take uncertainty into account, but there are no tools available for undertaking intelligence analyses that consider the degree of uncertainty in each probability estimate. With the aid of subjective logic, the certainty and uncertainty of all probability estimates can be quantified.

“Unless this is done, the uncertainty is swept under the carpet. We humans are stuck in our preconceived notions and follow the beaten track. We are unable to see things objectively. If the Americans had used this new mathematical logic, they would have seen that the uncertainty as to whether Saddam had access to weapons of mass destruction or not was too large,” Jøsang points out.

To express the uncertainty of their probability estimates, statisticians use a tool called the confidence interval. A confidence interval describes the likelihood that an event will occur with a probability within a certain interval. However, there are no tools that can simply handle confidence intervals as input arguments in complex models. As a rule, confidence intervals are used only for presentation of results.

Today, all input arguments in intelligence models must be entered with a specific probability, even if this figure is uncertain.

“What they really ought to say is ‘we don’t know this.’ However, such input arguments are not permitted in traditional analytical tools. With subjective logic, the input arguments may be completely uncertain and estimates can be made with these probabilities, even though they are fraught with uncertainty.”

Mathematical repertoire
The certainty and uncertainty of each probability can be represented by triangles, where the horizontal line describes the magnitude of the probability and the vertical height represents the uncertainty of the probability estimate. In order to describe situations, Jøsang is now working to expand the repertoire of mathematical operators, such as deduction and abduction, and how these mathematical operators can be combined.

By emphasizing the uncertainty of the input arguments, decision-makers can visualize the degree of uncertainty of their analyses in order to make better decisions. If the result is fraught with large uncertainty, one will be reluctant to make significant, important decisions, but perhaps call for more intelligence work instead.

“When we implement this logic, we can see the aspects of the theory that are incomplete and need to be straightened out. If the Americans had been in possession of this tool, perhaps they would have found that there was too much uncertainty with regard to the hypothesis saying that Saddam had weapons of mass destruction before taking such a momentous decision as to invade Iraq. Unless the uncertainty of the circumstantial evidence is taken into account, the analysis tool may erroneously conclude that there was a clear probability that Saddam had access to weapons of mass destruction.”

The gist of the matter is:

“In order to balance all circumstantial evidence it is crucial to describe the uncertainty of each individual piece of information. If this is not done, there is a risk that evidence with a large degree of uncertainty is compared to evidence that is basically certain. One should be cautious in taking large, momentous decisions if the results are fraught with a large degree of uncertainty.”

Jøsang emphasizes that his theory is sufficiently developed to be adapted to analytical tools.

Intelligence services: Applicable
The release notes that the Norwegian military intelligence service has expressed an interest in this new, mathematical logic, but the intelligence officer who had familiarized himself with the theory did not wish to make a statement to the UiO’s press office. The intelligence officer referred the office to Dean Tore Pedersen who is head of intelligence studies at the Norwegian Intelligence College. He made it clear to the office that his statement was of a general, academic nature:

“Subjective logic can be used to explore complex problems that include a large element of subjectivity and uncertainty,” Pedersen said in response to a question.

The U.S. Army is interested
American intelligence services are currently using an analytical framework called ACH (for Analysis of Competing Hypotheses). The U.S. Army Research Lab, the American equivalent of the Norwegian Defense Research Establishment at Kjeller, supports this research at UiO with NOK 2 million ($265,000) to explore how subjective logic can be implemented in their intelligence analyses.

“By incorporating uncertainty, subjective logic has the potential to revolutionize automated probability reasoning and improve intelligence operations. The method may enable the decision-maker to realize when the responses are too uncertain and that more information needs to be collected. We still need answers to a number of fundamental questions. The U.S. Army Research Lab is therefore collaborating with Professor Jøsang through the project ‘Advanced Belief Reasoning in Intelligence’ to determine whether and how his idea can be realized,’ says Dr. Lance Kaplan in the Networked Sensing & Fusion Branch, U.S. Army Research Laboratory.