���㽶��Ƶ

National security officials are “overwhelmingly overconfident,” which hinders their ability to accurately assess uncertainty, according to new research by , an associate professor of government.

When they thought statements had a 90% chance of being true, the statements were only true about 60% of the time, according to the study.

The findings are published in the.

About 1,900 national security officials from more than 40 NATO allies and partners were surveyed on the uncertainty of current and future states of the world, and delivered a total of 60,000 assessments. The officials were enrolled at the U.S. National War College, the Canadian Forces College, the NATO Defense College, and the Norwegian Defense Intelligence School.

In the U.S., Canada, and Europe, once military officers achieve the rank of colonel, they must obtain a master’s degree at a war college or other military institution as part of their professional military education. Participating institutions also contained a large fraction of civilian national security officials who work in foreign affairs ministries and intelligence agencies, among other areas. The study thus spanned both an unusually large, and representative, group of high-ranking national security officials.

The survey contained questions on international military, political, and economic affairs and asked respondents to estimate the chances that a statement was true, such as, “In your opinion, what are the chances that NATO’s members spend more money on defense than the rest of the world combined?”

Other questions involved making predictions, such as, “In your opinion, what are the chances that Ukraine and Russia will officially declare a ceasefire by a certain date?”

The results showed that national security officials are overconfident about the current and future state of the world—a cognitive bias that was consistent across all respondents, civilian and military professionals, men and women, and U.S. and non-U.S. citizens. They share these biases with the general public.

“National security officials are like many of us, in the sense that we tend to think we know more than we really do. This means that national security officials, like members of the general public, are consistently overconfident,” says Friedman, a faculty coordinator of the Rosenwald Postdoctoral Fellows Program at the. 

He says that overconfidence among national security officials is similar to biases he has found when conducting similar surveys with undergraduates, masters students, and financial professionals.

“However, the study also showed that it is possible to mitigate that bias substantially with just two minutes of training,” says Friedman. His research found that briefly showing national security officials data on patterns of overconfidence led study participants to make judgments that substantially reduced overconfidence—and promoted accuracy.

Image

Jeffrey Friedman is an associate professor of government at ���㽶��Ƶ. (Photo by Kata Sasvari)

The study also found that national security officials have a bias towards false positives— a tendency to think false statements are true.

This was demonstrated by flipping the wording of survey questions. In a subset of surveys, half of the participants were asked: “What are the chances that ISIS has killed more civilians than Boko Haram since 2010?” while the other half were asked the inverse of that question: “What are the chances that Boko Haram has killed more civilians than ISIS since 2010?” 

The answers that national security officials gave to these two questions consistently added up to more than 100%, illustrating that they were skewed towards false positives as the average estimates should have added up to 100%.

Friedman says that this finding indicates that national security officials appear to have a “tendency towards confirming rather than refuting possibilities they are asked to consider,” which could be especially problematic for national security officials given that there could potentially be multiple outcomes to consider in military scenarios, rather than just one.

The study did suggest one potential remedy for the overconfidence—reminding national security officials of the perils of being too sure of their convictions.

Before a random subset of the national security officials were given the survey, they received information about other cohorts’ overconfidence and biases. Through this two-minute training and informed approach, those participants were significantly better at assessing uncertainty.

Friedman says the four military institutions that took part in the study deserve credit for their participation. The work had a ripple effect: the first cohort was from the National War College, which was so pleased with the session that they invited Friedman back, and then other military institutions came on board following recommendations by past participants. 

“It was very rewarding to see how receptive the national security officials were to the insights and training, as the training was then built into the core curricula at the institutions,” says Friedman.

“Any organization that cares about improving people’s ability to assess uncertainty in a more accurate manner can implement this training,” says Friedman, who notes that the material is posted online and can be developed and integrated into curricula anywhere, including beyond the national security realm.

“By harnessing decision science tools, we can make people’s judgments better,” says Friedman.