CybersecurityCybercrime statistics wildly inaccurate, says researcher
A cybersecurity researcher is questioning the various statistics that government officials and IT companies use as evidence of the rampant and deleterious effects of hackers; Cormac Herley, a principal researcher at Microsoft Research, argues that the existing data on the estimated losses from cyberattacks is wildly inaccurate to the point that analysts have no idea what the problem’s economic impacts are; one expert, noting that estimates of the annual cost of cybercrime range from $560 million to $100 billion to $1 trillion, asks: “How can this be? How can you have estimates of the same problem ranging across three orders of magnitude?”
A cybersecurity researcher is questioning the various statistics that U.S. government officials and IT companies use as evidence of the rampant and deleterious effects of hackers.
Cormac Herley, a principal researcher at Microsoft Research, argues that the existing data on the estimated losses from cyberattacks is wildly inaccurate to the point that analysts have no idea what the problem’s economic impacts are.
As evidence he cites the fact that in three instances, corporate executives presented numbers that dramatically differed.
“How can this be?” Herley asked. “How can you have estimates of the same problem ranging across three orders of magnitude?”
He added these numbers “just didn’t make sense.”
In addition to their wide range, some of them are illogically high. For example, he says the $1 trillion figure would mean that every adult in the United States that was online lost $5,000.
Herley says that the lack of accurate data has serious consequences.
“Without numbers, we can’t make good policy or sound investment decisions,” he said.“Not only that, but we can’t figure out where key threats are coming from. Are the criminals making most of their money from key logging? Highly targeted phishing attacks? Brute-force attacks on people’s passwords?”
“It’s distressing,” he concluded.
In investigating the matter further, he found that the methods used to calculate these estimates were insufficient to yield accurate results.
The majority of the data was based on surveys which asked respondents to report if they had been the victims of cybercrime and how much they lost.
Unlike other polls, cybercrime surveys seek to measure specifically how much money was lost which makes an individual’s response much more important and varying numbers could alter the survey’s results. For instance if respondent claims to have lost $500,000 when in fact they only lost $50,000 that could dramatically change the survey.
In addition, a survey’s relatively small numbers has a tendency to skew the significance of data. In a 2006 survey conducted by Gartner Research, 128 out of 4,000 people claimed to have been victims. Based on Herley’s calculation 59 percent of losses came from the top 1 percent of respondents who had been victimized which was just one person.
Julie Ryan, a professor in information security management at George Washington University, shares Herley’s concerns.
“Understanding the impact of any crime is problematic,” but cybercrime is particularly problematic as most people don’t know enough about the technical aspects behind the attack she said.
Survey respondents rarely know how to distinguish if they were thevictims of a phishing attack or if anything was actually stolen or corrupted.
“So here we have a problem,” Ryan said. “Potential crime that is potentially undetectable, compounded by a target space that is mostly ignorant.”
Furthermore, Ryan said there is an information bias as most big corporations are reluctant to share their cyberattack history for fear of losing customers, while cyber security firms benefit from these details.
To provide more accurate statistics, Herley is pushing the companies and organizations that conduct surveys to be more transparent about their survey methods so that researchers can evaluate their accuracy.
In particular, Herley hopes that organizations publish median figures instead of averages as they are less likely to be affected by outliers and are therefore less susceptible to exaggeration.
