Internet blacklisting tool unveiled

Security researchers have taken a page out of Google's book in reinventing the blacklist, a tool for blocking internet attacks.

At this week's 17th Usenix Security Symposium, researchers from the Sans Institute and SRI International will present the results of their experiments with 'highly predictive blacklisting' (HPB), a service that tailors blacklists for particular networks using an approach similar to Google's PageRank. PageRank is Google's technique for making search results more relevant.

Have your say on the state of tech skills in the UK. Take the silicon.com Skills Survey 2008

Take the survey now…

The researchers have been investigating HPB since early last year, via an experimental service offered to contributors to DShield.

DShield is a community-based system that collaborates firewall logs from contributors in order to analyse attack trends, and is used as the data-collection system behind the Sans Institute's Internet Storm Center.

DShield and similar sites offer firewall filters enabling administrators to block a list of the internet's worst attackers, known as a 'global worst-offenders list' (GWOL), but this may contain many attacks that the network will simply never encounter, researchers said.

Local networks also create their own local worst-offender lists (LWOLs) but these aren't capable of dealing with attackers that are encountered by that network for the first time.

HPB is designed to be a middle ground between the two. It is based on DShield researchers' finding that groups of networks share various degrees of common attacker overlap: what the researchers called "correlated victims".

By taking this overlap into account, the researchers said they can create blacklists personalised for an individual network that can accurately estimate the probability that a source will attack that network within the next few days.

"In formulating HPB for a network 'A', we treat attack sources that have reportedly made attacks on networks correlated with 'A' differently from attack sources that attacked the same number but uncorrelated networks," researchers said in a document on the website of SRI International's Cyber-Threat Analytics project, which is co-ordinating the HPB research.

The researchers said: "Traditional blacklisting approaches, such as GWOL, treat these two attackers equally, therefore, ignore the characteristics of individual networks shown in the alert history." The project's contributors are SRI's Phillip Porras and Jian Zhang and the Sans Institute's Johannes Ullrich. The algorithm developed by the project appears to significantly improve blacklist accuracy, the researchers said.

"Our experiments show that the HPB exhibits a higher hit count than traditional blacklists for most of the contributors," they noted. "The experiments also show that HPB's performance is consistent over time, and these advantages remain stable across various list lengths and predict windows."

Original article: Researchers redefine the internet blacklist from ZDNet UK