British Library to archive 'uncensored' web

A trial project to archive 6,000 UK websites was announced on Tuesday by the UK Web Archiving Consortium. The consortium, led by the British Library, includes the Wellcome Trust, the National Archives and the Scottish and Welsh national libraries.

Each member of the consortium will choose content relevant to its subject. All types of web content will be included, from government documents to blogs.

Richard Boulderstone, director of e-strategy at the British Library, said that all types of material will be collected including "informal material" such as discussion forums. "Letters and other informal works tell us how society is actually operating," he said.

The British Library will not censor the material because it does not want to restrict what people can find out about in the future.

"We would like to take a snapshot of every year, as a sample of what the web looked like", said Boulderstone, suggesting that in the future people could look back to 2004 and see the swear words that web users were using.

Only a limited number of websites will be archived initially but "ultimately, we would like to archive the whole UK web," said Boulderstone.

One of the problems faced by the consortium is that, due to UK copyright law, permission is needed before a site can be archived. The British Library is working with the government to extend the law to allow them blanket access to all websites because "there are four million sites that we would like to capture - we cannot ask everyone for permission," said Boulderstone.

The UK Web Archiving Consortium is not the first to archive the web. The Wayback Machine, run by US-based Internet Archive, is a service that allows people to visit archived versions of websites.

According to Boulderstone, the British Library's approach differs from that of the Internet Archive because his organisation seeks permission from websites. In the future, the British Library hopes to improve on Wayback by archiving more frequently and with more depth, and through providing metadata so that information can be found more easily.