By Gemma Simpson
Published: Monday 25 September 2006
Email story to a Friend | Report Abuse
Name
Anonymous
Location
Austin
Occupation
Programmer
Comment
It's a bit silly that currently available methods, namely robots.txt which is honored by Google spiders, are not used by the news providers. Of course using readily available solution is not as fun as inventing one's own.
It's a bit silly that currently available methods,...
Anonymous
Do these guys know about robots.txt? Evidently not...
Anonymous
so putting sensitive content in specific folders a...
Karen Challinor
Reinventing the wheel? Just ask any experienced w...
Joe Whitehead
Agenda Setters 2009
Welcome to the ninth annual Agenda Setters poll – silicon.com's list of the top 50 most influential individuals in the technology and IT industries, from techies and CIOs to entrepreneurs and business leaders. Find out more in our latest special report.
Copyright © 2008 CBS Interactive Limited. All rights reserved. Top of page
RSS Feeds
The Round-Up The Weekly Round-Up: 03.12.09 'Ere guv, you'll never guess who I had in the back of my cab the other day…'
Stuart Roberts Shared services - how to get it right in your business Recession boosts uptake