By Munir Kotadia
Published: Friday 14 January 2005
Email story to a Friend | Report Abuse
Name
Orv
Location
Michigan
Occupation
System Administrator
Comment
robots.txt should be used with caution, and never as the sole way of protecting private information. "Polite" search engines like Google will honor it, but others may not. More importantly, anyone can read the robots.txt file, so it can become a map to all the stuff you don't want anyone to see!
Agenda Setters 2009
Welcome to the ninth annual Agenda Setters poll – silicon.com's list of the top 50 most influential individuals in the technology and IT industries, from techies and CIOs to entrepreneurs and business leaders. Find out more in our latest special report.
Copyright © 2008 CBS Interactive Limited. All rights reserved. Top of page
RSS Feeds
The Round-Up The Weekly Round-Up: 03.12.09 'Ere guv, you'll never guess who I had in the back of my cab the other day…'
Stuart Roberts Shared services - how to get it right in your business Recession boosts uptake