You are here: silicon.com > Networks > WebWatch

WebWatch

Dell gets Googled

Googlebot unearths confidential info for all to see...

Tags: googlebot, dell, google

By Elinor Mills

Published: 2 February 2006 08:40 GMT

Dell apparently learnt the hard way this week that companies have to be careful to ensure information they store on the internet and want to keep hidden is not automatically added to a search engine index for everyone on the web to see.

Specifications for future Dell laptops were accessible via Google's search site before the content was pulled from a Dell file transfer protocol site and from Google's cache.

Google, like the other major search engines, has an automated search engine that sends software robots called "spiders" out to crawl the web and find sites to add to the index of websites it maintains. Because the spiders follow links running from one website to others, they pick up sites on their own without webmasters having to manually submit them to search engines.

Webmasters can also provide the URL, or numerical web address, for pages they want crawled, and they can submit detailed site maps to Google, according to Google's "information for webmasters" pages.

Webmasters who want to keep some or all of their site private from the Googlebot can put a standard document called "robot.txt" at the root of the server that instructs the crawler not to download content. If the removal request is urgent, the webmaster can submit a request via Google's automatic URL removal system but must provide an email address and password first.

Content that has been removed can still be viewed through Google's cache, which is a "snapshot" and archive of each page crawled. Webmasters can prevent pages from being cached by inserting specific code on them.

Elinor Mills writes for CNET News.com

  1. Zones
  2. Management
  3. Networks
  4. Software
  5. IT Services
  6. Hardware
  1. Verticals
  2. Public Sector
  3. Financial Services
  4. Retail & Leisure
Read and write about internet access at the airports of the world at atlarge.com. Rate airports, and see what others have to say...

Peter Cochrane Peter Cochrane's Blog: How the telcos could save themselves Doomed network operators could thrive with a bit of innovation

Peter Cochrane Peter Cochrane's Blog: Facebook saves teen from prison Another unexpected impact of social networking


  • Jobs
Linking Strategist, Cornwall; 16K DOE + Benefits

You will have a good understanding of search engines and the workings of Google, Yahoo! An understanding of search engines and the workings of ...

Search Engine Optimization Specialist

Must be fluent in a second language, An in depth understanding of search engines, search engine optimisation and factors that influence high rankings ...

SEO Coordinator

You will have a good understanding of search engines and the workings of Google, Yahoo! A good understanding of search engines and excellent internet ...

Agenda Setters 2009
Welcome to the ninth annual Agenda Setters poll – silicon.com's list of the top 50 most influential individuals in the technology and IT industries, from techies and CIOs to entrepreneurs and business leaders. Find out more in our latest special report.





Quick Sitemap Links: