How to make Google sit up and take notice

The science of web search is a constantly moving feast. In his new column Net Effect, Stewart Baines outlines the basics.

Ever wonder how to rise above the noise?

Finding something on the internet - something you don't know much about, rather than a 'who is my local plumber' type of search - requires some skill.

As a business technology writer, it's part of my day job: I have to know how to phrase my queries, which sites will reveal certain types of results, where on a page I scan for meaningful content, where nuggets of useful data are buried or what is hyperbole. I get there eventually.

Most web users don't have the time or patience to develop the skills of a professional researcher so their search experience is the antithesis: type something into Google, choose the top three results, one of which will probably be Wikipedia.

So how do you make your site end up in the top three, or at least the first page of search results when users are searching for generic terms ('nuts and bolts') rather than specific ('Acme Inc')? (Note: you will nearly always come top in a search for your name unless you're really called Acme Inc.)

Search engine optimisation, or SEO - this sometimes arcane art - promises to raise your reputation among the billions upon billions of pages that Google scans and indexes every month. Many that practice SEO do so without really knowing for sure if it works. In fact, by being gung-ho with SEO, companies might actually damage their web marketing.

Let's start with a very basic introduction to how search engines work.

You submit your site to a search engine; it sends out spiders to search the entirety of your site that you want searched (pages that you do not want indexing should be in the robots.txt file). The spiders look at the keywords, meta tags, links and headlines. These pages are cached on the search engine's web servers and it uses proprietary algorithms on this vast database to prioritise the pages it thinks users want when they perform a search.

Initially search engines focused largely on meta tags submitted about a page's contents but this was open to widespread abuse, particularly by porn sites masquerading as something as innocuous as pages about lawn mowers or stock quotes, part of a practice known as 'spamdexing'.

The focus shifted then to the links between sites: the more links to a site, the more popular it was deemed. Again, this was open to abuse by 'blackhat' marketers in a practice known as 'link spam'.

With Google's rise to dominance, search techniques have evolved significantly. Google uses something like 200 different criteria to match the billions of pages in its caches with your search request.

One of these is PageRank, which estimates the likelihood that a given page will be reached by a web user who randomly surfs the web, and follows links from one page to another. Google also reviews links, keyword frequency, meta tags, headlines, site structure.

No one - outside of Google - knows quite how they do this, and the algorithms are constantly evolving. You can find out more on Google's help page.

We do know some basics: Google likes sites that…

Click here for page two