How do Search Engines Work?

Search engines help People Find Companies, Products & Services and minimize the time required to find information

Before your website can rank highly in the search engines, it is important to understand how search engines work.

In order for search engines to tell you where to find information, it must find the information first. To find the millions and millions of global web pages that exist, the search engines use programs called ‘spiders’, or robots. These spiders crawl the web and build indexes of URLs, keywords found on web pages and other pertinent information.

The search engines are basically made of three parts: the spider, the index and the software.

The spider visits a web page, reads it, and then follows links to other pages within the website. This is what ‘crawling’ is generally referred to. The spiders return to the site (frequency depends on varying factors) to look for changes.

Once the spider has crawled your site it adds all the information it found to the index. Sometimes referred to as a catalogue, it’s like a giant book containing a copy of every single page a search engine spider finds. If a web page changes, then the index gets updated with the new content. Sometimes it can take some time for new pages or content to be included in the index. This means you may see a visit by a search engine spider in your log files, but is not yet indexed. Until your web pages are indexed, it is not available to people searching with search engines.

So your pages are now indexed – how do the search engines decide who comes first?

While each search engines has its own unique way of ranking web pages (algorithms) there are common themes that they all share.

Like in real estate, it’s all about location, location, location. The more prominently you place keywords important to your website, the more relevant your website appears to be for those particular keywords. Frequency also plays a role — repeated too few times and you lose relevancy, yet repeated too many times and you can be seen as spamming the search engines. There is a delicate balance that must be reached between location and frequency.

The weight placed on each location also varies between search engines. That is why search results will often be different among the various search engines.

However, keyword placement and frequency are not all that the search engines look at in order to rank pages by relevancy. There are other variables that come into play that are commonly referred to as “off-page” factors. These off-page factors include the number of incoming inks to your site and click-through measurement.

The incoming links show the search engines how ‘popular’ you are among sites with similar topics or themes. Essentially, the more links or ‘votes’ you have, the better. However, the search engines do have sophisticated techniques that are used to screen out attempts by websites to build artificial links designed specifically to increase their placement in the search engines.

Click-through measurement is a way for the search engines to watch what results searchers click on for any given search term. Through this analysis the search engines may ‘drop’ pages from its top rankings and ‘boost’ lower ranked pages if they are generating more clicks than the top-ranked pages.

Simplified view of spiders crawling the web.

To learn more about an Internet Solution that really works for your business, use the form on “Contact Us” page to secure a free one-on-one consultation session- to help your business gain full benefit from a web presence.