SEO - MSNBot - Searching For Ways To Make Redmond Rise Again

MSNBot - Searching For Ways To Make Redmond Rise Again
By Jim Hedger

What would you do if you were tasked with designing a new search engine?

You have all the resources the world can offer and the certain knowledge that your project is so important to your employer that mountains, molehills, companies, code and really comfy office chairs will be moved, built or acquired to meet your needs, no questions asked. Your boss demands a product that is better than best and, having failed to notice how overwhelmingly essential search would become back when he came to dominate everything else, appears ready to back your project with missionary zeal and Machiavellian maneuvering. The cold hard truth is, the future of one of the largest corporations in the world, owned incidentally by the world's wealthiest man, may well rest on your shoulders. In this scenario, there are no obstacles, only the challenge of beating Google at Google's best game. Whoa....

MSN released the beta version of their long awaited proprietary search engine earlier this quarter. Beta releases are the software world's version of a dress rehearsal. Mistakes will happen, even in the best productions, and the beta stage is the place to field-test a product, finding and fixing inevitable problems before the real, commercial version of the product is introduced. MSN(beta) search has seen its share of bumps over the past few weeks including a short period when it appeared the search tool had crashed. Regardless of any minor mishaps in its first weeks, MSN(beta) Search shows very good results generated from a database of approximately 5 billion spidered websites it began compiling over a year ago. While MSN(beta) and the search tool found at MSN.Com are different search tools delivering very different sets of results, the results generated by MSN(beta) will eventually replace the Inktomi based listings shown on MSN.Com. That's when the real fun will begin. Please note, as other commentators have pointed out, this is a BETA version and likely to change in coming weeks before the undisclosed live release date.

When told to build a better mousetrap, MSN engineers set their goals fairly high and approached the problem from the most logical point possible. They seem to have looked at the best ideas everyone else has come up with and tried to incorporate them into their search tool. The results are better then expected with highly relevant site listings that have been compared to earlier versions of Google's index. That makes sense given that MSNBot the beta-search spider works very much like GoogleBot, looking for many of the same site elements including incoming links, contextual relationships between linked documents, and overall site context. MSNBot also seems to be interested in keyword-enriched titles and seems especially interested in anchor text.

MSNBot, like GoogleBot and Slurp finds sites for its index by following links from one page to another within or between sites. The majority of sites in MSN(beta)'s index were found by MSNBot as it followed links from sites it had already visited. A check of backlinks, or links recognized by MSNBot as being relevant to a specific site almost always shows much higher numbers than a similar check on Google or Yahoo leading us to conclude that, for the time being at least, MSNBot does not filter links to the same degree as its rivals. In other words, relevancy does not appear to be as strong a factor with this version of MSN(beta) than it is with Google, at first glance anyway. One of the biggest improvements MSN(beta) brags about is its ability to figure out the context of individual paragraphs found on a page and apply that context as a "relevancy" factor against pages that might be linked to from that paragraph. Subsequent paragraphs on the same page might be about totally different topics without undermining the contextual relevancy of the links found in the previous paragraph. Google tends to compare relevancy on a page to page basis, making it more difficult to address a wide ranging topic on one page.

As with Google and Yahoo's spiders, MSNBot likes well defined and functioning link paths within your website. Providing a clear and well explained path for MSNBot to follow is critical to good rankings. The easiest way to accomplish this is to establish a text-based sitemap page appended to your website and be certain there is a link to that sitemap page on each of the other pages in your site. For database driven sites, this can be accomplished by changing the "footer" attribute on the template that creates the base-pages. There is an important thing to note here, especially for webmasters of highly dynamic or commerce driven sites, use static URLs to link to products in your database and do whatever is necessary to avoid tracking systems that append unique user IDs to URLs.

This article is not going to provide a lot of details around these elements as some or even much of what is written is subject to sudden change (this is a beta version after all), and the beta version simply hasn't been around long enough to express reliable ideas in writing yet. Once you have ensured that MSN(beta)'s spider can travel from one end of your site to another, and has a way into your site from an outside reference, take a look at the following elements of your site.

MSNBot seems to really like the techniques used by SEOs at StepForth. StepForth pays a lot of attention to keyword enrichment of the basic but critical elements of a site. Assuming navigation issues have been taken care of, websites that use keyword phrases in titles, anchor text, and early in the page content are doing very well in MSN(beta)'s index. We do not know for sure what MSNBot thinks of meta tags however we recommend using the basic description and keywords meta tags along with robot exclude text when necessary. MSNBot, basically likes clean code with good, common sense SEO. In a previous article, we republished the guidelines MSN posted to the MSN(beta) search site.

MSNBot Guidelines, at a glance:

Incoming links from other websites with keyword-enriched anchor text used to phrase the links.

Easily read code that has been W3C validated.

As with all search engines, best results are found when you only address one topic per page.

Keep your page site reasonable, 150kb is the maximum size recommended in the MSN guidelines.

Apply keyword phrases to well written sentences early in the code. Don't use techniques such as keyword stuffing or invisible text.

Use a sitemap to ensure that every page in your site is open to MSNBot.

There is a keyword density rule for MSNBot, however, we do not think that keyword density is the same for every business sector. For instance, the optimal keyword density for Maryland real estate will be different than the optimal keyword density California real estate, even though sites found under those keywords will represent the same business sector.

Any common sense rule that applies to SPAM on other search engines applies at MSN(beta) as well.

The MSN(beta) search engine is slated for full release any time now but, as with other Microsoft products, that doesn't necessarily mean we're going to see it anytime soon. The engine has been very stable over the past two weeks and is providing very strong and consistent results. Any bugs that remain to be worked out are well hidden and do not seem to be effecting the search function in any discernible way. When MSN does release their search engine as a full-version at MSN.Com, they will have a good tool that presents a credible alternative and serious challenge to Google and Yahoo. The long days of mono-culture search are over.

About The Author
Jim Hedger is a writer, speaker and search engine marketing expert based in Victoria BC. Jim works with a limited group of clients and provides consultancy services to StepForth Search Engine Placement. He has worked as an SEO for over 5 years and welcomes the opportunity to share his experience through interviews, articles and speaking engagements. Hedger can be reached at jim.hedger@gmail.com.