Search: Google-Yahoo Comparison

I have just conducted an assessment of the relevance of the Google and Yahoo search engines (French versions www.google.fr and fr.search.yahoo.com). 70 users (students) each made 20 queries that they chose freely from 10 themes suggested at random (2 queries per theme, 1,400 queries in total):

• News
• Animals
• Geography-travel
• Literature
• Music
• Nature
• Celebrities
• Politics
• Health
• Sport

Each student was asked to note the quality of the first link proposed by each of the two engines blindly (i.e. no information enabled the engines to be identified). The links were presented in random order to avoid any bias. 2,800 pages were thus viewed and assessed. After examining the pages, the user was invited to enter a mark on a scale from 0 to 5 for each:

• 0 = Completely dissatisfied with the result
• 1 = Dissatisfied with the result
• 2 = Rather dissatisfied with the result
• 3 = Generally satisfied with the result
• 4 = Satisfied with the result
• 5 = Completely satisfied with the result

Google comes slightly ahead of Yahoo, by 0.2 points (3.6 compared with 3.4). The difference is not huge, but it is statistically significant (t test: p = 3.5 x 10^-5). Needless to say, this test is only partial, as, for reasons of human cost, it only concerns the first link proposed by each engine. However, it does provide an interesting indication of the relative relevance of each engine. I also note a very slight progression by Google with respect to a comparison I made in April [fr] using the Voilà engine (and not, unfortunately, Yahoo) with exactly the same protocol. However, this difference is not statistically significant.

The detailed examination of links returned is equally instructive. The first link offered by Google and Yahoo is identical in 27% of cases. In a previous study (using a slightly different protocol), conducted in December 2005, the proportion was 24%. The order of magnitude is thus similar.

The most surprising result came from the use of Wikipedia. This use was marginal in December 2005 (see study). At the time, for all 10 results on the first page, 2% of the links proposed by Google and 4% of those proposed by Yahoo came from Wikipedia. On the first link alone, Google offered no Wikipedia results (at least not in our sample) and Yahoo offered 7%.

The strategies have changed completely. Today 27% of Google’s results on the first link alone come from Wikipedia, as do 31 % of Yahoo’s.

How can this sudden interest in Wikipedia by both engines be explained? It is undoubtedly connected with the increasing difficultly engines have in calculating satisfactory ranking. The good old days of PageRank algorithms are over. It was quite well suited to a fairly stable network over time that was quite highly interconnected. The explosion of blogs and news sites has changed the situation considerably. The majority of the Web is now of a volatile and ephemeral nature. In all but exceptional cases, posts and news bulletins are hardly interlinked.

Faced with these difficulties, Wikipedia and a few other reference sites such as Doctissimo, Allociné, and major daily newspaper sites (La Tribune, Le Monde, Le Figaro, etc.) are gilt-edged securities. They have a good image, and on questioning users, I have established that even when Wikipedia and other reference sites don’t reply exactly to the question asked (for example in the news sector), the engine is well appreciated nevertheless. The user generally thinks “it’s not what I’m looking for, but it’s relevant all the same”.

The average mark allocated by users when the result is in Wikipedia is nearly one point higher, in the case of Google and Yahoo, than the mark allocated to other results.

Hence, pushing Wikipedia to the maximum is a paying strategy with relatively low cost. It is, however, dangerous. When users come to realize that, for example using the Firefox search bar, they can search directly in Wikipedia if they want encyclopedic information, in Wikio for news and blogs, in Allociné for movies and so on, the concept (outdated, in my opinion) of the general search engine will have had its day. Its limits are already being felt.

---

Read also

• SEO: Google and its image (December, 9th, 2007)
• SEO: Google, it's all tits and bums! (December, 12th, 2007)
  

---

Older

• Search-engine comparison of six search engines (December 2005)
  
• Google-Voilà comparison (april 2007) [fr]
  
• Jean-Marie Le Ray: Google/Yahoo/Microsoft comparison (october 2007) [fr]
  

---

New

• Subscribe to English-only RSS feed !