The Life of a Sentient Search Engine

Well my dear brother, the computer scientist, has posted something on his blog that shows that I continue to have no influence on him whatsoever. His thesis is that Web 2.0 phenomena like social networking sites etc. make traditional search like Google obsolete.

(sigh)

He indicated that personal content made possible is now the future of information acquisition (even though we’re begining to see the pendulum moving back the other way).

(You can’t see me but I’m shaking my head in both amusement and dismay…)

Folks, in his defence his field is computers, so he doesn’t have to live in the real world. Nevertheless, his attitude is troubling. I don’t want to seem like I’m beating up on him, I just see all to many people who are computer experts and very obviously not usability experts making these kind of assertions.

He talks about how he uses the links list for a wikipedia page on a topic to find links to more information. You could also use services like Del.icio.us for this as well. This has some legitimate basis, and I’ll be the first to admit that. These methods are a great way to find “vetted” sites. However, this method assumes that there will be useful links in said Wikipedia article or del.icio.us page, which my brother graciously half admits might not be the case.

Another problem comes when defining “vetted”. There are numerous sites I’ve seen used as sources and useful links in Wikipedia and in Del.icio.us lists that would never pass inspection with a real information professional (librarian). The fact is that there’s no accountability when it comes to this sort of publicly edited content. The links provided may be good, but, on the other hand, they’re more likely to be total crap.

If we’re supposed to find information on the web using prepreared lists, how are we supposed to find new stuff that might be more current or useful. If we become a society of surfers insead of searchers, the majority of us are almost certain to miss out on stuff that lies hidden because it remains undiscovered. If we stick to well worn paths it occurs to me that we might be risking a little intellectual stagnation.

The discussion of hidden brings up another interesting issue. This method of information retrieval does nothing to address the greatest flaw of Google: most of the best stuff is not on the open web (Gasp). Instead, it lies sequestered in the subscription databases that you can access through a library’s website or by paying big bucks. Wikipedia and Del.icio.us are just as impotent at providing access to this material as Google.

My brother also make the rather amusing assertion that “categorizing information is no longer difficult, either (Not a solved problem, but one that there are available solutions for.)”

(heavy sigh)

Where do I begin? Matt, catagorizing information is no easier today than it was 50 years ago. If anything, its more complex on the web than anything else. Folksonomies are nice. They provide a valuable service and make the process of finding more intuitive at times. However, they are not the solution. Indeed, sometimes they are part of the problem.

Getting anyone to agree on what to call something is like trying to avoid Chinese-made products, you can’t. After all, one man’s “evangelical christian” is another man’s “fundamentalist nutcase”. Therefore, the idea that socially constructed folksonomies are going to allow users to easily find all the material on a topic is laughable. Spelling vaiations can cause a lot of problems (after all, we’re such great spellerz these days right?) as do “regionalisms” (pop vs soda vs “coke”), and homonym (bear [animal] vs bear [carry]) and homophone (to, two, and too) confusion.

Despite its flaws, a controlled vocabulary is the surest way to do a precise and reliable search. The terminology isn’t always cutting edge (the Library of Congress Subject Headings’ continued use of the term “cookery” springs to mind), but it nonetheless gets the job done more precisely in most cases for a serious search.

Truthfully, the best role for tagging is browsing. Tags excel at this as they let the user jump from one association to another and explore information. If you are actualy wanting to find something specific, you need to use a more traditional search method.

I know I’m coming off as a cranky old librarian here, but the fact it that searching fo information is what I do and I feel that while Web 2.0 makes for many interesting possibilities, it can’t possibly come to replace the more traditional methods of information retrieval. Now a better designed search engine…

Update 7/3/08 It is incumbent on me to admit that I may have mistook his meaning somewhat, but I stand by what I said nonetheless (don’t worry, we’re still talking. I’m to be his best man when he gets married this December).