Non Text-Based Searches-- by Nate Winter

When I was in grade school, the first source I learned to consult for answers was a parent. For 99% of bizarre questions, parents could usually be counted on for a satisfactory response. But for annoying spelling questions, the heavy, dusty, printed dictionary was the faithful standby. Many of us know from personal experience that when it came to a spelling question, the dictionary always seemed like an inefficient resource. How are you supposed to find a word in the dictionary, a reference organized only by proper spelling, if you don’t know how to spell it in the first place? It’s really just a scientific, wild-ass guess. The circular logic of this predicament has frustrated kids and liberated parents for generations.

In the internet age, you can reference anything organized based on whatever information you have available… as long as it’s text. As amazing and complex as web-enabled search is, it’s still based on 26 characters, ten digits, and a handful of other symbols (in English anyway).

This situation is just as limiting in web-based search as looking in a printed dictionary for a word you don’t know how to spell. While the possibilities of text-based search are still being discovered, a next generation of search is inevitable—one that allows a user to submit queries that aren’t text to a search engine.

Think about something visual and abstract, like a painting by Jackson Pollack or Piet Mondrian. If you didn’t know who the artist was or even that it was art, how would you search to find out what it was? In a text-based search, it would be nearly impossible. But in an effective image-based search, you’d have a much better shot.

The same goes for an anonymous instrumental audio track or a mysterious video clip. If you don’t have specific words to describe it, text-based search is useless.

The process for a next generation search would go as follows. The user uploads their image, audio, or video file. The engine then compares advanced quantitative data such as file name, file size, creation date, etc. to it’s index of knows files to find matches. This type of technology already exists, too. Apple’s iTunes software looks at data from an audio CD and compares it to a database of artists, album titles, and song titles suggesting the correct information. Currently this technology only works for exact copies of full albums, so it will recognize Pink Floyd’s Dark Side of the Moon on CD, but not your Pink Floyd Party Mix. Although, this technology probably isn’t far off.

Admittedly this functionality is still text-based and the idea of uploading a file for the search engine to evaluate is not so far fetched either. The real challenge lies in a search engine having an index of images, audio clips, and video clips to compare with the user’s search file.

The next generation goes beyond text completely, into a realm where the search engine recognizes elements of the search file’s content.

Digital cameras already offer facial recognition technology, picking out human faces in the camera’s viewfinder and selecting them as auto-focus points. Now imagine what a search engine with advanced visual, and audio content matching could do when hooked up to a massive database. That’s essentially what government fingerprint scanners and facial feature matching systems can already do. It just needs to be repurposed to accommodate the infinite array of search files users would inevitably submit.

While it’s anyone’s guess as to how long it will take for search technology to get to this level, there’s little doubt it will do so. And probably in my lifetime. In the future, our children, when faced with a spelling question, will still probably ask their parents first. But when faced with the inevitable, "Look it up," they will simply speak the word they wish to spell into a microphone linked to an audio-driven search engine. There will be no guesswork, no frustration, no trial and error. The search results will be accurate and immediate, allowing our children to spend more time asking question and less time searching for answers. Great for our children, annoying for their parents.

-- Nate Winter

[This blog entry is not intended to be an entry in the Culture/Ed blog. It is posted for temporary purposes only as it is part of a blog on theoretical subjects.]