Home Welcome Messages Features
From social networking to swarm intelligence Sound strategy Partnering with parents Industry allies
The Parkinson’s predicament Merchants, moneylenders and middlemen Earnest money Search smarts Cultivating entrepreneurs Whole lot of shaking going on Aging gracefully A new dream for 21st-century science
More Research News
Research at Binghamton Discover-e
Contact Us
Feedback/Mailing Information How to reach us
Search smart

One day in the not-too-distant future, you’ll be able to type a query into an online search engine and have it deliver not Web pages that may contain an answer, but just the answer itself.

Search smart

User: “Who starred in the film Casablanca?”

Search Engine: “Humphrey Bogart and Ingrid Bergman.”

Not impressed?

Imagine asking a more nuanced question, such as “What do Americans think of offshore drilling?” A search engine will be able to respond with a report indicating trends in opinion based on what has been posted to the Web.

Search engines may eventually be used to conduct polling and even help sort fact from fiction, said Weiyi Meng, a professor of computer science at Binghamton University. He’s helping to make such futuristic possibilities a reality, both through his research and as president of a company called Webscalers.

The way Meng sees it, big search engines such as Google and Yahoo are fundamentally flawed.

You see, the Web has two parts: The surface Web and the deep Web. The surface Web is made up of perhaps 60 billion pages. The deep Web, at some 900 billion pages, is about 15 times larger.

Google, which relies on a “crawler” to examine pages and catalog them for future searches, can search about 20 billion pages, just a small fraction of the entire Web. Web crawlers follow links to reach pages and often miss content that isn’t linked to any other page or is in some other way “hidden.”

Weiyi Meng

Meng, along with colleagues at the University of Illinois at Chicago and the University of Louisiana at Lafayette, has helped pioneer large-scale metasearch-engine technology that harnesses the power of numerous small search engines to come up with results that are more accurate and more complete.

“Most of the pages on the deep Web aren’t directly ‘crawlable.’ We want to connect to small search engines and reach the deep Web,” he said. “That’s the idea. Many people have the misconception that Google can search everything, and if it’s not there it doesn’t exist. But we should be able to retrieve many times more than what Google can search.”

Not only can a metasearch engine probe deeper, it can also offer the latest information.

“In principle,” Meng noted, “small guys are much better able to maintain the freshness of their data. Google has a program to ‘crawl’ all over the world. Depending on when the crawler has last visited your server, there’s a delay of days or weeks before a new page will show up in that search. We can get fresher results.”

The concept is not new. In fact, the first metasearch engine was built in 1994.

“The big difference between our technology and the ones pursued by other people is that most of the other technologies do the metasearching on top of a small number of general purpose search engines, such as Yahoo, Google or MSN,” Meng explained. “We have a completely different perspective. We want to build large-scale metasearch engines on top of many small search engines.”


Continued >>

page 1 | page 2

©2009 Division of Research at Binghamton University. All Rights Reserved.
Photos and illustrations used throughout this site are copyright protected. For permission and terms of use, contact .