Exposing the Deep Web

There's an interesting article at the National Library of Australia called "Exposing the deep web to increase access to library collections" which covers guess what. You can read it at http://www.nla.gov.au/nla/staffpaper/2005/boston2.html .

It covers what the Library did to make sure their own little chunk of deep Web was exposed to search engines, most notably Google. In the middle of the paper is an eight-item list that any librarian seeking to make sure their Web materials get indexed could use as a starting point for a battle plan.

Interesting item in the list: "The Google harvester will only follow a maximum of 250 links per page." I had not heard that before. I'd heard that Google indexed a limit of 101K per page ( and I have my doubts about that) but I've never heard of a link limit.

Posted to Searching | TrackBack