Weblogs.com Turns the Corner

A little background. Weblogs.com was a place that automatically checked Web logs (including ResearchBuzz) and listed the most recently-changed pages.

Recently Weblogs.com "turned the corner" and changed the way they operate, because of the large number of Web logs that were being checked. Now, Web logs which wish to be included in the "changes.xml" list (more about that in a minute) have to alert Weblogs.com that their Web logs have changed.

Let's take these things one at a time. If you have a Web log and want to participate in Weblogs.com, you've got several options for alerting Weblogs.com that your 'blog has changed. If you're like me and you couldn't program your way out of a wet paper bag, use the manual method. Next time you update your Web log, fill out the form on this page: http://newhome.weblogs.com/pingSiteForm . The result page will have the form filled out. BOOKMARK THE RESULT PAGE. Next time you want to alert Weblogs.com of your page update, all you have to do is bring up the bookmarked page.

Now if you do know how to program, you have several options for alerting Weblogs.com when your page changes. There's a list of methods you can use at http://newhome.weblogs.com/directory/11/implementations . Implementations are available in Perl, PHP, AppleScript, etc. And if you are not into programming but want to do something technical and groovy, there's a bookmarklet available at http://aaronland.net/src/javascript/userland/bookmarklet.ping.html .

Now, back to the changes.xml file. When a Web log reports in as changed, its added to a changes.xml file. Now, here's the cool thing, and the whole reason I'm covering this in ResearchBuzz. Other sites can use the changes.xml file with their own Web sites. DayPop, the Web log/news search engine, is using changes.xml page to determine when its search engine should crawl Web logs. This means that in a best-case scenario DayPop can index sites ONE HOUR after they add new information (worst case, using changes.xml, is five hours.) Dan Chan, the man behind DayPop, says:

For Daypop, supporting the changes.xml file from weblogs.com was an easy step to take. The XML file is short and simple and easy to parse. It didn't take much work at all to grab what amounts to a list of updated weblogs along with update times and use that information to start recrawling those weblogs. This results in a "fresher" index -- one step closer to a true Just-In-Time search."

Now why am I covering this? Couple reasons. 1) This technology allows a search engine to make a big step forward. 2) The final criteria for inclusion in ResearchBuzz is "would a reference librarian find it useful?" The idea that one site acts as an aggregate and generates a .xml file that another site can use is really exciting, in my opinion. I'd like to explore more examples of this; expect more coverage of such things in weeks to come if I can find 'em.

Posted to Internet-Weblogs