29th July 2007, 08:13 pm
Remember Gigablast? It’s at http://gigablast.com/; it’s a general search engine with a few specialty searches as well. It’s been rather quiet lately, but I’ve noticed that it’s started posting the number of pages indexed in its main search engine.
This is something that all search engines used to do, but which was abandoned a few years ago as some search engines claimed that more wasn’t better, others claimed there were apples-to-oranges comparisons going on, etc. So it was kind of surprising to see Gigablast announcing the size of its index — which at this writing is 12,643,911,680.
(I remember nostalgically the cow I had when Google surpassed a billion pages in its index.)
Knowing that Gigablast is up to 12 billion pages is interesting enough. But it’s also interesting to know that number and then compare the size of Gigablast’s results with the size of Google’s. For example, searching Gigablast for Yahoo brings about 380,719,416 results. Searching Google brings about 707,000,000 results. Searching Gigablast for Obey the Toaster brings 138 results, while Google’s search for the same phrase brings about 364 results. I bet if you ran a couple hundred different searches and compared the results of each of the two engines, you could get a decent idea of how much larger Google’s index is than Gigablast’s.
… but it only matters up to a point. Of course the most important thing about a data pool is how easy it is to search. On the other hand, it’s always good to know how deep that pool is to start with.
29th July 2007, 07:50 pm
I’m afraid I’ve never really understood why LookSmart owned Grub. It just didn’t make sense. And apparently LookSmart didn’t understand either, because Grub has been sold to Wikia, which has released it as open source.
For those of you playing along at home: Grub is a distributed computing project aimed at spidering the Web. Like Seti@Home, Grub uses spare computer cycles of its users, only in Grub’s case the Web gets crawled instead of aliens searched for. LookSmart bought Grub back in March 2003 and, as far as I can tell, sat on it for over four years. Then Wikia came along and bought it. Wikia, you may know, is the search-engine-from Wiki concept that’s been coming along for the last several months.
Not only has Wiki acquired Grub, but has also released it under an open source license. Not a lot going on yet — seems like the project has been moribund for quite a while. Another one to put on my radar…