Find RSS Feeds in Yahoo and Google

(Notes to myself for later searching/exploring) Okay, I'm totally confused.

I'm doing some experimenting for finding RSS feeds within general search engine searches. So I took a Yahoo RSS feed and pulled out some characteristics that should be similar across most (all?) Yahoo RSS feeds -- the copyright disclaimer and the fact that the feed URLs have the string "rss" in them.

I added that up into the following Google query:

"Copyright (C) 2005 Yahoo! Inc. All rights reserved." inurl:rss

.. and got zippo results. Nada. Nothing. I ran a couple more searches to make sure I had the inurl: syntax correct, and then ran the phrase search by itself to make sure it pulled results and there wasn't something weird in there. Then I split the phrase, thinking that (C) might be goofing things up. The search:

"Yahoo! Inc. All rights reserved." inurl:rss

Finds over 700 results, but many of them are not from Yahoo, so I add the site: syntax:

"Yahoo! Inc. All rights reserved." inurl:rss site:yahoo.com

Finds 676 results, most from RSS-dev.

I went over to Yahoo.com and repeated the original search:

"Copyright (C) 2005 Yahoo! Inc. All rights reserved." inurl:rss

This search at Yahoo provides over 29,000 results, from a variety of properties in Yahoo. I'm trying to find RSS feeds for the directory so I search for dir in the URL:

"Copyright (C) 2005 Yahoo! Inc. All rights reserved." inurl:rss inurl:dir

... which takes me to 32 results.

I'm trying to figure out why these results aren't showing up in Google.

Hmm. I can abandon keywords entirely and just search Google for URL characteristics, with much more success:

inurl:getrss inurl:rss inurl:dir inurl:yahoo

(I don't use allinurl: so I can add and remove other search keywords without having to worry about where I'm putting them in the query.)

Doing that, I find 69 results as opposed to Yahoo's 34, and they all look like they're RSS feeds pointing to recently-added sites in the Yahoo directory, which is what I was looking for in the first place. When I search Yahoo for the inurl: query above I get 36 results.

So is Google not indexing the contents of these RSS feeds? Otherwise it would have gotten that copyright disclaimer. The feeds at Yahoo directory are of the filetype php, so maybe Google didn't recognize the content and didn't index it for that reason. Google's crawler is doing fine -- it appears to have more of Yahoo's directory feeds listed than Yahoo's search engine does! -- but how much of that content is ending up as keyword searchable is what I'm wondering about.

This is not to say anything good or bad about either search engine; I'm just looking into building a couple more tools and I wanted to do some experiments with feed-finding.

Posted to Internet-Technology-RSS | TrackBack