Scott On Writing.NET : ASP.NET Talk, Blog Enhancements

Search Your Site Using Google's Custom Search Engine

Scott Mitchell — Mon, 15 Sep 2008 12:51:00 GMT

Remember when people used to debate what search engine was the best among the bunnch? Do you remember back in the day when Yahoo! differentiated itself from other search engines by having actual human beings catalog, summarize, and rate large swaths of the World Wide Web? Today, search is a commodity. It's ubiquitous. Web surfers expect there to be a search box and expect accurate results returned in the blink of an eye. Because of these expectations, it's now more important than ever that your website offers search. I don't care if the site has only five static web pages that only get updated once in a blue moon, you still need search.

The good news is that Google offers free tools for quickly and easily adding search to your site. With Google's Custom Search Engine (CSE) you can add search to your site within a matter of minutes. All it takes is configuring a few settings and then copying and pasting a snippet or two of HTML and JavaScript into your site. A couple of months ago I used CSE to add search to this blog. You can find the search interface in the upper right hand corner of every page or by visiting the Search page.

To learn how to add a search engine to your site using CSE check out my latest article on DotNetSlackers: Implementing Search in ASP.NET with Google Custom Search.

Site Note: Another technique for making your site more searchable is to create an OpenSearch provider. By adding a simple XML file to your website you can have your site searchable through the search box that's baked into the user interface of browsers like Internet Explorer and Firefox. Learn more about this technique by reading: Helping Visitors Search Your Site By Creating an OpenSearch Provider.

FeedBurner and Changing a Blog's Feed URL

Scott Mitchell — Sun, 28 Aug 2005 21:23:00 GMT

This weekend I moved over my RSS feed - previously http://ScottOnWriting.NET/sowBlog/Rss.aspx - to a feed managed by FeedBurner (http://feeds.feedburner.com/ScottOnWriting). FeedBurner serves as a sort of feed URL proxy. Basically you give FeedBurner a link to your RSS feed and it creates a feed based on that feed. You then point your subscribers to the FeedBurner feed and FeedBurner serves up your site's content, maintains statistics on who's subscribing to your blog, and so on.

I decided to move to FeedBurner to realize three benefits (keep in mind that ScottOnWriting.NET (still) runs off of an old version of Scott Watermasysk's .Text blogging engine, as I've yet to upgrade to Community Server; previous to today, I was actually using a pre-0.94 version, but today "upgraded" to the official 0.94 release downloadable from the .Text GotDotNet Workspace):

Subscription statistics - FeedBurner provides a number of free statistics, including number of subscribers, number of requests, and aggregator breakdown.
Someone else handles the bandwidth - currently requests to the RSS feed on ScottOnWriting.NET consume roughly 1.5 GB of traffic per week, or 6 GB of traffic per month (in total, ScottOnWriting does about 11 GB of traffic per month). That's a lot of 1s and 0s that would be nice to offload to another party. (I don't believe the pre-0.94 version of .Text I was using supported conditional HTTP GETs (although if I'm not mistaken the "official" 0.94 release does; had I been using a version that supported conditional GETs this bandwidth requirement would be an order of magnitude lower, I'd wager, perhaps just a GB for the month.) (To clarify, while FeedBurner does make requests to the blog's RSS URL, it caches the results for a period of time, thereby reducing the bandwidth demands for my server.)
FeedBurner has a couple of neat “publicizing“ tools - FeedBurner includes a number of tools to easily make links to add your blog to My Yahoo!, MyMSN, newgator Online, and so on. Additionally, there are nifty little tools you can use to “show off“ how many folks subscribe to your blog, a la:

When changing over your RSS feed URL the main challenge is making sure that your existing subscriber base starts to use the new feed URL. There are, to my knowledge, to ways this can be done, with the first of the two ways being the ideal way:

Have your old feed URL emit an HTTP 301 status code - The HTTP 301 status code is a message from the server to the client saying, “Hey, this resource has been permanently moved to URL xyz.“ The client, then, can make a new request to the specified URL; too, if there's some database being used to track the URL, this message informs the client that it's time to update the database and use the new location. If I'm not mistaken, virtually all modern aggregators support HTTP 301 status codes and will automatically update a site's feed URL to use the newly specified location.
Tell people of your new feed URL - if you do not have control over your blog website you may not be able to take the steps needed to replace the current feed URL with an HTTP 301 status code. In this case, the only approach I know of to inform users of the new feed URL is simply through word of mouth. That is, you'll just have to post on your blog an entry telling users to update their aggregators. As Kent Sharkey has noted, though, the results may be somewhat disappointing.

Since I run ScottOnWriting.NET myself (well, through a web hosting company), I have control over these matters. The only challenge, then, was getting .Text to play nice. In .Text version 0.94 the site's RSS feed comes from a file named Rss.aspx. This file, though, does not actually exist; rather, in the Web.config file all requests are handed off to a .Text HTTP Handler. When a request comes in for Rss.aspx, .Text generates the appropriate output.

To get Rss.aspx replaced with an HTTP 301 status code, the first step is to create an Rss.aspx file in your blog's root directory. The code needed for this page is alarmingly simple - all you want to do is return an HTTP 301 specifying the new feed URL, like so:

(Of course replace the http://feeds.feedburner.com/ScottOnWriting Location header value with the URL of your new RSS feed...)

Creating this file is not enough. In fact, even after creating this file if you visit Rss.aspx through your browser you'll still see the complete RSS feed rather than being auto-redirected to the specified URL. This is because the ASP.NET engine is handing off the request to the .Text HTTP Handler rather than handling the request itself. If you look at the <httpHandlers> section in the Web.config file you'll find an entry like:

This entry says, “Any request for an ASP.NET page should be handled by the class Dottext.Framework.UrlManager.UrlReWriteHandlerFactory,Dottext.Framework,” and HTTP Handler. This includes requests for Rss.aspx. Hence we need to add the following line to the <httpHandlers> section:

That tells the ASP.NET engine to take care of requests to Rss.aspx. At first I naively thought that I was done, but I had just unwittingly setup an infinite loop! When a request comes into Rss.aspx, it sends back a 301 status code to the client, saying, “No, no, no, you want to go to this FeedBurner URL.“ This is what we want to tell people coming through a browser or aggregator, but remember that FeedBurner also needs to know the URL of the site's feed, which, at this point, I had set simply as Rss.aspx! So when FeedBurner periodically checked to see if a new version of my feed was available it requested Rss.aspx, which told it to check itself, which says to check Rss.aspx, which says to check itself, which... you get the point.

Instead, what I needed to do was rename .Text's Rss.aspx to something else, like RssFromText.aspx and instruct FeedBurner to use this alternate, “secretive” feed URL. With this setup, a user who already subscribes to ScottOnWriting.NET through Rss.aspx will automatically be switched over to FeedBurner. FeedBurner's RSS content will be populated from RssFromText.aspx, which will be generated from .Text, reflecting the most recent blog entries. No more infinite loops!

To accomplish this I had to edit blog.config to tell .Text that it should use RssFromText.aspx as its RSS feed URL as opposed to Rss.aspx. This involved updating the appropriate <HttpHandler> line like so:

With this addition, requests now to Rss.aspx are sent back an HTTP 301 status code, but FeedBurner can still slurp down the site's content through RssFromText.aspx.

Next, you'll probably want to update the link to the site's feed in the My Links section (since this will point the users to Rss.aspx, but you want them to go directly to the FeedBurner link). (In actuality, this step is probably optional since even if you do leave it as Rss.aspx, when the attempt to view that page through a browser or slurp it through an aggregator, they'll get the 301 status code and auto-redirect to the FeedBurner URL... but still, for completeness let's change this link.) To accomplish this, simply edit the MyLinks.ascx file in the ~/Skins/skin_name/Controls/ directory. With version 0.94 you'll find two HyperLink controls that .Text automatically looks for and sets their NavigateUrl properties to Rss.aspx - these controls have IDs Syndication and XMLLink. Even if you explicitly set the NavigateUrl properties to your FeedBurner URL, .Text will overwrite it and the link will be rendered as Rss.aspx. If you try to simply remove these HyperLink controls you'll find that (at least with 0.94) .Text will barf. What I did was simply set their Visible property to False. I then added two HyperLink Web controls of my own that referenced the new feed URL.

There's one more facet that should be changed, although I've not made the change since (to my understanding) you'd need to actually hack the .Text source code, recompile, and re-deploy. In the portion of the web pages in your blog you'll find a tag that's used for RSS feed auto-discovery:

Ideally the href attribute would contain the URL of your new feed... but I didn't feel like going through the headache of pecking through the source, making a change, testing, and so forth. So I just left it as-is, figuring in the worst case someone will “discover” my feed to be Rss.aspx, which will automatically be updated to the FeedBurner syndication URL as soon as their aggregator makes its first request to Rss.aspx.

The FeedBurner service looks pretty cool upon first glance. Once this new feed URL gets some use and I get some metrics in FeedBurner's database, I plan on sharing some of the stats... it'll be interesting to see what the most popular aggregator out there is for those who are ASP.NET developers (the primary audience of this blog, I imagine), among other data points.

Searching Your Blog Got a Whole Lot Easier

Scott Mitchell — Tue, 01 Feb 2005 14:54:00 GMT

If you check out the ScottOnWriting.NET homepage you'll now find that in the upper-left hand corner there's a spiffy “Search” box. Go ahead, type a query in there... you'll be redirected to http://scottonwriting.net/Search.aspx?s=ScottOnWriting.NET&count=10&first=1&query=query, which will show you the results of your search.

The new search page - which you, too, can add to your site in a matter of minutes - is possible in large part thanks to Microsoft's new MSN Search. Today Microsoft unveiled their new “built from the ground up” search and one of the cool features it that the search results can be returned as RSS. (To try it out, go to the MSN Search page and enter a query; at the bottom of the Web page you'll find that little, friendly orange RSS button.) Once I found this, I realized adding search to ScottOnWriting.NET would be a cinch if MSN Search let me search for pages on a particular domain. Thankfully, you can search by a specific site by simply adding site:siteName syntax in the search query.

Creating a search engine for a local site, then, is as easy as having a page that requests the remote RSS file based on the query entered by a user, and displaying the resulting RSS in the page. With my free RssFeed control doing exactly this is a no-brainer, requiring just two lines of code. Therefore, the code and markup for the search page for ScottOnWriting.NET basically looks like:

Source Code
// Build up the request URL for the RSS data
string requestUrl = string.Format("http://beta.search.msn.com/results.aspx?q={0}+site%3a{1}&format=rss&first={2}&count={3}", searchQuery, “http://ScottOnWriting.NET/“, startIndex, resultsPerPage);

// Bind the results to the RssFeed control
searchResults.DataSource = requestUrl;
searchResults.DataBind();

Markup
<skm:rssfeed width="80%" id="searchResults" runat="server" Font-Names="Verdana"
Font-Size="Medium" CellPadding="5" ShowHeader="False" GridLines="None">

Some example results can be seen here.

This is pretty cool, but there are some gotchas.

The RSS feed doesn't provide any information if there's more results. There may be 15 results for the query DataGrid, but the RSS feed, by default, will only return the first 10. There's no way of knowing for sure if there are more results unless you grab another RSS feed, this time asking for records starting at 11. What complicates things is that if there are exactly 10 records, getting the RSS feed starting at 11 won't return nothing; rather it will re-return the first 10 records. Ick. My current solution is to just display a Next button if there are 10 records on the page... it's a hack, since it won't really do what is expected for queries that return a multiple of 10 records, but it's good enough.
I don't know how legal all this is. If you look at the RSS feed's <copyright> tag it says:

Copyright © 2005 Microsoft. All rights reserved. These XML results may not be used, reproduced or transmitted in any manner or for any purpose other than rendering MSN Search results within an RSS aggregator for your personal, non-commercial use. Any other use of these results requires express written permission from Microsoft Corporation. By accessing this web page or using these results in any manner whatsoever, you agree to be bound by the foregoing restrictions.

I doubt MS is going to care too much if I use these results for my dinky little blog, but clearly they'd care if I was a company that used this approach to provide search on my Web site in lieu of buying Index Server or Google's searching server, or whatever.

Of course, MSN Search is not the first search engine to provide a programmatic means for accessing their results. Google offers the Google API, a set of Web services that can be used to run queries on Google. In fact, I've written an article on 4Guys on how to create a search engine with the Google API: Searching Google Using the Google Web Service. (And I will likely write up an article on using the MSN Search technique...) The downside of using the Google API is that users are limited to 1,000 queries per day and creating a page to work with the API takes a bit more heavy lifting than just bringing back RSS data and displaying it (although not much more).

A future blog entry, or 4Guys article, will delve into the MSN Search techniques I used to add search to ScottOnWriting.NET, but don't let that stop you from adding this feature to your site today. Armed with RssFeed you could do it without further information in a matter of minutes.

Enhancing Your .Text Blog - Allowing Readers to Rate Blog Entries

Scott Mitchell — Thu, 15 Apr 2004 20:12:00 GMT

One of the things I really like about .Text is just how easy it is to customize and enhance it. Of course, the complete source is available, so you can customize to your heart's content, but I'm talking about customizing it without modifying the source and recompiling. I've had past blog entries about customizing and digging into .Text (see Giving .Text a Calendar View and Analyzing your .Text Blog), and wanted to share my latest enhancement here on ScottOnWriting.NET: the ability for readers to rate a blog entry and leave feedback on why they made their rating (just like how MSDN's online articles have a “Rate this feedback“ section at the end of each article). The end result, you'll agree, looks pretty much just like Microsoft's rating interface, save that mine only allows you to rate from 1 to 5 instead of 1 to 9. It uses cookies to do a half-assed effort at ensuring that folks only rate a blog entry once.

If you're interested in adding this rating User Control to your .Text blog, I'll have the source for the User Control and an article discussing it on 4Guys sometime next week. What is cool about .Text (at least .Text 0.94, the version I'm using) is that the page layouts are specified in User Controls themselves (in the /Skins/skinName/Controls/ directory). This means that I could, with just a quick edit in Notepad, adjust the main page so that after each blog entry in addition to the Feedback (xxx) link, there's also a Rate Entry link, which will take you directly to the interface to link the blog entry.

The natural extension to this would be to allow users to read / view the most popular (and perhaps least liked) posts...

(The Rate Entry option is only available by visiting the ScottOnWriting.NET Web site directly. I guess I could attempt to embed the necessary HTML into the RSS feed, but I doubt I'll do that, since for that (I believe), I will need to edit the .Text source and recompile/redeploy.)