Scott On Writing.NET : Blog Enhancements

New Software Running ScottOnWriting.NET

Scott Mitchell — Thu, 27 May 2010 18:50:00 GMT

When I started this blog in July 2003 there weren't many available blog engines build atop ASP.NET. One of the more interesting ones at the time was Scott Watermasysk's .Text blog engine (which eventually became part of Community Server. Over the past nearly seven years ScottOnWriting.NET has continued to be powered by .Text, even though the code base was discontinued circa 2004. There were two primary reasons I stuck with .Text for so long:

I could find no easy way to migrate my existing content from .Text into Community Server. I believe there was such a tool created in the early days of Community Server, but I had no luck with it, as I recall. Eventually, this product disappeared and the only migration tools I could find were from older versions of Community Server to newer ones, but none for .Text.
There were URL changes between .Text and Community Server, so switching over would immediately result in a slew of broken URLs. I am a strong believer in URLs as public interfaces and view broken URLs and believe each broken URL makes the Internet that less useful. Plus it makes the Internet Founding Fathers - Tim Berners-Lee, Vint Cerf, and Al Gore - cry.

Well, this week I decided to take the plunge and move ScottOnWriting.NET and its hundreds of posts over to a new blog engine, namely Community Server (albeit an older version of Community Server - when will I learn?). I wrote some custom SQL scripts to (the best of my ability) move over all blog posts and categories (or tags, as they're called now-a-days). I also spent a good deal of time writing some regular expressions and very big switch statements to properly reroute URLs. For example, if you try to visit an old ScottOnWriting.NET URL, like http://scottonwriting.net/sowBlog/posts/145.aspx, you should be automatically redirected to the new URL, http://scottonwriting.net/sowBlog/archive/0000/00/00/162659.aspx. Likewise, if you visit an old category URL or an old moth/year archive URL you should be redirected to the new URLs. If this is not the case, if you find a broken URL or other problem on this site, please let me know.

Note that I did not mention bringing over the past blog comments. I'm still on the fence as to whether I want to spend the time to do this. There were hundreds of comments left over the years, many of them quite helpful and many that added substantially to the discussion. However, I've already sunk a number of hours into this migration and am hesitant to burn too much more time. Plus I still have a number of aesthetic and cosmetic things to do, like update the site's CSS, add in some of the widgets from the original site that aren't part of the default Community Server setup, and so forth. My goal is to eventually import these past comments, but I wouldn't be at all surprised if this is still on my TODO list come 2011.

In closing, please do let me know if you run into any problems with the new blog engine.

Thanks!

Search Your Site Using Google's Custom Search Engine

Scott Mitchell — Mon, 15 Sep 2008 12:51:00 GMT

Remember when people used to debate what search engine was the best among the bunnch? Do you remember back in the day when Yahoo! differentiated itself from other search engines by having actual human beings catalog, summarize, and rate large swaths of the World Wide Web? Today, search is a commodity. It's ubiquitous. Web surfers expect there to be a search box and expect accurate results returned in the blink of an eye. Because of these expectations, it's now more important than ever that your website offers search. I don't care if the site has only five static web pages that only get updated once in a blue moon, you still need search.

The good news is that Google offers free tools for quickly and easily adding search to your site. With Google's Custom Search Engine (CSE) you can add search to your site within a matter of minutes. All it takes is configuring a few settings and then copying and pasting a snippet or two of HTML and JavaScript into your site. A couple of months ago I used CSE to add search to this blog. You can find the search interface in the upper right hand corner of every page or by visiting the Search page.

To learn how to add a search engine to your site using CSE check out my latest article on DotNetSlackers: Implementing Search in ASP.NET with Google Custom Search.

Site Note: Another technique for making your site more searchable is to create an OpenSearch provider. By adding a simple XML file to your website you can have your site searchable through the search box that's baked into the user interface of browsers like Internet Explorer and Firefox. Learn more about this technique by reading: Helping Visitors Search Your Site By Creating an OpenSearch Provider.

Giving a CAPTCHA a Whirl

Scott Mitchell — Thu, 06 Jul 2006 21:17:00 GMT

Comment spam is evil. I've been getting on the tune of 25-50 comment spams per day the past several weeks. My custom utility to quickly delete comments in .Text has helped delete comment spams after the fact; additionally, SQL triggers have, to date, proactively nuked over 20,700 comment spams (although they have also stopped valid posters who have added common spam 'keywords' to their posts).

Since the SQL triggers are clearly no longer working, I'm going to give CAPTCHAs a try. In theory, CAPTCHAs can be broken... easily. However, I'm hoping/assuming that the vast majority of scum known as comment spammers aren't using programs that can decode CAPTCHAs and aren't using the social engineering/free pr0n!! techniques detailed here. Rather, I'm assuming a scant few of the comment spams (the ones I get like one or two of a day) are entered by actual humans; the majority, of which I get a blast of, say, 30 in a five second period, are probably coming from a pretty dumb HTTP screen scraping/HTTP posting program. If my assumptions are sound, then the CAPTCHA ought to drastically reduce the amount of comment spam appearing on my blog.

I've taken down my comment spam-related triggers and replaced them with Miguel Jimenez's free Clearscreen SharpHIP CAPTCHA Control. The nice thing about this control is that it automatically checks for validity on postback so it integrates with .Text without having to modify the codebase. One downside, however, is that any other postback sections on the web page will no longer work unless the CAPTCHA is filled out (you may have noticed the blog entry rating feature has been removed).

Hopefully the CAPTCHA cuts down on the comment spam, as having to wade through and delete 50+ comments every day or two is really started to get mundane and annoying. (I do take removing comment spam seriously, though. I challenge you to find a single piece of irrefutable spam in one of my blog entries that's older than, say, a week.)

Quickly Deleting Comments in .Text

Scott Mitchell — Tue, 28 Mar 2006 16:27:00 GMT

While I'm certain that Community Server has the ability to easily delete comments en mass, this blog still uses Scott Watermasysk's close to originally-released version of .Text (gasp!). I've been meaning to upgrade, I promise, but with the assorted customizations I've done and the time needed to make the transition, it's hard to justify taking the time to fix something that's really not broken.

My one complaint, however, has been comment spam. This blog is blasted with dozens of comment spam messages on a daily basis. Most of these are filtered out proactively with some machine-level tests (essentially looking for keywords that indicate spamminess), which I've blogged about before. In fact, to date the technique I use to stop comment spam has quashed 13,046 messages from making their way to the ScottOnWriting.NET blog.

Regardless, comment spam still creeps in and deleting it through the .Text admin page is a pain. You see a list of the comments, each with a Delete button. Cicking Delete takes you to a screen that says, “Are you sure you want to delete this?” Hitting Confirm takes you back to the main page with the comment deleted. This is OK when there are one or two comment spams to delete, but a carpal tunnel-inducing nuissance when you are blasted with, say, 15 spam messages in a two minute period. Ideally I would be able to check a number of checkboxes and click a “Delete Selected” button.

To further put off the upgrade to Community Server and to get some good ol' ASP.NET 2.0 practice, I decided to create a simple ASP.NET 2.0 website locally that would provide such an interface. And I was able to create it in under 5 minutes with having to write less than 10 lines of code. 2.0 rocks.

Anyway, the code is available for download here. It's just a single page, Default.aspx, which lists the 50 most recent comments in a GridView. Each GridView row has a CheckBox. At the bottom there's a “Delete Selected Comments” Button that, when clicked, iterates through each GridView row and, if selected, programmatically invokes the SqlDataSource control's Delete() method (which is wired up to the blog_DeletePost sproc). (The exact sprocs/queries used here may differ for more modern versions of .Text or Community Server.)

For those three people still using .Text who haven't upgraded to Community Server, enjoy!

FeedBurner Summary

Scott Mitchell — Thu, 15 Sep 2005 16:33:00 GMT

In August I moved over my RSS feed from the default .Text RSS feed source (Rss.aspx) to using FeedBurner's free service. Part of the challenge in this process was having existing subscribers automatically switch from using Rss.aspx to using FeedBurner's generated feed (http://feeds.feedburner.com/ScottOnWriting). I ended up getting everything to work by retooling Rss.aspx to send an HTTP 301 status code to aggregators, which instructed them to update their information using the new feed URL. For more on the reasons why I switched to FeedBurner along with how I made the needed changes in .Text to send an HTTP 301, refer to FeedBurner and Changing a Blog's Feed URL.

I've now been using the FeedBurner service for coming on three weeks, and wanted to share a quick review of the service. In my previous blog entry I mentioned the three motivating factors that prompted me to switch to FeedBurner were:

Subscription statistics - FeedBurner provides a number of free statistics, including number of subscribers, number of requests, and aggregator breakdown.
Someone else handles the bandwidth - currently requests to the RSS feed on ScottOnWriting.NET consume roughly 1.5 GB of traffic per week, or 6 GB of traffic per month (in total, ScottOnWriting does about 11 GB of traffic per month). That's a lot of 1s and 0s that would be nice to offload to another party. (I don't believe the pre-0.94 version of .Text I was using supported conditional HTTP GETs (although if I'm not mistaken the "official" 0.94 release does; had I been using a version that supported conditional GETs this bandwidth requirement would be an order of magnitude lower, I'd wager, perhaps just a GB for the month.) (To clarify, while FeedBurner does make requests to the blog's RSS URL, it caches the results for a period of time, thereby reducing the bandwidth demands for my server.)
FeedBurner has a couple of neat “publicizing“ tools - FeedBurner includes a number of tools to easily make links to add your blog to My Yahoo!, MyMSN, newgator Online, and so on. Additionally, there are nifty little tools you can use to “show off“ how many folks subscribe to your blog, a la:

The free FeedBurner service provides three traffic metrics:

Feed Circulation, which shows how many folks subscribe to your blog, how many requests there were to your RSS feed, and how many click throughs there were. The circulation data can be broken down by day or by hour, and shown in terms of the current week, the current date, the current month, and so on. Here's a bar graph that shows the circulation for ScottOnWriting.NET since I started using FeedBurner in late August:
Readership - the readership stats allow me to see what aggregators are being used to subscribe to my site's content, along with a breakdown of bots and browser aggregators. As the following graphic shows, the most popular aggregator that's reporting itself (or that FeedDemon knows of) is Bloglines, followed by RssBandit (my aggregator of choice).
Item Stats - this shows the click throughs on an item-by-item basis over a specified date range. For example, the most popular blog entry of mine since late August (at least in terms of subscribers “clicking through“) is How Big is Too Big a ViewState? with 171 click throughs.

The second motive of mine for using FeedBurner was to have someone else (namely FeedBurner) bear the bandwidth costs. (Thanks, guys!) Anywho, in the three weeks prior to using FeedBurner the daily bandwidth for ScottOnWriting.NET was 388 MB. Since moving to FeedBurner my daily bandwidth average has dropped to 211 MB. FeedBurner alone is saving me 177 MB per day, which is more than 5 GB per month. Sweet!

One of my initial concerns with FeedBurner was that once you were using FeedBurner you were “locked in.“ That is, I worried that if, down the road, I wanted to switch back to hosting the RSS feed on my site (or use some FeedBurner competitor, or if FeedBurner went out of business), I'd be SOL, since how would I get my subscribers who subscribe to my FeedBurner feed to switch to a different feed? I'd need to HTTP 301 my FeedBurner feed and since that's hosted with FeedBurner, they have the ultimate say as to whether or not that would be possible.

This fear was assuaged by a blog post by FeedBurner cofounder and CTO, Eric Lunt. In the post Eric mentions that it is possible to have your FeedBurner feed use an HTTP 301 and spells out their business rules for implementing this (namely, different actions are taken by the feed as time progresses... the HTTP 301 isn't used indefinately, it eventually unwinds to use, basically, a 404). From Eric's entry:

... when you start directing subscribers to FeedBurner, you may, in the future (way way way in the future) change your mind and want those subscribers pointing back to your original feed. You would probably also like this to happen automatically, and you would probably like some fallbacks for subscribers who don't get redirected for some reason. To date, there has been no simple way to do this. Steve Gillmor first raised this point with us during an interview late last year, and it has also been discussed more recently. We think we have the best feed management service, we think that providing publishers with the ability to do whatever they want is always the right answer, and most importantly, we think your subscribers are your subscribers, not ours or anybody else's.

So, beginning today [June 10, 2005], we're providing a detailed service for publishers who choose to leave FeedBurner. When you delete your FeedBurner feed, we have added an option to redirect your feed. If you select this, we begin a one month process of transitioning your subscribers back to your source feed.

That's reassuring to know. If you couldn't guess, I highly recommend FeedBurner. The stats and bandwidth savings make this free service an invaluable one, and the ability to leave FeedBurner clealy and crisply takes away any potential downside in switching over your feed.

For more entries on customizing .Text be sure to check out the Blog Enhancements category.

FeedBurner and Changing a Blog's Feed URL

Scott Mitchell — Sun, 28 Aug 2005 21:23:00 GMT

This weekend I moved over my RSS feed - previously http://ScottOnWriting.NET/sowBlog/Rss.aspx - to a feed managed by FeedBurner (http://feeds.feedburner.com/ScottOnWriting). FeedBurner serves as a sort of feed URL proxy. Basically you give FeedBurner a link to your RSS feed and it creates a feed based on that feed. You then point your subscribers to the FeedBurner feed and FeedBurner serves up your site's content, maintains statistics on who's subscribing to your blog, and so on.

I decided to move to FeedBurner to realize three benefits (keep in mind that ScottOnWriting.NET (still) runs off of an old version of Scott Watermasysk's .Text blogging engine, as I've yet to upgrade to Community Server; previous to today, I was actually using a pre-0.94 version, but today "upgraded" to the official 0.94 release downloadable from the .Text GotDotNet Workspace):

Subscription statistics - FeedBurner provides a number of free statistics, including number of subscribers, number of requests, and aggregator breakdown.
Someone else handles the bandwidth - currently requests to the RSS feed on ScottOnWriting.NET consume roughly 1.5 GB of traffic per week, or 6 GB of traffic per month (in total, ScottOnWriting does about 11 GB of traffic per month). That's a lot of 1s and 0s that would be nice to offload to another party. (I don't believe the pre-0.94 version of .Text I was using supported conditional HTTP GETs (although if I'm not mistaken the "official" 0.94 release does; had I been using a version that supported conditional GETs this bandwidth requirement would be an order of magnitude lower, I'd wager, perhaps just a GB for the month.) (To clarify, while FeedBurner does make requests to the blog's RSS URL, it caches the results for a period of time, thereby reducing the bandwidth demands for my server.)
FeedBurner has a couple of neat “publicizing“ tools - FeedBurner includes a number of tools to easily make links to add your blog to My Yahoo!, MyMSN, newgator Online, and so on. Additionally, there are nifty little tools you can use to “show off“ how many folks subscribe to your blog, a la:

When changing over your RSS feed URL the main challenge is making sure that your existing subscriber base starts to use the new feed URL. There are, to my knowledge, to ways this can be done, with the first of the two ways being the ideal way:

Have your old feed URL emit an HTTP 301 status code - The HTTP 301 status code is a message from the server to the client saying, “Hey, this resource has been permanently moved to URL xyz.“ The client, then, can make a new request to the specified URL; too, if there's some database being used to track the URL, this message informs the client that it's time to update the database and use the new location. If I'm not mistaken, virtually all modern aggregators support HTTP 301 status codes and will automatically update a site's feed URL to use the newly specified location.
Tell people of your new feed URL - if you do not have control over your blog website you may not be able to take the steps needed to replace the current feed URL with an HTTP 301 status code. In this case, the only approach I know of to inform users of the new feed URL is simply through word of mouth. That is, you'll just have to post on your blog an entry telling users to update their aggregators. As Kent Sharkey has noted, though, the results may be somewhat disappointing.

Since I run ScottOnWriting.NET myself (well, through a web hosting company), I have control over these matters. The only challenge, then, was getting .Text to play nice. In .Text version 0.94 the site's RSS feed comes from a file named Rss.aspx. This file, though, does not actually exist; rather, in the Web.config file all requests are handed off to a .Text HTTP Handler. When a request comes in for Rss.aspx, .Text generates the appropriate output.

To get Rss.aspx replaced with an HTTP 301 status code, the first step is to create an Rss.aspx file in your blog's root directory. The code needed for this page is alarmingly simple - all you want to do is return an HTTP 301 specifying the new feed URL, like so:

(Of course replace the http://feeds.feedburner.com/ScottOnWriting Location header value with the URL of your new RSS feed...)

Creating this file is not enough. In fact, even after creating this file if you visit Rss.aspx through your browser you'll still see the complete RSS feed rather than being auto-redirected to the specified URL. This is because the ASP.NET engine is handing off the request to the .Text HTTP Handler rather than handling the request itself. If you look at the <httpHandlers> section in the Web.config file you'll find an entry like:

This entry says, “Any request for an ASP.NET page should be handled by the class Dottext.Framework.UrlManager.UrlReWriteHandlerFactory,Dottext.Framework,” and HTTP Handler. This includes requests for Rss.aspx. Hence we need to add the following line to the <httpHandlers> section:

That tells the ASP.NET engine to take care of requests to Rss.aspx. At first I naively thought that I was done, but I had just unwittingly setup an infinite loop! When a request comes into Rss.aspx, it sends back a 301 status code to the client, saying, “No, no, no, you want to go to this FeedBurner URL.“ This is what we want to tell people coming through a browser or aggregator, but remember that FeedBurner also needs to know the URL of the site's feed, which, at this point, I had set simply as Rss.aspx! So when FeedBurner periodically checked to see if a new version of my feed was available it requested Rss.aspx, which told it to check itself, which says to check Rss.aspx, which says to check itself, which... you get the point.

Instead, what I needed to do was rename .Text's Rss.aspx to something else, like RssFromText.aspx and instruct FeedBurner to use this alternate, “secretive” feed URL. With this setup, a user who already subscribes to ScottOnWriting.NET through Rss.aspx will automatically be switched over to FeedBurner. FeedBurner's RSS content will be populated from RssFromText.aspx, which will be generated from .Text, reflecting the most recent blog entries. No more infinite loops!

To accomplish this I had to edit blog.config to tell .Text that it should use RssFromText.aspx as its RSS feed URL as opposed to Rss.aspx. This involved updating the appropriate <HttpHandler> line like so:

With this addition, requests now to Rss.aspx are sent back an HTTP 301 status code, but FeedBurner can still slurp down the site's content through RssFromText.aspx.

Next, you'll probably want to update the link to the site's feed in the My Links section (since this will point the users to Rss.aspx, but you want them to go directly to the FeedBurner link). (In actuality, this step is probably optional since even if you do leave it as Rss.aspx, when the attempt to view that page through a browser or slurp it through an aggregator, they'll get the 301 status code and auto-redirect to the FeedBurner URL... but still, for completeness let's change this link.) To accomplish this, simply edit the MyLinks.ascx file in the ~/Skins/skin_name/Controls/ directory. With version 0.94 you'll find two HyperLink controls that .Text automatically looks for and sets their NavigateUrl properties to Rss.aspx - these controls have IDs Syndication and XMLLink. Even if you explicitly set the NavigateUrl properties to your FeedBurner URL, .Text will overwrite it and the link will be rendered as Rss.aspx. If you try to simply remove these HyperLink controls you'll find that (at least with 0.94) .Text will barf. What I did was simply set their Visible property to False. I then added two HyperLink Web controls of my own that referenced the new feed URL.

There's one more facet that should be changed, although I've not made the change since (to my understanding) you'd need to actually hack the .Text source code, recompile, and re-deploy. In the portion of the web pages in your blog you'll find a tag that's used for RSS feed auto-discovery:

Ideally the href attribute would contain the URL of your new feed... but I didn't feel like going through the headache of pecking through the source, making a change, testing, and so forth. So I just left it as-is, figuring in the worst case someone will “discover” my feed to be Rss.aspx, which will automatically be updated to the FeedBurner syndication URL as soon as their aggregator makes its first request to Rss.aspx.

The FeedBurner service looks pretty cool upon first glance. Once this new feed URL gets some use and I get some metrics in FeedBurner's database, I plan on sharing some of the stats... it'll be interesting to see what the most popular aggregator out there is for those who are ASP.NET developers (the primary audience of this blog, I imagine), among other data points.

Searching Your Blog Got a Whole Lot Easier

Scott Mitchell — Tue, 01 Feb 2005 14:54:00 GMT

If you check out the ScottOnWriting.NET homepage you'll now find that in the upper-left hand corner there's a spiffy “Search” box. Go ahead, type a query in there... you'll be redirected to http://scottonwriting.net/Search.aspx?s=ScottOnWriting.NET&count=10&first=1&query=query, which will show you the results of your search.

The new search page - which you, too, can add to your site in a matter of minutes - is possible in large part thanks to Microsoft's new MSN Search. Today Microsoft unveiled their new “built from the ground up” search and one of the cool features it that the search results can be returned as RSS. (To try it out, go to the MSN Search page and enter a query; at the bottom of the Web page you'll find that little, friendly orange RSS button.) Once I found this, I realized adding search to ScottOnWriting.NET would be a cinch if MSN Search let me search for pages on a particular domain. Thankfully, you can search by a specific site by simply adding site:siteName syntax in the search query.

Creating a search engine for a local site, then, is as easy as having a page that requests the remote RSS file based on the query entered by a user, and displaying the resulting RSS in the page. With my free RssFeed control doing exactly this is a no-brainer, requiring just two lines of code. Therefore, the code and markup for the search page for ScottOnWriting.NET basically looks like:

Source Code
// Build up the request URL for the RSS data
string requestUrl = string.Format("http://beta.search.msn.com/results.aspx?q={0}+site%3a{1}&format=rss&first={2}&count={3}", searchQuery, “http://ScottOnWriting.NET/“, startIndex, resultsPerPage);

// Bind the results to the RssFeed control
searchResults.DataSource = requestUrl;
searchResults.DataBind();

Markup
<skm:rssfeed width="80%" id="searchResults" runat="server" Font-Names="Verdana"
Font-Size="Medium" CellPadding="5" ShowHeader="False" GridLines="None">

Some example results can be seen here.

This is pretty cool, but there are some gotchas.

The RSS feed doesn't provide any information if there's more results. There may be 15 results for the query DataGrid, but the RSS feed, by default, will only return the first 10. There's no way of knowing for sure if there are more results unless you grab another RSS feed, this time asking for records starting at 11. What complicates things is that if there are exactly 10 records, getting the RSS feed starting at 11 won't return nothing; rather it will re-return the first 10 records. Ick. My current solution is to just display a Next button if there are 10 records on the page... it's a hack, since it won't really do what is expected for queries that return a multiple of 10 records, but it's good enough.
I don't know how legal all this is. If you look at the RSS feed's <copyright> tag it says:

Copyright © 2005 Microsoft. All rights reserved. These XML results may not be used, reproduced or transmitted in any manner or for any purpose other than rendering MSN Search results within an RSS aggregator for your personal, non-commercial use. Any other use of these results requires express written permission from Microsoft Corporation. By accessing this web page or using these results in any manner whatsoever, you agree to be bound by the foregoing restrictions.

I doubt MS is going to care too much if I use these results for my dinky little blog, but clearly they'd care if I was a company that used this approach to provide search on my Web site in lieu of buying Index Server or Google's searching server, or whatever.

Of course, MSN Search is not the first search engine to provide a programmatic means for accessing their results. Google offers the Google API, a set of Web services that can be used to run queries on Google. In fact, I've written an article on 4Guys on how to create a search engine with the Google API: Searching Google Using the Google Web Service. (And I will likely write up an article on using the MSN Search technique...) The downside of using the Google API is that users are limited to 1,000 queries per day and creating a page to work with the API takes a bit more heavy lifting than just bringing back RSS data and displaying it (although not much more).

A future blog entry, or 4Guys article, will delve into the MSN Search techniques I used to add search to ScottOnWriting.NET, but don't let that stop you from adding this feature to your site today. Armed with RssFeed you could do it without further information in a matter of minutes.

My AggBug Experiment

Scott Mitchell — Wed, 13 Oct 2004 13:58:00 GMT

This blog is powered by Scott Watermasysk's .Text blogging engine, but it's an older version of the software, one that doesn't support statistics. More recent versions of .Text provide a rough estimate as to how many folks have read your blog through the use of AggBugs. Prior to the popularization of syndicated content, determining how many folks read your content was a rather simple task of parsing the Web server's log files. But with syndicated content, examining the number of people who requested the RSS feed does not give a strong estimate. For example, my aggregator might slurp down a particular RSS feed 24 times over the course of the day, yet I may read a particular entry only once or twice, or maybe not at all.

To surmount these problems, AggBugs are commonly used. An AggBug is a tiny bit of HTML in the RSS feed that typically is an <img> tag that requests a 1x1 transparent GIF file. Each time someone loads up a particular entry in their aggregator (or visits the particular blog entry page), a request is made to that GIF file. To determine how many times your blog has been read, you can simply lookup in your Web server's log files to see how many times that GIF has been requested.

Since the version of .Text I use lacks the AggBug feature, I have never really had a good idea as to how many folks read my blog entries. The suspense has been killing me, so I decided to implement by own AggBug feature for a particular blog entry, Sample Chapter of my .NET Web Services DVD. (Rather than using just a simple 1x1 GIF, I created an ASP.NET page that returned a 1x1 GIF, but logged more information than just a simple hit (namely the user's IP, their User-Agent string, and so on.) It's now been 24 hours since that blog entry was posted, and here are the results:

553 unique IP addresses requests
860 total requests
Twice as many people (2) hit my site with Netscape 3 than did with IE 4 (1). Only 2 folks visited with Opera, which is as many people as visited with Safari.
The vast majority (95+%) use IE 6.0 (I believe the popular blog readers - RssBandit, FeedReader, FeedDemon, etc. - use IE as their browser by default, so this makes sense).

It'll be interesting to see how the readership for a particular blog entry shrinks or grows as the days progress. I'd wager something like 80% of the views occur in the first week (maybe two), and the remaining 20% occur over the rest of the year.

The Highest Rated Entries

Scott Mitchell — Fri, 30 Apr 2004 15:54:00 GMT

A couple weeks ago I enhanced my blog so that readers could rate blog entries. I even wrote an article with the requisite SQL queries and User Control code and markup on 4Guys: Creating a Content Reader. I decided to take a few minutes out of my day and display the five highest rated articles. If you visit my blog you'll see on the left-hand side, beneath the Post Categories list, is a Top Rated Entries list. It shows the five highest rated entries that have at least three votes, and uses partial page caching so that it only updates twice per hour.

While whipping out this little blog add-on only took a few minutes, I should have known better that I wouldn't be able to just write code without going hog wild and writing an entire article, which stretched that planned five minute break into an hour long break. In any event, you can learn more at my latest 4Guys article, Improving the Content Rater.

Follow-Up to Allowing Readers to Rate Blog Entries

Scott Mitchell — Mon, 19 Apr 2004 16:27:00 GMT

In an ealier entry I discussed a new feature I added to allow my blog readers to rate individual blog entries, much like one can rate the articles on MSDN. I have written an article on creating the content rater at 4Guys called, aptly, Creating a Content Rater. The article includes the source code for the User Control, along with the stored procedures and table schema I added to my blog (which is running .Text 0.94).

Enhancing Your .Text Blog - Allowing Readers to Rate Blog Entries

Scott Mitchell — Thu, 15 Apr 2004 20:12:00 GMT

One of the things I really like about .Text is just how easy it is to customize and enhance it. Of course, the complete source is available, so you can customize to your heart's content, but I'm talking about customizing it without modifying the source and recompiling. I've had past blog entries about customizing and digging into .Text (see Giving .Text a Calendar View and Analyzing your .Text Blog), and wanted to share my latest enhancement here on ScottOnWriting.NET: the ability for readers to rate a blog entry and leave feedback on why they made their rating (just like how MSDN's online articles have a “Rate this feedback“ section at the end of each article). The end result, you'll agree, looks pretty much just like Microsoft's rating interface, save that mine only allows you to rate from 1 to 5 instead of 1 to 9. It uses cookies to do a half-assed effort at ensuring that folks only rate a blog entry once.

If you're interested in adding this rating User Control to your .Text blog, I'll have the source for the User Control and an article discussing it on 4Guys sometime next week. What is cool about .Text (at least .Text 0.94, the version I'm using) is that the page layouts are specified in User Controls themselves (in the /Skins/skinName/Controls/ directory). This means that I could, with just a quick edit in Notepad, adjust the main page so that after each blog entry in addition to the Feedback (xxx) link, there's also a Rate Entry link, which will take you directly to the interface to link the blog entry.

The natural extension to this would be to allow users to read / view the most popular (and perhaps least liked) posts...

(The Rate Entry option is only available by visiting the ScottOnWriting.NET Web site directly. I guess I could attempt to embed the necessary HTML into the RSS feed, but I doubt I'll do that, since for that (I believe), I will need to edit the .Text source and recompile/redeploy.)

Some More Interesting (To Me) Comment Statistics

Scott Mitchell — Sun, 28 Mar 2004 15:03:00 GMT

After posting yesterday's analysis of comments on my blog, I decided to run a few more statistics. My previous entry examined if there was a correlation between when a blog was posted (either day of the week or hour of the day) and how many comments it received. As the stats showed, there was virtually no correlation between what hour of the day the blog entry was posted and the number of comments, but there was a slight correlation between the day of the week the entry was posted - specifically, entries made on Monday, Tuesday, or Wednesday had a much higher probability of generating more comments than those posted on Thursday, Friday, Saturday, or Sunday. (Andy Smith mused that perhaps I had more interesting things to say on Monday as opposed to Friday; my assumption is that I am equally as likely to say interesting things on any given day of the week.)

Anywho, I decided to examine when, on average, my readers made their comments, both in terms of time of day and day of week. The results aren't too surprising. For the day of the week, people were more likely to post during the week, especially during the middle of the weed (Tuesday, Wednesday, or Thursday):

Day	Total	% of Total
Sunday	28	8.8%
Monday	36	11.4%
Tuesday	67	21.1%
Wednesday	64	20.2%
Thursday	62	19.6%
Friday	41	12.9%
Saturday	19	6.0%
Total	317	100.0%

As for the hour of the day, the most probable posting time was between 8:00 AM and 12:00 PM PST (GMT -8). This correlates to the working hours during most of the US, especially near the lunch hours in all four mainland US time zones. There was also an increase in posting later at night, 9:00 PM and 10:00 PM. I wonder if this was people posting at home, or people in Western Europe posting at their lunch break.

Hour	Total	% of Total
12:00 AM	9	2.8%
1:00 AM	10	3.2%
2:00 AM	6	1.9%
3:00 AM	8	2.5%
4:00 AM	2	0.6%
5:00 AM	13	4.1%
6:00 AM	9	2.8%
7:00 AM	11	3.5%
8:00 AM	18	5.7%
9:00 AM	19	6.0%
10:00 AM	19	6.0%
11:00 AM	21	6.6%
12:00 PM	23	7.3%
1:00 PM	17	5.4%
2:00 PM	12	3.8%
3:00 PM	13	4.1%
4:00 PM	20	6.3%
5:00 PM	12	3.8%
6:00 PM	7	2.2%
7:00 PM	16	5.0%
8:00 PM	8	2.5%
9:00 PM	17	5.4%
10:00 PM	17	5.4%
11:00 PM	10	3.2%
Total	317	100.0%

These stats - the distribution of when comments are made, and when comments are made relative to the date/time the blog entry was made - are not on my blog homepage, ScottOnWriting.NET. They are output cached for 15 minutes or so, but if you are interested you can always get a quick glance at when comments are made here on my blog.

The last statistic I was interested in computing was the delta between when a blog entry was made and when a comment was made. That is, does the average person who leaves a comment make the comment in an hour after the entry has been made? 12 hours? 24 hours? 48? I ran the following query to determine this average delta:

select avg(ParentDateAdded)
from
(select dateadded,
(select datediff(hh, c.dateadded, b.dateadded)
from blog_content c
where c.id = b.parentid) as ParentDateAdded
from blog_content b
where b.blogid=0 and b.posttype=3) as blah

The results? The average time between when a blog entry was made and when a comment was left was 253 hours, or ten and a half days. This number is skewed, though, by the fact that there are a handful of comments that are made to a blog entry several months after the entry was made. If we look at the median delta, it is a more reasonable 33 hours.

With all of this information at our disposal, it's not surprising that Monday, Tuesday, and Wednesday are the days of the week whose blog entries generate the most comments. After all, the average commenter leaves his or her comment a day and a half after the blog entry is made, and most people leave comments on Tuesday, Wednesday, or Thursday, so it follows, naturally, that the best time to make a blog entry would be on Monday, Tuesday, or Wednesday.

So, let me leave you with this observation. Assuming your blog's comments follow the same commenting distribution that mine does, if you enjoy comment discussions on your blog and have something interesting to blog about, make sure you save it til Monday, Tuesday, or Wednesday for maximum comments.

Analyzing Your .Text Blog

Scott Mitchell — Sat, 27 Mar 2004 15:30:00 GMT

I'll be the first to admit it - I love statistics. I like seeing pretty graphs and numbers and sums and averages and calculations and standard deviations and variances and forecasts and trends. Anyway, one thing that annoys me with blogs is that it's impossible to know how many people read it. Yes, there are Web site logs, which say that I had X total Web page requests per day, and Y unique sessions per day, but these numbers, when it comes to blogs, are terribly misleading. The #1 requested page, for instance, is the syndication feed, /rss.aspx. One person, who leaves their aggregator running 24-7, might request this page 12, 24, or 48 times per day. Fine, you say, take the number of requests to the syndication feed and divide by, say, 15, to get a rough estimate on those who read the blog via aggregators. But even that number could be way off. For example, BlogLines might hit my syndication feed 48 times a day, but there could be dozens, hundreds, or even thousands of readers who read the blog from that single requestor.

So I've resigned myself to the fact that I won't ever be able to get a truly accurate guage on my blog readership, but what I can get a handle on is the comments. As the default .Text interface shows, my blog has had - as of the time of writing this - 91 entries and 311 comments. This morning my curiosity got the better of me and I wondered how those comments were distributed. That is, do blog entries that occur on Mondays generate more comments on average than those blog entires made on Friday? Also, when is the best time (relative to my time zone) to make a blog entry? Midnight? In the morning? Around lunchtime? In the afternoon? After dinner? Again, the “best time” I am interested in if posting at a certain time is more likely to generate more comments. (Another metric worth looking at is trackbacks (seeing if the date/time an entry is made increases/decreases the likelihood of folks using trackbacks), but I'll save that anaysis for another day.)

Here are the statistics for my blog, as of March 27, 2004:

Day of Week	Avg. Comments	Total Comments	Blog Entries
Sunday	4.2	21	5
Monday	4.4	44	10
Tuesday	3.2	54	17
Wednesday	4.1	65	16
Thursday	3.7	77	21
Friday	2.3	37	16
Saturday	2.2	13	6
TOTAL:	3.4	311	91

Hour of Day	Avg. Comments	Total Comments	Blog Entries
12:00 AM	7.5	15	2
8:00 AM	0	0.0	1
9:00 AM	7.3	44	6
10:00 AM	3.7	26	7
11:00 AM	2.2	28	13
12:00 PM	4.3	34	8
1:00 PM	3.2	25	8
2:00 PM	3.7	41	11
3:00 PM	2.5	10	4
4:00 PM	0.5	2	4
5:00 PM	1.7	5	3
6:00 PM	3.5	21	6
7:00 PM	0	0	3
8:00 PM	5.2	26	5
9:00 PM	5.5	22	4
10:00 PM	3.7	11	3
11:00 PM	0.3	1	3
TOTAL:	3.4	311	91

The black lines in the graphs are trendlines. As you can see, the later in the week a blog entry is made, the less likely it is to receive comments. However, the hour of the day the entry is posted makes little difference on the number of comments it attracts. Of course, the strongest correlation between a blog entry and the number of comments it receives is likely the quality of the blog entry, but, assuming a "comment-provoking" blog entry is as likely to be made on one day of the week versus any other, the data shows that it's best to make that blog entry earlier in the week than later.

Interested in running these statistical reports on your own .Text blog? If you have access to run queries on the SQL box your .Text blog uses, you can get these reports with just a single SQL statement. To get the comments by the day of week the blog entry was posted, use the following query:

SELECT DayOfWeek, 
       AVG(CONVERT(decimal(5,0),PostCount)) as AvgPosts, 
       SUM(PostCount) as TotalComments, 
       COUNT(*) as BlogEntries
FROM
  (SELECT DATEPART(dw, DateAdded) as DayOfWeek, 
          (SELECT COUNT(*) FROM blog_content bc2 
           WHERE bc2.posttype = 3 AND bc2.blogid = 0 AND
                 bc2.ParentID = bc.ID) as PostCount
   FROM blog_content bc
   WHERE posttype = 1 AND blogid = 0) as SOWBlog
GROUP BY DayOfWeek
WITH ROLLUP

To get the comments by hour the blog entry was posted use:

SELECT HourAdded, 
       AVG(CONVERT(decimal(5,0),PostCount)) as AvgPosts, 
       SUM(PostCount) as TotalComments, 
       COUNT(*) as BlogEntries
FROM
  (SELECT DATEPART(hh, DateAdded) as HourAdded, 
          (SELECT COUNT(*) FROM blog_content bc2 
           WHERE bc2.posttype = 3 AND bc2.blogid = 0 AND
                 bc2.ParentID = bc.ID) as PostCount
   FROM blog_content bc
   WHERE posttype = 1 AND blogid = 0) as SOWBlog
GROUP BY HourAdded
WITH ROLLUP

If enough people run (and publish) these queries, and the day of the week / hour of the day are standardized to some standard time zone, like GMT, we could determine, globally, when the best time to make a blog entry was for maximum commenting. :-) Another interesting statistic to help ascertain when folks were reading your blog would be to determine the average number of comments made per given day of the week / per given hour of the day. (Recall that the above statistics look at the comment count based on the day of the week / hour of the day the blog entry was made.)

Giving .Text a Calendar View

Scott Mitchell — Fri, 13 Feb 2004 02:13:00 GMT

Many blog engines provide a calendar view when visiting the blog via the Web. Typically the calendar shows the current month, with days that have one or more entries rendered as hyperlinks. Clicking on the link for a given day displays the posts for that day. Unfortunately .Text doesn't provide such a built-in calendar view, so I decided to create my own. (As to why .Text doesn't include a calendar view, Scott Watermasysk indicated that there were issues with the ASP.NET Calendar control and CSS. Not being a CSS expert, I'm not sure exactly what issues there are, but in my own experiment I'm happy with the rendered display of my calendar, and all stylesheets seem to be playing nicely together...)

In any event, take a minute to check out the Scott on Writing homepage - up in the upper-right hand corner you can see a calendar view. Pretty cool, eh? I'd like to share how I accomplished this.

DISCLAIMER: To add the Calendar control I decided to take a path of least resistance, and actually had the entire thing written in under ten minutes. I used a user control - a compiled control might be a better choice. There may be more efficient techniques to use than I employed. If you have suggestions on improvements, feel free to post them; if you want to use the code, feel free, but realize it was created hastily with a “let's get something working before my fiancee gets home from work”-attitude.

TECHNICAL NOTE: I am using .Text 0.94 - I have no idea if this code will work “as-is“ with different versions. I imagine it would, although if the database schema has changed, that would require a slight update of the SQL query used to get the blog entries for a given month/year....

I started by creating a User Control and adding a Calendar. My aim was to create an event handler for the Calendar's DayRender event, and then see if the particular day being rendered had any posts and, if so, display a hyperlink to the correct /archive/ URL. (To view the posts on MM/DD/YYYY, you can simply visit /archive/MMDDYYYY.aspx - ah, the power of HTTP handlers.)

Of course, I didn't want to have to hit the database once for each day in the month, so I loaded all of the blog entries from the month into a DataTable (ordered by the date of entry, in ascending order). The SQL query to run (which was moved into a stored procedure) was:

SELECT DateAdded FROM blog_Content (nolock)
WHERE PostType=1 
      AND BlogID = @BlogID
      AND DATEPART(yyyy, DateAdded) = @Year 
      AND DATEPART(m, DateAdded) = @Month
ORDER BY DateAdded

Here, @BlogID is the ID of the blog whose dates of posts you want to show (if you are hosting multiple blogs on a single .Text install, each blog has its own BlogID). @Year and @Month are the year and month of blog entries to get. (The PostType=1 gets only blog entries - not comments, tracebacks, etc.)

In my User Control I have a DataTable (named dates) scoped at the class level that is populated with the results of the above SQL query. The code that performs this follows:

private void LoadMonthData()
{
   SqlConnection myConnection = new SqlConnection(connection string);

   SqlCommand myCommand = new SqlCommand("blog_GetBlogEntriesForMonth", myConnection);
   myCommand.CommandType = CommandType.StoredProcedure

   // selDate is the currently selected date...
   myCommand.Parameters.Add(new SqlParameter("@Year", selDate.Year));
   myCommand.Parameters.Add(new SqlParameter("@Month", selDate.Month));

   SqlParameter blogID = new SqlParameter();
   blogID.Value = 0;
   blogID.ParameterName = "@BlogID";
   myCommand.Parameters.Add(blogID);

   // Fill the dates DataTable...
   SqlDataAdapter myAdapter = new SqlDataAdapter(myCommand);
   myAdapter.Fill(dates);

   myConnection.Close();

   // Keep track of how many records are in the DataTable...
   dateCount = dates.Rows.Count;
}

In addition to having the dates DataTable scoped at class level, I also have two additional ints: one keeps track of how many total entries there are for the month (dateCount), the other stored the index of the current record being examined in dates (this variable is named currentDate).

In the event handler for the Calendar control's DayRender event, I check to see if the current date being rendered equals the date in the current row in dates being examined. If it does, I change the TableCell's Text property to a hyperlink and update currentDate accordingly - otherwise I do nothing. The code for this event handler follows:

private void dayRender(object sender, DayRenderEventArgs e)
{
   if (currentDate < dateCount && 
         e.Day.Date == Convert.ToDateTime(dates.Rows[currentDate]["DateAdded"]).Date)
   {
      e.Cell.Text = "<A href="\" +="" ?? archive sowBlog>
               String.Format("{0:d2}{1:d2}{2:d4}",
                     e.Day.Date.Month, e.Day.Date.Day, e.Day.Date.Year) + 
                     ".aspx\"><U>" + e.Day.Date.Day + "</U></A>";
      // There might be multiple entries per day, so advance currentDate until
      // we're on a different day...
      do
      {
         currentDate++;
      } while ((currentDate < dateCount && 
            e.Day.Date == Convert.ToDateTime(dates.Rows[currentDate]["DateAdded"]).Date));
   }
}

There are some other slight details involved, but the above description spells out the bulk of the work that needed to be done. In order to insert the calendar view I edited the PageTemplate.ascx User Control for my blog's appropriate skin. I also had to do a bit of tweaking with the #leftmenu, #main, and #rightmenu CSS items to get the calendar to fit without forcing the user to have to scroll off to the right.

If you are interested, you download the complete source code for the Calendar User Control here.