Scott On Writing.NET : Blogs, RSS, Web 2.0

Creating and Consuming Syndication Feeds in ASP.NET 3.5

Scott Mitchell — Mon, 23 Feb 2009 12:21:00 GMT

Do you run a website that routinely publishes new content? If so, does your site offer a syndication feed? A syndication feed is an XML file that summarizes the most recently published content and is commonly used in blogs, news sites, sports sites, social networking sites, and other content producing websites to provide a machine-readable format of the latest content. For the longest time the .NET Framework did not include built-in support for syndication feeds, meaning that if you needed to generate a syndication feed or parse the syndication feed from another site you either needed to:

Write the code yourself, as demonstrated in this article of mine from March 2003: Syndicate Your Website's Content with RSS, or
Use an open-source or third-party product, such as my open-source project, RssFeed, or Microsoft's ASP.NET RSS Toolkit.

The good news is that syndication functionality was added to the .NET Framework version 3.5, namely with the addition of the classes in the System.ServiceModel.Syndication namespace. This namespace includes a classes that model a syndication feed and the items that compose the feed, along with classes for formatting a syndication feed into either RSS 2.0- or Atom 1.0-compliant XML. (RSS 2.0 and Atom 1.0 are the two leading syndication feed format standards.)

For more information, check out my latest article on DotNetSlackers.com, which looks at creating and consuming syndication feeds using these new classes in the .NET Framework 3.5 from an ASP.NET application: How to Create a Syndication Feed For Your Website.

Happy Programming!

Blogs are the Most Important Innovation of the 21st Century

Scott Mitchell — Fri, 13 Feb 2009 13:35:00 GMT

A case could be made that blogs and blog hosting websites like WordPress and Blogger are, so far, the most important innovation of this century, technological or otherwise. Prior to the advent of blogs, publishing online was a task reserved for the computer savvy. Sure, sites like Geocities and Tripod lowered the barrier, but still required a good deal of work and at least a passing familiarity with HTML. It wasn't until blogs that the average person - my mom, my grandfather, etc. - could, with a few points and clicks of the mouse, get their thoughts, feelings, and information out to an audience of, potentially, billions. In mathematics there is the concept of different cardinalities of infinity. While both the set of integers and the set of real numbers are inifinite sets, there are 'more' items in the set of real numbers. It's a 'bigger' infinity, if you will. In a similar vein, both e-mail and blogs shrink the world and allow people who may be vast distances apart to communicate with one another instantly, but blogs go a step beyond e-mail (or the telephone or fax machines) in that it allows for one-to-many communications. With an e-mail message you can send urban legends and corny jokes to your aunt in Baltimore, but with your blog you can share them with the world at large.

Blogs have had a very profound affect on society as a whole. They have democratized the publishing process and made the idea of citizen journalism a reality and, in doing so, they have quickly neutered industries that were flourishing a mere 10 years ago. Why buy a newspaper when you can get your information online? Why pick up a copy of Sports Illustrated when you can read more interesting and in-depth articles written by ex-GMs, current athelets, and impassioned fans? And what's the motivation to purchase a programming book when there are countless developers sharing their tips and tricks, what works and what doesn't, on their blogs each day? Of course, there are benefits to be had by big box newspaper and magazines and having content presented on pieces of paper. Print media is not dead, but it has been relegated to a niche market, and blogs are a big reason why.

Blogs have also had a profound impact on the Web 2.0 landscape, namely in the areana of syndication. Ten years ago content syndication was in its infancy. Blogs were the first application that used syndication - Dave Winer invented RSS in 1997 to syndicate content from his blog, Scripting News. As the popularity of blogging grew, the RSS standard matured, the Atom standard was created, and non-blog sites started embracing syndication. Today, virtually every website that produces content on a regular or semi-regular schedule provides a syndication feed. We have blogs to thank for that development.

It may seem like an over-exaggeration to say that blogs have had such a profound impact, but consider this - how do you think my daughter will react when I tell her, 15 years hence, that there was a time in my life when only people who were 'computer experts' could publish content online. I think she'll be flabbergasted, and I think that surprise and disbelief she'll have is a testament to how important an innovation blogging has been.

What Do You Use to Read / Consume Blogs, News Sites, and Other RSS Feeds?

Scott Mitchell — Mon, 09 Jun 2008 16:39:00 GMT

When I first started blogging and reading others' blogs, I tried out two stand-alone desktop applications for keeping up to date with my favorite bloggers and other news sites:

RssBandit - an open-source application created in C# by Microsoftie Dare Obasanjo.
FeedDemon - an application created by Nick Bradbury, who also authored the notable HTML and CSS editors, HomeSite and TopStyle.

While both RssBandit and FeedDemon have slick UIs and are easy to use, I haven't used either for several years. My main gripe was not with the programs, per se, but the model itself: I didn't like having a separate program for reading blog entries. For starters, it wedded my blog subscriptions to a single computer. Second, it meant yet another program I'd have to launch at startup and yet another icon cluttering up my task tray. Back in October 2005 I wrote a blog entry lamenting stand-along blog readers: The Future of Third-Party Offline Aggregators? Are RssBandit and its Kin Dead Weight?

There are a number of popular offline aggregators available. By 'offline' I mean that these aggregators can be used while not connected to the Internet. ... The future of aggregators, in my opinion, are those that are either online ... or are part of the experience of existing 'everyman' applications (i.e., email or web browsing) and, preferrably, are preinstalled with the software. The online aggregators seem to make a lot more sense, having a number of advantages of their offline kin:

Not bound to a particular computer - I can be at home, at the office, or on vacation - my subscriptions travel with me.

Can utilize the 'social network' - services like Findory make it easy for me to get recommended news and blog items based on my clickthroughs. Services like del.icio.us allow me to share my online habits/sites/subscriptions with others with like interests. I can see what the most popular feeds are, or explore the subscriptions of those whose interests match mine.

Easier to 'install' and 'uninstall' - want to install My Yahoo! on your computer? Fire up the ol' browser and enter http://my.yahoo.com - couldn't be easier. And uninstalling's as easy as not visiting the site again.

No resource consumption - doesn't matter if I subscribe to one feed or a hundred - the disk space and bandwidth consumed on my computer stay constant when using an online service.

Another advantage of online blog readers (or any online application, for that matter) is uniquitous upgrades. When Microsoft releases a new version of Office, it is applied only to those peoples' computers who buy the upgrade and install it. When Microsoft releases a new version of Hotmail, however, the update is applied to all users instantaneously. This leads to more rapid application updates, features, and bug fixes.

Since my blog post in 2005, we have seen better integration of RSS feed support in the 'everyman' applications. Both IE and Firefox have RSS subscription capabilities (albeit rather primitive support), as does Outlook 2007. And virtually every online portal website has the ability for users to subscribe to RSS feeds. Third-party offline blog readers are always going to be at the far end of the long tail, especially with the commodity-like status of RSS aggregators these days. I don't think third-party offline readers will every necessarily die off, but they will be used only by a select and small crowd of experienced computer enthusiasts who prefer them over more mainstream or online options for some very specific reasons. And, for most people, those benefits, whatever they may be, are not strong enough to outweight the cost of downloading the application, installing it, setting it up, and learning how to use it.

I'm curious - what do you use to consume blogs and other RSS feeds? Do you use a stand-alone program, or something that's integrated with Outlook? Do you use an online service?

These days, I use Google Reader to subscribe to and keep up to date with the myriad of blogs, news sites, sport sites, and online comics I follow. Google Reader gives me one spot - accessible anywhere in the world - where I can catch up on and manage my RSS subscriptions. Google Reader also has the early stages of social networking baked in. You can share particular blog items and see your friends' shared items. And Google Reader can offer recommendations on feeds you may like based on what feeds people with similar interests have subscribed to.

Supporting HTTP Authentication and Forms Authentication in a Single ASP.NET Web Site

Scott Mitchell — Fri, 23 Jun 2006 14:24:00 GMT

Authenticating users visiting an Internet-based website is usually accomplished in one of two ways:

Using Forms authentication - here a user is prompted for their credentials through a standard web page form
Using HTTP authentication - RFC 2617 defines two standard HTTP protocol-level authentication schemes - Basic authentication and Digest authentication. When a browser interfaces with a site using this form of authentication, it displays that familiar modal dialog box prompting you for your credentials (see this image).

Many websites use Forms authentication, as it provides more control over the user experience and many platforms (such as ASP.NET) include tools for adding Forms authentication support. (In fact, with ASP.NET 2.0, you can build a complete user account-based website using Membership and Forms authentication without having to write a line of code; see Examining ASP.NET 2.0's Membership, Roles, and Profile for more information.)

However, Forms authentication is not a standard; while it's easy for a human to provide their credentials through Forms authentication, it's much more difficult for a computer program since the HTML inputs for the credentials can differ from site to site. With the standard HTTP protocol-level authentication schemes, however, the workflow for authentication is much more mechanical and can easily be supported by a computer program. In fact, tools like wget, Microsoft's Background Intelligent Transfer Service (BITS), and many RSS readers support built-in HTTP authentication.

Traditionally, Forms authentication is handled at the ASP.NET level, while the HTTP authentication schemes are configured from the web server level. This means that Forms authentication can easily be configured to authenticate users against a credential store located in some XML file, a database table... whatever. With HTTP authentication, the user credential store consulted for authentication is typically the Windows User Store.

Consider the case of a blog or forum site, where we have certain resources that we want to protect. For example, imagine that at a forum site there is a forum that is “private,“ meaning that only a set of users can view the forum's posts. It's easy to establish authorization rules in Web.config and use Forms authentication to identify users. However, imagine that our forum website offers RSS feeds of the forum's most recent posts. How do we share the RSS feeds for the private forums? RSS readers can't authenticate via Forms authentication.

One option, which is what is used by CommunityServer, is to use a unique RSS subscription URL per user. The only issue here is that if that particular URL is leaked somehow, anyone can consume the private forum feeds. Rob Howard and I discuss this feature in CommunityServer in this forum post: Enable RSS on non-public forums. Another option is to support both HTTP and Forms authentication and have it so that they both use the same credential store (such as that database table or XML file or what have you...)

Back in April 2006, Atif Aziz and I published an article on MSDN Online on how to build an ASP.NET web application that can do just this. In particular, the article examines MADAM, a collection of HTTP Modules for handling this sort of functionality. With MADAM you can define in Web.config what conditions would require the authentication scheme to switch from Forms authentication to HTTP authentication. For example, you might instruct MADAM that whenever anyone was attempting to access a particular resource, to switch to HTTP authentication. The MADAM HTTP Modules then detect when such a condition unfolds and intercept the HTTP response to include the necessary headers for the specified HTTP authentication scheme. Similarly, when the user agent responds with the user's credentials, the MADAM HTTP Modules work to validate the credentials against the configured credential store.

If you've ever needed to meld Forms authentication and HTTP authentication in a single web application using a single credential store, check out MADAM. You can learn more about it and download the complete source code at Supporting HTTP Authentication and Forms Authentication in a Single ASP.NET Web Site.

Free, Focused ASP.NET Headlines - Consolidated Your Way!

Scott Mitchell — Tue, 03 Jan 2006 15:54:00 GMT

Ever since creating RssFeed (an open source ASP.NET control designed to display content from an RSS feed in an ASP.NET web page), I've used the control to display the latest headlines from www.asp.net on the 4Guys ASP.NET homepage. About a month ago fellow MVP Sonu Kapoor dropped me a line and we discussed integrating feeds from his site - DotNetSlackers - onto the 4Guys homepage. DotNetSlackers slurps ASP.NET-related websites and blogs from around the Internet and provides a consolidated feed along with category-specific feeds. Initially I used Sonu's ASP.NET category feed on 4Guys, but, since the feed included blogs, found there to be too many off-topic posts (i.e., headlines not focused strictly on ASP.NET content). Furthermore, the feed URLs point back to a summary on DotNetSlackers rather than to the actual syndicated item.

After brainstorming a bit, Sonu came up with a neat solution. Since he already was culling the Internet grabbing various feeds from various focused websites, why not create a page where visitors could say, “I am interested in feeds X, Y, and Z,” and then have DotNetSlackers provide a single feed that consolidated those feeds? For example, you could say, “I am interested in 4GuysFromRolla.com's latest articles, ASPAlliance.com's latest articles, and K. Scott Allen's excellent blog.” After selecting these three blogs, DotNetSlackers would create a single feed for you that you could display on your website or add to your feed reader. Not only does this free service provide a way to selectively consolidate various .NET-focused feeds, but Sonu is kind enough to link directly to the syndicated content. (Sonu does ask that if you do use this combine feed service on your website that you do give a plug to his site, DotNetSlackers.com.)

You can check out the feed building service over at http://www.dotnetslackers.com/CombineFeeds.aspx

Google Base - Search Crap Easier!

Scott Mitchell — Wed, 26 Oct 2005 18:41:00 GMT

The 'blogosphere' is abuzz with Google's latest pre-pre-Alpha release, Google Base (the first pre- is in there because the product isn't even released yet, save for http://base.google.com, which is up sometimes, down othertimes, and doesn't really do anything other than generate a fevered pitch among bloggers). Google Base. From a good ArsTechnica article on this pre-pre-Alpha service, “ Google Base is Google's database into which you can add all types of content. We'll host your content and make it searchable online for free.” In a nutshell, purportedly this service will allow Sally Housecoat and Joe Meatball to upload and add their own content to Google Base, which is sort of going to be this online database.

To me, it appears as if the aim of Google Base is to be that big ol' database in the sky. “Have some data? Go ahead and stick up in that cloud up there. Need to search that data? Here are the web service APIs, knock yourself out.” Users will be able to tag their uploaded information with labels, geographic information, and other categories, and the results, one would imagine, would be integrated into other Google services, such as Google Maps, Froogle, Google Local, and so on. (For example, one could say, “Show me all garage sales going on in my zip code in the next two weeks.”)

Some are calling it the craigslist and eBay killer; some see this as the potential end of classified ads in newspapers.

I am not so excited about this product for a couple of reasons. First, it's pre-pre-Alpha... If this is at all like the Google Reader released a month ago or so, then even when Google Base is officially released, it'll still need a lot of work. Second, and more importantly, they're allowing users to add to this database. If they just let any ol' person add any old thing, the quality of Google Base will quickly approach zero. Look at USENET - except in moderated or very specific forums visited only by a small sect of people, a large percentage of the stuff posted there is crap. Google Base will need some sort of moderation or community involvement that will keep this data pure. And how many people are going to keep using Google Base when they do a search for garage sales in their area and show up only to find that they moved it to next weekend, but forgot to update Google Base?

Let's just say I'm pretty pessimistic when it comes to any service that basically trusts the general public to add to their catalog, and I'd hope Google would know this better than anyone else... a-hem. Let's call this the Scott Mitchell theorem: The quality of any piece of information is inversely proportional to how many people contribute to it.”

The Future of Third-Party Offline Aggregators? Are RssBandit and its Kin Dead Weight?

Scott Mitchell — Wed, 05 Oct 2005 22:58:00 GMT

There are a number of popular offline aggregators available. By 'offline' I mean that these aggregators can be used while not connected to the Internet. For example, my aggregator of choice is RssBandit, which is an offline aggregator. There's also the likes of FeedDemon, SharpReader, and a whole slew of other choices. But what does the future hold for such offline aggregators?

The future of aggregators, in my opinion, are those that are either online - My Yahoo!, BlogLines, Start.com, Google's personalized homepage, Findory, Rojo, and so on - or are part of the experience of existing 'everyman' applications (i.e., email or web browsing) and, preferrably, are preinstalled with the software. The online aggregators seem to make a lot more sense, having a number of advantages of their offline kin:

Not bound to a particular computer - I can be at home, at the office, or on vacation - my subscriptions travel with me.
Can utilize the 'social network' - services like Findory make it easy for me to get recommended news and blog items based on my clickthroughs. Services like del.icio.us allow me to share my online habits/sites/subscriptions with others with like interests. I can see what the most popular feeds are, or explore the subscriptions of those whose interests match mine.
Easier to 'install' and 'uninstall' - want to install My Yahoo! on your computer? Fire up the ol' browser and enter http://my.yahoo.com - couldn't be easier. And uninstalling's as easy as not visiting the site again.
No resource consumption - doesn't matter if I subscribe to one feed or a hundred - the disk space and bandwidth consumed on my computer stay constant when using an online service.

Of course the major disadvantage for online aggregators is that they require the user to be online. While broadband is becoming more ubiquitous, it's not universal, so those who can only get online in bursts, will, obviously, enjoy offline aggregators, as they can download the content while online and the peruse offline. (Similar to the benefits of USENET over online forums.) But these third-party aggregators are going to be crowded out of the marketplace once this feature becomes standard in email/news clients. When Outlook Express makes it a cinch to subscribe to RSS feeds and view the feeds offline, what point is there for SharpReader or any other offline reader?

Granted, these third-party apps can provide new features with a much quicker release schedule than Microsoft or any other large software company, but who's going to use them other than just a fringe population of super-geeks? I like RssBandit. I still use RssBandit. But I have a hard time seeing RssBandit (or any other offline aggregators) having much relevance in the aggregator space in the near future. I'm honestly close to just switching over permanently to online aggregators.

Am I mistaken here? There are some applications that are better suited for the web, some that are better suited for the desktop, and some that have their place both on the web and on the desktop. I think the only place aggregators have on the desktop is for offline access, and I don't see space for offline players outside of, perhaps, an offering from Microsoft and an offering from one other competitor. I mean, how many people do you know that don't use a Microsoft product for offline email access? I wish the best for today's third-party aggregators, but can't see many (if any) of them having any sort of non-trivial install base a few years out.

Google Video, Personalized Search, and MSAdSense - Oh My!

Scott Mitchell — Tue, 27 Sep 2005 01:32:00 GMT

One of my favorite attributes of the web is the abbreviated length of the software cycle. With desktop software there's this whole mess of burning bits to a CD and shipping them to some physical location, which requires that users actually take the time out to go to said store. Even for desktop software distributed via the Internet, there's still platform limitations and bulky downloads. All of these factors that impede deployment and adoption of desktop applications are removed with web-based apps. It seems like almost once a week that there's some news coming out of Yahoo!, MSN, or (more often than not) Google.

Case in point, a slew of “search site” news came rolling out over the past few days. One of the coolest is Google Video's commercial-free airing of Everybody Hates Chris, a new comedy series on UPN about Chris Rock's childhood. Furthermore, Google Video, which used to use a program one needed to download and therefore limited it to playing only on Windows boxes, now operates as a Flash movie in the browser. That means cross-browser and cross-platform support, as well as the removed nuissance of having to download a program. Also, the Flash movie automatically resizes to your browser's resolution, automatically buffers the video and (on my DSL connection) starts playing nearly instantly. You can also seamlessly jump around the stream with little to no delay. Nice!

If that's not enough, Google also recently improved their Personalized Search feature. When logged in and searching, each search result has a “Remove this Result” link that you can click to optionally filter out search results for your particular search phrase, for all search phrases, or, heck, you can remove the entire domain. A useful tool for removing splogs and other spammy sites that clutter up the results page with useless information. This information is not only helpful in cleaning up your own results, but the information will likely be aggregated and used by Google to help improve their search engine's heuristics.

Finally, word is out that Microsoft will be releasing their own AdSense-like program. This program, called adCenter, has been in beta since March of this year, although only available in France and Singapore. However, the beta will, I expect, be opening up in the US sooner than later, seeing as there are a slew of adCenterp-related articles on all the tech news sites. It'll be interesting to see how Microsoft fares in this market, which is already cornered by the likes of Google and Yahoo. More competition is always better, of course, and hopefully Microsoft can introduce some innovative tools and metrics that force Google and Yahoo to increase their functionality and featureset. Viva la capitalism! :-)\

For a computer nerd who likes trying out new programs, it really is an exciting time we live in. As one who doesn't have cable and will, hopefully, never need/get it, I'm looking forward to a day when the computer can more effectively replace the television. Seeing Google Video's advances and capabilities in airing a sitcom makes me think we're that much closer.

Google Blog Search

Scott Mitchell — Fri, 16 Sep 2005 13:48:00 GMT

Google's added yet another beta product to their lineup, this time it's the Google Blog Search. This new search service competes directly with other blog search engines, such as IceRocket and Technorati. I've been using IceRocket to search the “blogosphere” as of late, and they seem to have very few splog sites in their results. Furthermore, IceRocket sorts the results chronologically (whereas Google sorts by relevancy by default) and has neat little tools like the Blog Trends Tool. However, I do find that IceRocket's response time can be a bit slow at times; that is, doing a search or going to the next page of results might take a couple seconds, whereas with Google it's instantaneous. Both IceRocket and Google Blog Search provide an RSS (or Atom) feed of the search results.

What's most disappointing with Google Blog Search (and IceRocket, to a lesser degree) is the predominance of splog entries. If you do a search on anything remotely spammy - lasik, cialis, texas holdem, etc. - the majority of the results are going to be splog sites. Mark Cuban points the finger at Blogger.com in his post A splog here, a splog there, pretty soon it ads up... and we all lose:

What makes the problem particularly frustrating is that it doesn’t cost anything to setup a blog on what is probably the most common blog host, blogger.com from Google. It’s fast, its easy, it’s free and it can be automated. [Note from Scott: you can make a new blog entry in Blogger.com by simply sending an email message to a specified address...] So blogs are coming at us left and right. We are killing off thousands a day, but they keep on coming. Like Zombies. It’s straight from Night of the Living Dead. Brain dead splogs. Coming at us by the thousands.

Blogger is by far the worst offender. Google seems to be working hard to adjust their relevancy indexes to exclude splog from having influence on search rankings, but they don’t seem to be doing anything more than removing reported splogs. Kind of like going after the zombies one at a time with a shovel. Can we get some help on this Google?

Keep in mind that Mark is one of the owners/investors in IceRocket... Speaking of Mark Cuban, he also has a great entry on Google's Blog Search as well, comparing it to IceRocket and listing the major concerns he finds with Google's latest offer: Welcome to the show Google BlogSearch.

Hopefully Google will figure out a good compromise, one that eliminates the vast, vast majority of splog sites but that doesn't nullify any (or many) legit sites.

There are also a number of features currently missing from Google's Blog Search that will, I'm certain, be added eventually. Some of these include:

No integration with Google Search History
No “Blogs” tab atop the Google search results (akin to the Images, News, Groups headings)
No way to submit my blog's RSS/Atom feed. According to Google Blog Search Help, “If your blog publishes a site feed in any format and automatically pings an updating service (such as Weblogs.com), we should be able to find and list it. Also, we will soon be providing a form that you can use to manually add your blog to our index, in case we haven't picked it up automatically. Stay tuned for more information on this.”
No means of categorization. It would be nice to be able to drill into blogs by topic rather than just having to do a keyword search.
Lack of meta-statistics on the blogosphere. A buzz index like IceRocket provides or other metadata that can be gleamed from Google's massive index would be most appreciated.