This post is to announce that the new version of Newsfeeds has actually passed all stages of testing last Thursday. This is a bit of a champagne moment as it marks the end of a very long process. Initially another approach was taken by the former team lead and after a lot of consideration I decided to discontinue that code branch. Instead the current upgrade is almost a complete rewrite of the module incorporating what I believe to be the most important features that were still missing: aggregation, feed format flexibility and caching.
The new module uses a new (2.0) and adapted version of the RssToolkit as posted on CodePlex. This component is able to recursively (OPML is also supported) aggregate feeds of various formats into a single Rss 2.0 feed. We tried using the RssToolkit that comes with the DNN installation but it was not recent enough nor adapted to our task. So the RssToolkit code was incorporated into the module and adjusted to fit its needs. It is unlikely that this situation will stay like this forever. In all likeliness we will either go solo on aggregation and/or move the RssToolkit code we have to the DNN core. But for now it is in the module (and hence also translated to VB thanks to SharpDevelop).
The new aggregation does bring one drawback: the module is less forgiving about feeds that do not conform to one of the standards Rss 0.91, Rdf, Rss 2.0, ATOM 1.0. The aggregator (i.e. the code that merges the feeds) needs to read the contents of the feed and merge this into one stream of items. This means that the feed must adhere to its standard or the aggregator will not be able to find the content where it expects. I have already received several feed urls from big news corporations that did not conform to its standard. This is somewhat disturbing. If you experience this the only option you have is to use your own transformation and use the XML module. It will not pass through the aggregator.
One other note on aggregation. Merging multiple feeds really only makes sense if the items are dated. This is only the case in Rss 0.91, 2.0, and ATOM 1.0 feeds. Rdf and Rss 1.0 feeds do not require a date/time stamp on individual items. So aggregating feeds that use those standards is kind of nonsensical. For now the module will just interleave them one by one.
DNN has its own caching mechanism for modules. In short the HTML of the module is cached for a specific timeframe. During this timeframe, the module code will therefore not even be called. Instead the HTML is injected directly into the output stream from cache. This is all good and well, but it does mean that the Newsfeeds module will not be able to check for any updates of feeds it needs to make. So instead the new module now has its own solution of ‘multilayered’ caching. If it can render immediately it will do the XML/XSL transformation. This means it will be slightly outperformed by the HTML caching the DNN platform would do. But the advantage of the new solution is that we can fine tune what and how we cache. It does however rely on your setting of the module cache time in the module settings. If you leave that at non zero, it will override the mechanism we provide in the module.
The new mechanism caches at two levels: the feed and the aggregated feed. It will cache all incoming feeds in SQL. This will prevent it to trip up on not being able to retrieve a particular feed. For each feed you can specify a refresh time. When called, the module will check if any feeds need to be refreshed and will get them if necessary. The aggregated feed is stored as XML on disk. This will also make it possible to reuse the feed (not yet implemented as we’re waiting on the core team to make a move in regards to an overhaul of syndication in the platform). The new approach in the Newsfeeds module should ensure we have all the caching we need for any future developments.
Another change is the way in which the merged feed is rendered. This used to be a straightforward XML/XSL transformation using the standard ASP.NET XML component. Now, we use the same mechanism as the XML module. This means we transform in code using a parameterized stylesheet. This allows us to pass in variables like ‘number of items to display’, etc. These parameters are set under module settings. This solves one of those pressing issues we had: how to limit the amount of items displayed on screen. In the forums you could find the solution to this using a non-default XSL sheet. Now you can specify it as a parameter under module settings.
One oft heard complaint was that a DNN page with the Newsfeeds module would not render until the feed had been retrieved. This I felt was a very urgent issue to solve. In my opinion the ideal situation would be that the page would render before any feeds were being retrieved by the server. That way the user would see the page immediately. The feed result would then be passed through Ajax to fill the module on the client.
The new module does just this. It first performs a check whether Ajax should be used. This depends on:
- whether it is installed
- if the module is not being cached by DNN as one piece of HTML, and
- if the module needs to refresh one of its feeds.
If all these conditions are met then the module will render an empty container that calls back after the page has loaded to retrieve its contents. Then, the necessary feeds are retrieved cached, and merged. The result is sent to the waiting module. You’ll see the DNN Ajax loading gif displayed in the meantime.
I just want to end with a word or two about security and RSS. A feed is sent in plain XML from a source to the module upon request. This source can be an internal page. If this is so the module will call this module (for the RSS) within the current context (i.e. the current user’s login). Assuming you cache the feed this may lead to the following security breach: if you cache a feed from content that is not visible to all, you potentially will show the cached content to someone with a different security clearance. Keep this in mind as you set up caching and security in your portal.
Newsfeeds 04.00.00 was compiled against DotNetNuke 04.06.00. So this is the minimal version you need for the module to work. The module now uses the database so you’ll also need to be running on SQL 2000/2005.
Find the module here:
All comments/questions/etc can be asked here: