Experiences from the fields
Last week I blogged about the new release of DNN Events: version 05.02.03,
which replaces the previous one (05.02.02) and is mainly a bug fix release.
Currently,
we (Roger Selwyn and myself) have worked on 2 versions of Events: a bug fix release (05.02.03)
and a new version (Events 6) for the DNN6 environment. And there is a story to
go with it.
The start
The 05.02.03 was not planned, but came
around when some small, but annoying, bugs popped up around the 05.02.02
release. About 3 weeks ago, Roger and I decided to do an intermediate, unplanned,
bug fix release: 05.02.03. The problem was that we use CodePlex/TFS for our code repository and
we have moved everything already to our
alpha and published Events 6
release.
So we started again with the source pack of 05.02.02 and worked
disconnected from CodePlex TFS for our
sources (Note: we used DropBox to share our coding for this intermediate version). Roger
did work on the bug fixes and also back ported some of the changes and enhancements from our
Events 6 release
back to 05.02.03. I, in the mean time, worked on the SQL data scripts and
made some enhancements for our automatic build and packaging tool to get it
compliant for DNN5 installs. We have made a beta version of our bug fix release
available on CodePlex, saw the download counter reach 40-50 download without
any problematic feedback. So far so good.
We prepared our final Events 05.02.03 release and replaced the beta version
on CodePlex. With confidence I changed the 05.02.03 version into a stable, preferred
release of DNN Events.
The results: not a happy event
It did not take long before problems were reported: wrong DLL versioning.
While Roger (who made the finale 05.02.03) was out, I created a new "stable"
05.02.03 and replaced the current one on CodePlex. Again, not correct. Due to
the Events 6 (DNN6 related) version in the works, some of the data script changes
has slipped back into my "stable" version. Sh*t. Luckily, Roger was able to
create a finale stable version of DNN Events 05.02.03 and also prepared a SQL
repair script for the a few people around who suffer from our intermediate
"work" (and do not do back-up's or testing before going live).
How did we do it before?
Release tracker - In the previous years, there was the DNN Release tracker process
(which we did not like). Any
core module release could take weeks if not months before it was formally released.
However, the release tracker process would have saved us this time from the
errors that we have made.
Special Events website - At the start of my involvement with the Events module (2007), we had a
special website for distributing Events beta releases to users with real
interest in the module. Users, who also gave us real feedback on any problems in
the beta version. The current CodePlex beta downloaders do not give us feedback
on beta version of the module.
Automated UI testing - At some point in time, we created automated test scripts
with DejaClick,
a FireFox AddIn with screen recording, replay and script editing facilities.
This helped us to execute tenths of user scenario's automatically (creating
events, editing settings, changing display formats, enrollments, approval,
etc.). However, due to the dynamic nature of DotNetNuke (controls do not have
fixed name but are dynamically generated) and the frequent releases of the core
platform quickly made the management and editing of the tests scripts a daunting
task. Back to manual testing which developers -and that includes us- do not
really like to do over and over again.
My conclusion - In Dutch (my native language) I would
say: "Wet van behoud van ellende", which translates to something like "There is
no such thing as a free lunch". We have not reached a level of getting enough
feedback on the quality of our new version; if others test, it takes much too
long before we can release; we are not able to execute easy maintainable
automatic UI tests and we have unrealistic confidence in our own development
quality (and/or do not like to repeatedly test the same things manually).
What do we want?
Roger and I discussed our experiences. We both are very committed to release
high quality version of the Events module, and we have proven to be able to do
so over the past years. The 120.000 downloads in total with an average of 50
downloads per day supports this.
We believe in the principle of automatic testing, so I will continue to look
for testing tools that are really easy to work with, even in a dynamic
environment like DotNetNuke. I am currently investigating
Telerik Test Studio to see if this can work for us, and I am open for any
experiences or suggestion from YOU on this point.
We also believe in the feedback of users (beta testers), but also realize
that we have to trigger a response more actively. We are looking for a way of
gathering names and email addresses of people who are really interested to test
& provide feedback on new releases. The
DNN Events forum? A Facebook page? Any suggestion?
What have we learned?
Maintaining high quality and bug free software is difficult and incorporates
more work then you expect. And this reminds me on one of my statements in my PhD
dissertation almost 30 years back: Making software and raising children have
remarkable similarities: it take a lot of time and effort, only to find out in
the end that there are bugs in the system that you have put in yourself.
Life has not changed much over the years.
Have fun with the new Events module.
Looking forward hearing from you!