Agile – Page 2 – Rick Tonoli

Implementing an Effective Build Dashboard (for Hudson) (Part 2)

This is a follow on of my previous post and details how to implement the code available in the original post by Fabio Parera here, together with the modifications I made for our environment. I suggest you look at the original post first though to understand the extra stuff that I did not implement (I removed “expected duration” and “pinging servers”).

To start, modify Hudson to generate a code coverage percentage file, in our environment (.NET) we used NCover and NUnit but adapting your tools to do the same shouldn’t be a problem, the concept is simple:

Generate the coverage percentage file(s). See this post on setting up NCover and Nunit in Hudson
Extract the percentage from the file(s) and put into a single file in the root of your project. This is now accessible as http://<ci-server>/jobs/<project>/<coverage-percentage-file> after every build.

You need to install the Greasemonkey Add-on for Firefox for the script to work, do that now. After you’ve installed, you need to modify Firefox’s config and set greasemonkey.fileIsGreaseable to true:
- Open up Firefox and open up the page about:config (type it in the link box).
- Find the greasemonkey.fileIsGreaseable setting and, if it’s not set to true, double click it to set it.

Now extract the contents of this file to a working folder. Please note that this code is neither pretty nor up to date but is completely functional, feel free to modify at will.
Edit build-dashboard-radiator.html. Each project is defined in a section like this:

Create a section like this for each project you want to monitor, changing the href to your projects base directory.

(Optional) Edit build-dashboard-radiator.css and modify the style-sheet so it’s suitable for your display, the sizes for the fonts are located towards the end of the script. The current sizes are great for a 40″ screen.
Edit build-dashboard-radiator.user.js
- Change coverage_minimum to whatever value you want, anything more than this will be displayed as green, less, red.
- Find the section that looks like this:

you need to link the names of commiters to your project(s) to pictures, do this here. The names are what’s extracted from the version control system you use and should be set by every team member before checking code in. To find out what it is in Hudson, find a build and have a look at the json object generated from it, you can find it here (look for the value of “fullName”):

http://<server>/hudson/job/<project>/lastBuild/api/json

Note: This is, in fact, the source for all the information in the script, if you’re thinking of extending the monitor this is the first place to look for information on your build. Another place to look would be

http://<server>/hudson/job/<project>/api/json

which lists information related to the project as a whole (not build specific). Also, you can substiture lastbuild in the previous link for any build number.

Next, after you’ve saved all your changes, open Firefox and load the html file. Drag and drop the js file into the browser and this should now install if you have Greasemonkey installed correctly

You’ll know it’s installed correctly if you see this in your status bar

That should be it, hit refresh on your browser and bask in a cool dashboard monitor.

I’d like to thank the original poster (Fabio Parera), and of course all the people he thanks, for linking this awesome monitor, it was exactly what I was looking for and I hope it turns out that way for you too. Let me know how it goes and if you can think of any way of improving it.

Implementing an Effective Build Dashboard (for Hudson) (Part 1)

Information radiators are one of the core building blocks of any agile environment, you should never hide or feel the need to hide anything you’re doing. Information radiators encourage (or at least they should) a feeling of pride in teams allowing them to show off what they’re doing. They should be freely accessible for everyone to see what’s going on and share in the joy of what’s being created. They also encourage (if used correctly) good behaviour in teams: no-one likes public shame.

In line with this and following on my post of setting up a Continuous Integration environment with Hudson and .NET I decided to go about putting in place an effective Build Dashboard. For me an effective dashboard (or information radiator) is one that gives as much relevant information as possible with the least amount of effort to read. So, lots of flash, none of the boring stuff, easily accessible by as many people as possible.

Of the various methods out there I went for displaying information on a big screen, this gives you both the high visibility and the information all in one. Some of the other methods like lights and sound are nice, but limited in some way.

In our environment Hudson is our CI tool of choice and it has a pretty nice built in default dashboard:

.. which works great at short distance but not really great at long distance. There are also lot of plugins available to customize this dashboard and make it better viewable but none of them appealed to me as fit for purpose.

Looking around I finally found a post by Fabio Pereira that looked promising. Essentially the monitor is an HTML page with backing style sheet and javascript to extract information from your CI Server (like Hudson, using it’s API facilities). The standard one that he supplies in his blog looks like this (sorry for the bad quality of the pic) (when implemented in our environment):

For each project is should the following:

Name of project.
Build result (green for passed, red for failed, orange for building).
Time since last build.
How long the last build took.

I made the following modifications:

Added in a unit test coverage percentage (with colors indicating above/below agreed minimum).
Changed the font size based on how long ago the project last built. In other words, old projects shrink, lots of tiny font projects = not much being done.
Added a picture displaying who checked in code that triggered the build. Encourage responsibility for what’s done.
Added a color (yellow) to indicate an unstable (but not failed) build.

And now it looks like this:

Also, it’s strategically located:

dashboardlocation — Location of Dashboard

In the end it turned out to be pretty effective. It was fully automated, highly visible, everyone (team and customer) had access to it and could see it clearly and understand at a glance what was happening on each project. Broken builds and low unit test coverage are quickly exposed and no activity on projects can be seen quite clearly. Only thing I need to figure out now is how not to burn the image into the screen (no screensaver ) but I guess if there’s enough churn on a project that shouldn’t happen, more motivation to continuously check in and build I guess…

As usual I’m always looking for ways to improve so feedback is more than welcome. Can you think of ways to improve what I’ve done?

In part 2 of this I’ll got through the code behind this dashboard and try and explain it a little so that implementing it becomes easier, I’ll also attach the actual code (not pretty, but effective nevertheless 🙂 )

Performance appraisals, do we really care?

I often wonder why companies think the current way of doing Performance Appraisals are actually effective (you know, a few times a year, you sit with your manager and he tells you what a good/bad boy you were). For me they’re unnecessary and this is why: Firstly, “appraising” someones “performance” is something that should have the goal of INDIVIDUAL self improvement (for the appraised) in whatever areas he feels would improve his ability to deliver work, it shouldn’t be some blanket template applied to a group of employees (ok, you lot are “Developers” and you lot are “Senior Developers”) but rather a personal negotiated path to improvement. Secondly it should be something that occurs continuously, not a few times a year; self improvement doesn’t happen in leaps and bounds, it happens in tiny steps, learning from your mistakes and building on your successes.

I have a theory though as to why they still occur. Traditional “managers” exist to micromanage and ensure maximum productivity out of employees. Their focus is on creating the “perfect employee” that will obey orders and sacrifice all for “the company” without little or no concern for the employee as a person. What they’ve then done is pawned off the responsibilty of dealing with “human” issues (self improvement etc) to the dreaded Human Resources department, a set of people divorced from the actual work environment of the employee who apply templates to people in order to shape them into a perfect little workforce. What a crock of shit. How about managers step up to the plate and do what they should be doing: creating the environment for people to excel in, helping employees find paths to individual self improvement and backing off and letting people do what they were HIRED to do? Then we can finally get rid of the HR department and bring the responsibility back to where it should be, firmly in the hands of your leader.

Continuous Integration with Hudson and .NET

Recently I was tasked with setting up a development environment for a .NET shop. I’m a great fan(atic) of Continuous Integration (CI) and set about thinking of a good way of doing this (and a fun/cool way as well).

For those of you new to Continuous Integration, I think Martin Fowler describes it best on his Continuous Integration site:

“Continuous Integration is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily – leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible. Many teams find that this approach leads to significantly reduced integration problems and allows a team to develop cohesive software more rapidly.”

I’d been using CruiseControl for a while now, but for Java projects, and I had a recommendation to look at Hudson as an alternative. I evaluated it and decided it would be a much better solution for the following reasons:

It’s easy to use web based interface makes setup, configuration and use a breeze (over the XML driven environment of CruiseControl). This, combined with…
Hudson’s extensible plugin architecture makes it easy to create custom functionality. Finally add to this…
A large community support/development base

and you have a great, easy to use, well supported product with numerous extensions that will inevitably cover everything you need to do.

Ok, I know, CruiseControl has lots of this as well, but come on, who want’s to struggle with XML all their lives?

Installation

Installation is pretty easy, follow the instructions here based on your setup / choices. Personally we installed Tomcat 6 and deployed the Hudson WAR via the Tomcat Management interface. One additional thing we did was to setup a HUDSON_HOME environment variable to point to a disk with lots of space, I like to keep LOTS of build information (I love metrics) so space was an issue.

Configuration

Configuration is a breeze, once Hudson is up and running, access it via a web browser (normally http://<server address>:<port>/hudson) and select Manage Hudson. Before configuring Hudson itself (Configure System) I suggest you install any plugins you want to use. Do this by selecting Manage Plugins.

Plugins

The plugin interface is divided into four sections, Updates, Available, Installed and Advanced, all pretty self explanatory (right?). Use one of the following methods to install your plugin(s):

Select the plugin(s) in the Available tab, scroll down to the bottom and select Install.
Download the plugin(s) (generally from Hudson plugin repository), select the Advanced tab, fill in any proxy settings (if needed) and upload the plugin.

either way your plugin will be available once you restart Hudson.

For our installation these were some of the key plugins we installed, and why:

Hudson Backup – Allows you to backup/restore your Hudson configuration, we found this invaluable for obvious reasons.
Hudson build timeout – This plugin allows you to set a timeout threshold on builds, in case it goes into a “hang” state.
Claim – This plugin allows a team member to claim responsibility for fixing a broken or unstable build. Broken builds are BAD, and should be fixed immediately; it’s always good if someone takes ownership.
Green Balls – Green Balls instead of Blue Balls. People respond better to the Green/Red colour combination than Blue/Red.
Hudson GIT – We chose GIT as our repository and this plug-in enables access to it. No need to stress the importance of good Source Code Management (right?)
Hudson Doxygen – This plugin enabled the generation of Doxygen documentation from the source code. It’s a good tool to (visually) check the overall quality of the in-code documentation and I believe it encourages people to write coherent understandable documentation (although it’s only truly effective if someone periodically goes over it.)
Hudson MSBuild – MSBuild is used to build our projects from the command line. This plugin enables this functionality.
NCover – This plug-in enables the gathering and presentation of NCover results within Hudson.
Hudson NUnit – This plug-in enables the gathering and presentation of NUnit statistics within Hudson
Hudson Seleniumhq – This enables running and gathering of Selenium tests. This exercises our front end to ensure we don’t break anything already in place.
Task Scanner – This plugin scans for open tasks in a specified set of files in the project modules and displays the results.
Warnings – This plugin collects the compiler warnings of the project modules and visualizes the results.
Hudson instant-messaging – This enables instant messaging capabilities in Hudson. Used by the Jabber plugin.
Jabber – Sends build notifications to jabber contacts and/or chatrooms. Also allows control of builds via a jabber ‘bot’.
Twitter – This allows the sending of (basic) tweets after builds have completed to a specific twitter account. Visibility is important on any project.
Hudson Violations – This allows the gathering of data from various “violation checking” tools (we use it for StyleCop and Simian specifically) and presents it in Hudson. Also allows for limits to be set on “violations” forcing builds into an unstable state if necessary.
Hudson Release – This plugin allows you to configure pre and post build actions that are executed when a release build is manually triggered. We use this plugin to do our “releases”
The Continuous Integration Game – Enables scoring of builds, encourages good behaviour and is generally a good fun plugin to have.
Chuck Norris – A “fun” plugin, fun is important. Chuck get’s upset when the build breaks, plus a few cool Chuck lines to boot.

Hudson

Ok, now for Hudson itself. Hudson’s configuration is pretty self explanatory and well documented (click on the question marks for context sensitive help). I’ll run through what we did to get our server ready for jobs (some of these options only appear once the plugins are installed, which is why I recommend doing them first):

Home Directory – This is picked up from the HUDSON_HOME environment variable.
System Message – Appears on your landing page, write something cool here about your CI environment.
Enable Security – checked, with these options:
- Random TCP Ports, Access Control using Hudsons own Database, Logged in users can do anything
Ant – added an Ant installation for use by our Release Plugin.
MSBuild Builder – Configured for our builds, needed for .NET building
Selenium Remote Control – Configured the htmlSuite runner for running our Selenium tests.
Git – Configured the location of the Git executable (include the actual exe as part of the path).
Email Notification – Pretty standard SMTP information, remember to set the Hudson URL to something people who receive the email can use to access the build machine (ie, don’t use something like localhost). Also, set the System Admin Email address.
Continuous Integration Game – Activate it.
Jabber Notification – Enable it. Use the context help to understand the options and complete the need stuff. One suggestion, change the Bot command prefix to something other than !! to prevent outside people accidentally accessing your build bot and messing with your system.
Global Twitter Settings – Setup twitter access, Hudson will tweet based on the options you give, configurable per project as well.

Done, click Save

Creating Jobs

We were constrained to use .NET so we chose to build all our projects as freestyle projects, and the following describes how and what we configured using this option:

Click New Job, fill in a job name and choose Build a freestyle-software project.

Again, most of the options are well documented in the context sensitive help. Here are some of the options that we configured (I’ve left out most of the the common ones that are self explanatory):

Main Section
- Discard Old Builds – We chose to only keep 100 builds, for metrics purposes (did I mention I like metrics?), may change this later.
Advanced Project Options Section
- Quiet Period – Changed this to 15 secs to prevent problems with synchronization and building.
Source Code Management Section
- Git – We use git as out SCM of choice, we have a “central point” where everyone synchronizes to and the build server monitors for change. Just added a file based location (was on the same server as the Hudson so this was the easiest option) to point to the git repository. The rest of the options were left as defaults.
Build Triggers Section
- Poll SCM – checked, polls SCM every minute for changes. This triggers the build.
Build Environment Section
- Configure Release Build
  - Release Version Template – This label is used by the Release Manager to label builds. Can use the environment variables defined in the next section.
  - Release Parameters – Define parameters that you need during the release build here. We defined RELEASE_VERSION to allow for a release number to be passed through to the ant build script, defined later. These parameters will be prompted for when selecting Release Build from the projects menu.
  - Before Release Build – Steps to run before the build. Multiple steps can be defined.
  - After Release Build – Steps to run after the build. Multiple steps can be defined. We define a single Ant Build step to package, compress and add the build number to the relevant artifacts for the particular project. This compressed package is then uploaded to a FTP “release” site for consumption by other parties (QA people, release people etc).
- Abort the build if stuck – We set this at 10 min, fail if it gets stuck, just in case.
Build Section
- Here you can define a number of build steps to construct your solution and prepare it for release. These are the steps we performed, in order (for most projects, some were excluded for particular projects):
  - Windows Batch Command to pre-configure the environment. Essentially we modified the existing app.config and web.config and merged in environment specific values to override default values for things like databases and file paths. The tool we used for this was the MergeConfiguration tool supplied with Microsoft Enterprise Library 4.1. It takes a delta file (xxx.dconfig) and merges it with the existing file (app.config / web.config) and creates a new web.config / app.config. One thing to note here was that we had to move the existing app.config / web.config file to a temporary file (eg base.config) BEFORE merging and merge base.config with xxx.dconfig to create app.config / web.config
  move “<path>\app.config” “<path>\base.config”
  “<path>\MergeConfiguration.exe” “<path>\base.config” “<path>\ci.dconfig” “<path>\app.config”
  - Windows Batch Command to run in the database scripts using osql.exe. All databases are recreated from scratch using supplied scripts each build. We did this as a failsafe to make sure the SQL scripts matched the code that was being deployed.
  osql -S<server> -U<user> -P<password> -i<path>\ResetCIDatabase.sql
  osql -S<server> -U<user> -P<password> -d<database> -i<path>\Initialise.sql
  - MSBuild step, no parameters, just build the solution from the root.
  - Windows Batch Command to run the Doxygen command line utility to generate the code docs for the project using a custom doxyfile.config file.
  “<path>\doxygen.exe” doxyfile.config
  - Windows Batch Command to run the StyleCop command line utility (StyleCopCmd) to check the code for style violations. Some things to note about the command line version were that the settings file MUST be called Settings.Stylecop and be referenced without a path (ie, put it in the root of the project). Run the command without parameters to see what else you can do with it. This generates an XML output of violations that is used by the Violations plugin.
  “<path>\StyleCopCmd.exe” -of stylecop-output.xml -d . -r -sc Settings.Stylecop -ifp AssemblyInfo*
  - Windows Batch Command to run the Simian command line utility to check the code for duplication. A very handy tool to catch you cut and paste warriors out there, generates an XML file used by the Violations plugin.
  “<path>\simian-2.2.24.exe” **/*.cs -formatter=xml:simian-output.xml -failOnDuplication-
  - Windows Batch Command to execute Nunit and run Ncover against the results to generate a report on Unit Test coverage. Nunit executes the unit tests, NCover takes the XML output of Nunit and generates reports with trends. Very handy tool. One thing we haven’t figured out yet is how to run the unit tests that require a running web service, although supposedly the //iis parameter does it (not for use though). One thing we haven’t figured out yet is how to run the unit tests that require a running web service, although supposedly the //iis parameter does it (not for use though)
  “<path>\ncover.console.exe” //x coverage.xml “<path>\nunit-console.exe” /noshadow /nologo <full path to Unit Test DLL’s, space seperated> //h coveragedir //at ncover3.trend
  - Windows Batch Command to release the built artifacts to the internal environment. We “release” out files onto the CI server (running IIS) for Selenium to run against. Currently we do not have a need to run Selenium but when the need does come up, this will facilitate it.
  xcopy PrecompiledWeb <path of built artifacts> /I /R /V /E /Y
That’s it as far as building the project for now. There are a few more things that we have not implemented yet, but will in time, like Selenium tests, but at the time of this blog they were not done.

Post Build Actions Section

This section handles all the reporting back of builds. Here are some of the important things we enabled/configured:

Archive the artifacts – We chose to archive the entire build for reference purposes. Maybe a bit overkill.
Activate the Chuck Norris Plugin – Chuck is watching you!
Scan workspace for open tasks – This plugin scan the workspace for TODO’s, FIXME’s and NOTE’s (the tags we use to signify something to do in the code). Reports on the occurence of these. These are bad, putting TODO’s in code is pointless, rather just DO it.
Report Violations – gathers all the input XML files and creates a single Violations report. Used to pull in the Simian and StyleCop violations into a report.
Scan for Compiler Warnings – checks the standard out for compiler warnings, we set it to scan only for MSBuild parser.
Publish Doxygen – publish the Doxygen generated files, using the Doxygen HTML directory.
Email notifications – enabled the email notifications to send mail to everyone in the team whenever an event happened on the build (fail, fix, build etc.)
Activate the Continuous Integration Game (see the plugin link above for more details on this)
Allow broken build claiming – Allow people to claim broken builds. Ownership is important, someone must take responsibility for fixing the broken build. Remember, it’s not about blame, it’s about fixing it.
Jabber notification – activate and entered everyones Jabber accounts. This allows instant notification of build events to developers. Again, everyone gets notified of everything.

That’s it, your job is created and ready to build.

I hope this proves useful to some of you out there, I didn’t have too many issues with configuring Hudson in a .NET environment and those I did have were solved eventually. There are still a few issues/concerns that we need to iron out like:

How do we control versions and releasing versions?
How do we deal with build dependencies and releasing dependencies with projects? (Hudson supposedly handles this well, yet to be proven though)
How do I get NUnit to run the Web Service Unit tests?
Automating releases into a QA environment.

Other than that things are running pretty smoothly, will post more (shorter) entries as we make more progress.

The chatty daily standup…

One of the “ceremonies” of Scrum (man that sounds so culty) is the “Daily Standup”, a daily planning meeting where teams realign and coordinate their efforts for the upcoming day. It’s a concise, no frills no fluff zero waste of time meeting that answers three questions: what have I done, what am I doing and what are my issues, and is generally over quite quickly…but what if it isn’t, what if it degenerates into a technical discussion about the implementation of widget-x or how the build script is not building something correctly or which framework to use to solve some problem…

I’ve found this to happen quite often and all too often I see people coming down on the team with statements like “cut the meeting off at 15 min” and “remind the team to stick to the 3 questions”… all good advice but you’re fixing the wrong problem. I believe this behavior to be an indicator of a dysfunction within the team itself. In a truly collaborative environment teams should be discussing and solving problems constantly and not just at certain points through the sprint (like the daily stand up). If they’re only collaborating around the daily stand up then something is wrong and the team has a dysfunction preventing constant collaboration…

But haven’t we always been working this way?

When I was a kid I had a Commodore 64 with a cassette player, awesome little machine with cool games! I decided I had to write my own game, this is where it all started some 25 odd years ago. I decided I would write a “maze-like” game where an “avatar” would navigate through a maze picking various “gems” and scoring points, that was my goal anyway.

I started by skilling myself up in the basics of, well, BASIC (from books and magazines I had), once I knew enough to do the basics I put code to screen and started churning out lines of code, every now and then I would take a checkpoint and verify that everything still matched my goal…

draw the sprite, done, run and check the results, looks cool, figure out how to move it, done, write to code to make it move, done, run the code and see if it moves as desired, bugs!, change the code to work properly, sweet!

Excellent… time for a break, have something to eat and think about if I could do anything better, nah, the results looked good, was working as well as I had hoped.

I repeated this process until I had a working “game” knocked out. Every time I had a problem I couldn’t overcome I would simply change what I wanted, as long as it matched what I ultimately wanted, a (cool?) game with an avatar moving around a maze and scoring points by picking up stuff…

Sound familiar?

My questions on agile processes

Something I have found very interesting in today’s “agile environments” is the adoption of processes.

I’m caught in an endless loop on my thoughts around the place of processes and how they should fit in a team.

Generally processes are put in place in order to protect the team, sort of like a safety net. Given this chain of thought people should be interested in following the process, and should try following processes as close as possible. At the end of the day, Its almost like wearing a safety belt.

This is where it gets tricky though, in many cases developers, testers or any other team members may not not see the need for this process, they may believe it is more of an impediment than anything else.

How should this be dealt with ?

Should the team be wholly responsible for the decision to keep or lose the process ?

Should the team be educated every time a process is put in place ? ( Its too often I have noticed that people simply follow processes mindlessly)

In true scrum nature it suggests that the team make these decisions. What if some of these processes have been put in place by parties outside of the team ? Furthermore these parties that may have a good understanding for the need of the processes ?

So I go back to my safety belt analogy, sure its a pain to put it on, but if you are unlucky enough to be involved in an accident I can assure you that you will wish you had been wearing it. Unfortunately in scenarios like this it seems its only when its to late when one may value the process ?

Another question I have is around how often should the necessity of a process be revisited ? Is agile not about constant improvement, about constantly looking at what went well and what could have been done better ?

Here is my take, but please add comments im really excited to hear other opinions on this…

I believe that processes should be revisited constantly, possibly with each scrum retrospective, its important to remember when doing this to consider the impact of failing to implemt and follow the process.

If the process is enforced by external party, and you feel that you dont see the need for it, discuss this with the party responsible and try to understand why it is they deem the process necessary, it may turn out that you have already covered the risk that this very process is trying to prevent. After all is this not what scrum is all about ? COMMUNICATION ?

An (agile) environment…

Scrum often talks about “ideal development environments” (Mike Cohn has a nice blog post on this entitled The Ideal Agile Workspace, well worth the read) and this often excludes anything related to engineering (hardware, software etc.), for the obvious reason that Scrum does not profess to offer guidance on engineering practices (but clearly states they are important) but rather on “management” practices.

Anyway, to the point of this post. I have a challenge at my current job in designing a development environment (hardware/software, not physical environment) geared for agile development. I had some high level ideas:

Somewhere to play – I figure it’s important for team members to have an area to “play” in, experiment with ideas, test theories, do proof of concept, without it impacting other teams, especially in environments where multiple teams are building toward a common project.
Somewhere to build – teams need somewhere to actually construct things, typically this will be their own machine, together with a (decent) source repository to share and control versioning of code. Probably overlaps with their “play” environment.
Somewhere to share – Collaborating teams are good teams (if not great teams). Encourage teams to collaborate by not only co-locating members but giving them access to tools to share information (wiki’s, blogs etc).
Somewhere to test – Generally I believe there are four “kinds” of testing, integrated into the sprint at various points: developer testing (does my stuff work and have I broken anything), integration testing (does everything works nicely together), quality assurance testing (does everything performs at a predefined “quality” level, this should include, but not be limited to, security, performance, durability etc.) and user acceptance testing (does the product meet the requirements). Each level of testing adds an additional feeling of “warm and fuzzy” and, as such, I believe each of these environments should be clearly separated, demarcated and controlled.
Automation – everything that is repetitive should be automated, plain and (maybe not so) simple. There is, however, a small “addendum” to this, in my opinion: The degree of risk determines the degree or automation, the higher the risk, the less the automation. Take for example releasing a product into a production environment, a simple enough task to automate, but can you risk it going pear shaped? I don’t know, am I wrong here? Perhaps this is just a trust/quality thing and if you have enough quality (and confidence) checks in place, releasing to a production environment is a trivial task that CAN be automated?

In line with the above this is what I think would make a good environment:

Every team member has his own environment (machine) that is capable or running as much as possible of the required frameworks/tools/databases etc.
Where something cannot reasonably be run on a team members machine (think large databases), establish a common “development” environment where a reasonable copy of the system can be run on. Set rules for working on this common environment (typically a team can sort this out amongst themselves). The team has full control of this environment. Possibly virtualize this environment so that it can be backed up/restored/recreated/duplicated easily.
Create a shared environment where team members can collaborate ideas, a wiki and a blog are good ideas, hell, even a twitter like setup might be interesting, of course this should not be used by management to track progress!
Create a source control environment with a high degree of stability (backups etc) and control. Define solid rules for using this system, retroactively “fixing” a source control system is a nightmare. Some good ideas for versioning/branching can be found here (specifically for CVS, but the concepts can be applied to any decent version control
Create an integration environment that is a closer match to the production environment, automate the building of systems to this machine on a continuous basis, giving feedback to team members (via emails/web pages etc). This is the first “gate of quality” for the work being produced. This environment should be automated as much as possible (if not completely), again consider virtualizing for the same reasons as the development environment. Consider this a “sanity check” environment to make sure everyones contributions “play nicely” together. This environment can run all the testing frameworks (although consider also running on the development environment?):
- Unit testing (and test coverage testing)
- Code style checking etc.
Create a quality assurance environment that near perfectly matches your production environment. Work is automatically “promoted” to this environment if it has succesfully passed all the integration quality requirements. Consider doing functional and non-functional tests (performance, security etc) here (although the integration environment is also a good candidate for this, but the quality assurance environment should be a closer “replica” of production, so more than likely will behave similar to it). This is where team testers will “see” the results of the team, it should be treated similar to a production environment and accorded the same level of “respect”. Members of the team that are doing the physical development must consider this environment as “production”. Remember also to “automate what you repeat” at this point, tests that are run on the system should be automated as much as possible. This is the second “gate of quality” for the work being produced
Create a user acceptance environment where business will ultimately see and comment on work done. This is the final “gate of quality” for the work being produced. This environment should be treated EXACTLY like the production environment and code should only be “promoted” to this system, not built. After business accepts on here, the code is “release ready”. Consider automating the process of releasing code to this environment, if possible, however the types and degree of testing that needs to be run on QA in order to promote it to this environment may prevent it.
Finally, of course, production environment… but that’s a whole different kettle of fish and I’ll leave that up to more qualified people to build 🙂

Ok, so those are my (perhaps random) thoughts on the subject. Have I left stuff out? Sure, I probably have. Will this work practically? Not sure, I think it should, I’m going to give it a try. Is it “agile enough”? Who cares, it can evolve, change. I think it can work though, with enough automation and enough discipline it should be a decent solution.

I welcome any (constructive) opinions.

Why do we do what we do?

I thought of doing a few short posts on the why’s of agile and specifically scrum. Product owners are driven by ROI, so why should we not adopt the same attitude regarding the various “ceremonies” and practices we adopt in our day to day agile development lives… why do I do <something> and what is the benefit of doing it?.

I think it’s important that people understand the rationale behind doing something (like time-boxing for example) before simply adopting it, unless, of course, you like the idea of arranged marriages 😉

Here’s a couple of starters I’ll try cover:

Why is it important that the team decides what work should get done?
Why do scrum “by the book” first, and then change it?
Why do we time-box?
Why are the roles, and the clear separation of roles, so important in scrum?
Why have a retrospective?
Why demo?
Why use a physical scrum board?
Why should the team run the scrum board?
Why use a tool to help you run your project?
Why the daily stand-ups?

and others …

Feel free to give me some answers, but I’m not really interested in someone else telling me the answers. This is more of a personal journey of discovery so that I myself can understand why I do what I do, and yes, the answers may not be perfect, but every great journey must start with a first step…

Anarchy anyone?

I was thinking today, while watching a team fail miserably at planning, that we all know what we would do if asked to choose between Agile or Waterfall; but what if you had to choose between “MakeItUpAsWeGoAlongNoRules”-Agile and Waterfall. In other words, “Aspects of Agile” vs Pure Waterfall. I’d have to say I’d go for Waterfall…

(for this next piece to work you need to understand that governments exist to control you in a coercive and violent manner)

If you lived in country that was governed by a central authority, you’d have rules in place, albeit not all pleasant, and made up by a small group of individuals, but you’d still have rules that govern everyday life. Compare this to a state of “anarchy” (the Mad Max style “no rules leather bound bike riding shotgun wielding” kind) where there are no rules, or at the most constantly contradicting rules. What would you choose? I equate Waterfall to a central authority style coercive control and “aspects of agile” as the Mad Max style anarchy. Given this choice, and for nothing more than pure “survival” (read “sanity”), I’d have to go with Waterfall!

Of course, the true nirvana is true agility, which I guess you can equate to the type of true anarchy that rejects all form of coercive control, and has a basic set of rules, related to people and interactions between people, based on universally preferable ethics.

Of course I may be way off here, but these are just my thoughts.