Accuracy and Precision

Today I got asked what I thought was the difference between accuracy and precision as it pertains to relative estimation, I wasn’t ready for the question though and it’s been sitting on my mind bugging me so here are my thoughts:

I would think accuracy would be how close the estimation was to the truth and precision is how often you predict the same estimation given the same or similar complexity, in other words how repeatable you are. Accuracy is how right or wrong you are and precision is about how consistent you are (irrespective of right or wrong).

So, if you correctly estimate over and over, you’d be accurate and precise but if you estimate incorrectly, but are consistent about the “incorrectness” then you would be precise but not accurate. If you estimate correctly only every now and then for the same given complexity then you would be accurate but lack precision.

Interesting to hear what others think (like if I’ve lost my mind completely 🙂 )…

 

Continuous Improvement

Thinking about continuous improvement the other day I tried to figure out what makes a continuous improvement process work and what makes it stick, these are my thoughts:

  • It should be a slow, gradual process of change, no “sudden movements”.
  • It should be a continuous flow of change, no settling down into a rut.
  • It should be a culture, not a bunch of individuals with ideas.
  • It should be unanimously accepted.
  • It should be measurable.
  • It should be celebrated.
  • It should be evangelized.
  • There should be room for failure, and for success.
  • It should be rewarded.
  • It should be encouraged.
  • It should become default behaviour.
  • It should be transparent.

What else?

 

A great team

For me agile is about many things but first and foremost it is about people and dynamics between people, you get the right recipe going and anything is possible. I work with what I think is a great team, but what makes them a great team? I’m not entirely sure but here are some of my ideas of traits of a great team (in no particular order):

  • There’s a good level of respect not only for each other as people, but in each other’s ability to do stuff.
  • They trust each other.
  • The team has built in redundancy so anyone can take time off at any time, no silo’d experts.
  • There’s lots of laughter and healthy tomfoolery, they know when to work, but they also know when to play (at work!).
  • They allow failure and learn from it,  focusing on the problem, not the person.
  • High levels of collaboration, both at work and at play. They help each other out constantly, no-one is ever “too busy” to help.
  • They’re cross functional, there’s a good mix of skills amongst the team, helping with redundancy.
  • Everyone feels they can talk openly about issues and problems.
  • There is no fear (of failure, of specific people), they take chances and enjoy challenges.
  • They self organize, there is no one leader, they all lead, they all follow.

There’s more I’m sure, but those are the ones I can think of right now. At the core of it I think people must feel like they belong, that they are respected for their skills and trusted with responsibility and above all treated like people, not resources.

Thoughts?

Velocity for the layman

WARNING:This is a brain dump of an idea I’ve been playing around with, some confusion may occur!

As a ScrumMaster I often struggle to explain the concept of velocity to people, a critical concept to grasp to understand how to report on progress and capability. I often get these questions thrown at me:

  • How many features can we complete in X weeks? or…
  • How many bugs can one developer fix per day ?
The danger of these questions, for me, is not so much in the question (which doesn’t make sense) but in what happens afterwards:
  • 1 developer = 1 story per sprint so 2 developers = 2 stories per sprint, right?
  • 1 developer can fix one bug in 1 day, we could double our rate by doubling our people!! or…
 So, to the challenge: how to explain the danger is making all pieces of work equal and how to explain the benefits and logical sense of measuring in velocity.
Let’s start with the analogy: You’re going on holiday, you need to travel from city A to city B (your goal) with 5 stops between.
Doing it wrong would be to assume that you multiply the time to the first stop by the number of remaining stops and that would be your final travelling time.
Doing it right would be to look at the distance between each stop, calculate the average speed you can maintain, and then add up the travel time between each stop. Even better would be to:
  • Adapt your average speed based on environmental conditions. Bad weather, day/night travelling, fatigue, additional rest stops, travelling through mountains? These all affect your average speed.
  • Keep track of your average speed as you travel, this should give you an indicator of exactly what average you can maintain.
  • Check what affects the distance between towns like road closures, alternate routes.
  • Be ready to adapt to changing conditions, like accidents and sudden road closures.

Given the above you should be able to accurately predict when you will arrive at your goal. As a bonus you can also work backwards. Given your need to arrive at a particular time, you can sum up the total distance and divide it by the time you have and this should give you the speed you should try to average.

So, to bring it back to velocity, you average travel time can be equated to the teams average velocity (or capability). Stops are user stories/bugs and distance between cities is the effort of doing the story. Conditions of road, road closures, mountain travelling, predicted bad weather etc is your complexity.

Here’s hoping this analogy makes sense, if it doesn’t please let me know. I still think it’s missing something though so I may add to it at some point…

When will we be finished?

I’m a firm believer that all work should have an agreed end date, but end dates are always a bone of contention and cause endless confusion and wasted arguments and disagreements. These usually hover around the “when will we be finished?” question and, more to the point, no properly defined (empirical) way of working it out.

That in mind I thought I’d lay down the process we use to arrive at our end date. I will also attempt to do this without using the word scrum or agile, not because I don’t like or agree with them, but because those words tend to create predefined thoughts and have a life of their own (unfortunately).

 

So first the basics: In a nutshell we work in iterations, releasing something (hopefully of value) every 2 weeks based on a prioritized backlog. The backlog is constantly updated as stories are added / removed / completed.

Now for the fun. In order to calculate an end date with reasonable empirical accuracy you need the following: remaining effort and an average effort per sprint / day the team is capable of.

Effort Scale

Effort is difficult to define but key to the entire process. Everyone should agree and understand what is meant when we say “X amount of effort”. What you must realize is that it does not matter what unit of measure you use to define your “effort” as long as:

  • it is consistently used across all stories
  • it is understood by everyone
  • something of effort X closely matches another thing of effort X,  for small values of X. 
  • the units should increase roughly exponentially. This is to allow for indication of greater uncertainty with significantly higher numbers. Some people use the Fibonacci sequence (1, 2, 3, 5, 8, 13 …). Also remember to allow for zero effort.

We struggled to settle on our effort scale but eventually had to settle on a time relative scale (for various reasons):

  • Very Easy (1 point) – roughly a day to complete
  • Easy (2 points) – between a day and 2
  • Medium (4 points)  – roughly half a sprint
  • Hard (7 points) – roughly a full sprint
  • Epic (20 points) – unknown or way more than a sprint

and of course 0 points for quickies.

Effort per story

Assigning effort to stories is an exercise the entire team should do but failing that at least the majority of the team. Stories should also be continuously rechecked to see if the effort is still correct, especially stories with high effort assigned to them.

The process we use is as follows:

  1. Discuss the story
  2. Vote on an effort, if there’s dispute, use the highest of the choices, if there’s huge uncertainty, assign the highest effort.
  3. Rinse and repeat

Remember also to compare stories of similar effort. Another way or form of checksumming is to, during planning, assign tasks to the story (as in what you’re going to do to achieve the story) and then assign a rough hour estimate to do the tasks. Sum up the hours for the story and an equivalent effort story should probably have a similar amount of hours, although this is only really useful for well understood stories.

Average Effort Per Iteration

Calculating the effort per iteration (sometimes called the velocity) is a matter of adding all effort units (points in our case) for all completed stories for a sprint. It’s important to only tally up completed stories or the whole process is pointless. Average effort is just the sum of the effort for X number of iterations divided by X. Obviously the more data you have the better. What we also do is remove the highest and lowest values (as anomalies) from the calculation, to account for those strange iterations (over Christmas, Easter etc.)

Remaining Effort

Remaining effort is simply the sum of all effort that has not been completed.

The Formula

Calculating the end date then becomes the following:

By Day: date today + (remaining effort / average velocity / number of work days in iteration) divided by 5 and then multiplied 7 to account for weekends = your end date

By Iteration: current iteration number + (remaining effort / average velocity)  rounded up to whole number = final iteration of work, end of iteration is end date

and that’s it, you’ve predicted a future date based on historical data using a reasonably empirical method. You can improve the accuracy of this estimation by doing the following on a regular basis:

  • Re-evaluate the backlog from an effort point of view, focusing particularly on high effort stories.
  • Recalculate your average velocity at the end of each sprint and factor it in.
  • Factor in holidays and leave into your calculation.

Comments or suggestions welcome

Star Trek, a model for Agility

As a Trekkie I’m prone to watching reruns of old Star Trek episodes from time to time and I started seeing many similarities between an agile approach to “getting things done” and the way the crew of the Enterprise work. Yes, granted, it’s a militaristic structure but that’s not to say you can’t still be agile:

  • Crew members exhibit a general knowledge of most fields, with expertise in a particular field (see how well the bridge crew interchange roles during disaster situations where members go down).
  • Away teams are generally cross functional based on the task at hand, and usually range in size from 3 to 7.
  • The Captain issues orders (“what”) but relies on the crew member(s) to execute them (“how”), and does not interfere unless further clarity is needed.
  • There is a deep level of trust and respect between crew members.
  • There is constant collaboration, communication channels are constantly open from anywhere on the ship (and off).
  • They celebrate their victories and learn from their mistakes, and adapt as needed.
  • At any point in time, the crew is working on whatever will add the most “value” to the mission (work is prioritized).
  • There is an understanding that collaborative teamwork is the way to success.
  • There is a good work/life balance, even on a Starship.
  • The crew are passionate about their roles.
  • Mission planning sessions only plan enough to get the mission going, and are then adapted on the way.

Any other Trekkies out there see similarities I’ve missed?

I’m not a child…

Dear Manager,

I am one of your developers, when you hired me you treated me with respect, you asked me about my skills and abilities and (I assume) hired me because of these. Please could you explain to me why, now, the respect is gone and I’m being treated like a mentally deficient child?

If you want me continue adding value please consider the following:

  1. I am not 6 years old, stop treating me like a child.
  2. Using fear, intimidation and sarcasm to control me does not work, the more you try the more I will resist. Try a little more collaboration and a little less intimidation…
  3. If you would like me to tell the truth, don’t force me to lie by shouting at me every time I tell you the truth.
  4. Stop rejecting reality and inserting your own version of it, this is a sign of insanity.
  5. If you force me into a corner, I will lie to get out (see point 3).
  6. If you want me to be responsible, give me ownership of the problem AND the solution and stop telling me what to do by when.
  7. Stop treating me like the “bad guy” and take responsibility for your bad management decisions.
  8. I’m not giving my opinion to irritate you, I’m giving it in an attempt to add value. After all, you hired me for my expertise, right?
  9. Respect that I am a person, not a robot or a number, and that I have a life outside of work.

regards,

A. Developer

Trust…

Forcing impossible deadlines, ordering people to be at work during certain hours, denying leave, bringing in consultants and hinting at using them as a yardstick for productivity… I often wonder what makes people resort to these tactics and if they know the net effect of them? Decreased morale, lowered productivity, lack of any desire to do anything, “stick it to the manager” mentality, tiredness, increased stress, fear of job loss…I can go on but you get the point.

What brings this on? Not entirely sure but I can speculate…people react differently to varying degrees of stress and who can say what goes through someones mind when faced with pressure from above from someone who does not fully understand the given situation and lacks (or has incorrect) facts. I would say though, having worked with various managers under varying degrees of stress, that this is indeed the role of middle management, to absorb, explain, clarify and defend the team in which they trust, assuming the trust relationship is there.

Trust, that’s perhaps the key factor here and I think the core of where these decisions come from. Trust is the difference between saying “all leave has been denied until project X is done” and “as a team you guys decide whether you can absorb the impact of this leave but whatever you decide, I will back you”. Trust is not telling your team “you will be at work between the following hours…” but rather saying “there’s an incorrect perception linking the current level of productivity to peoples hours of work, how would you guys think we should approach correcting this perception?”.

Patrick Lencioni, in his book “Overcoming the Five Dysfunctions of a Team”, lists Trust as the foundation of any good team:

“…members of great teams trust one another on a fundamental, emotional level, and they are comfortable being vulnerable with each other about their weaknesses, mistakes, fears and behaviors…”

“Team”, in this case, includes everyone trying to achieve a given goal…everyone. Often I see management reeling under the pressure from above and, instead of approaching their team, bringing them into their “world of pain” and trying to collaboratively find a solution, they will resort to giving orders and clamping down. Inevitably though, the harder they squeeze, the more they lose control, forcing them to squeeze harder…a vicious circle from where the only thing that results is broken trust, something that’s almost impossible to fix.

Change is good

I’m often amazed at the number of dogmatic “prophets” out there when it comes to the latest fad’s of agile development. I myself am about getting productive and like to take the best of ALL worlds; change is not only inevitable but necessary, even changing of a process BUT, only if the change leads to improvement! I believe a process of continual introspection and change by a team, combined with a good way of measuring the effect of that change,  is essential for good team performance.

That in mind we made the following changes to the way we practice scrum:

  1. We added a default story to cover recovering technical debt (or refactoring if you will) in that story. Essentially we assign a “story point cap” to this story (say 5 points) that’s based on our current velocity and duration of sprint. We then assign “technical debt recovery” tasks to this story until the team feels there are enough tasks to fill the story points. I know, sounds like the wagon pushing the horse but it’s something we’re trying that other teams have used effectively.
  2. We’ve started estimating (wait for it…) HOURS on tasks. Stories are still broken into story points but the tasks are estimated in “hour blocks” (2, 4, 8, 16) as they are put up. The purpose of this is to act as a type of “sanity check” for story point estimation. An added benefit I found was that the team members seemed to think a lot more about a task when they had to assign time to it.
  3. We’ve started using a round table instead of the tradition square meeting tables. I found this very effective in encouraging participation and collaboration.
  4. Story point estimation was done using a technique I learned on training that speeds up the estimation incredibly (works well with lots of stories). This is how it works (well, how we did it anyway):
    1. All the stories are explained to the team and placed in a pile on the table.
    2. A wall/board is divided into sections, each representing a story amount (we used 1, 3, 5, 8, 13, 20, 40, 100).
    3. Each team member takes a pile of cards and sticks them in the column(s) they believe it belongs, no-one talks or interacts other than with the product owner to understand the story further.
    4. If a team member disagrees with where a card is, he/she simply moves it to where he believes it SHOULD be.
    5. This carries on until all the tasks are done. Remember, the key is NO COMMUNICATION between team members during this process, this has the effect of forcing people to think about why someone made a decision to place something in a particular column, or why they’re moving it.
  5. We’ve started experimenting with TDD and Paired Programming, which seems to be going well.

One additional thing I’m toying with is moving to a more Kanban based board, the team has problems with taking on too much work and I think creating a way of limiting the pulling of work will work.

Finally on how to measure if these changes are effective, well, to be honest I’m not entirely sure. I think the best way is simply to see if the team becomes better at actually doing what they predict they can do while maintaining a sustainable pace and having fun; other than that I guess we wait and see…

I’d be interested to know what kind of changes you brought about in your team and the effect of them (both good and bad), let me know…

Implementing an Effective Build Dashboard (for Hudson) (Part 3)

Not really part of the dashboard configuration but more of a cool thing I thought I’d share the underlying layers (hardware, OS etc) behind our monitor.

I decided to go with full portability and hands free start-up, I did this as follows:

  1. Installed Puppy Linux on a 2gb USB stick.
  2. Installed Firefox 3.x (needed for the Autohide extension).
  3. Installed Autohide extension (auto hides parts of Firefox and forces full screen on start-up).
  4. Found an old derelict PC and plugged it in, started up, configured the video display (once off on start-up).
  5. Configured the build monitor as per previous posts.
  6. Changed the start-up of Puppy Linux to auto start Firefox.

And there you go:

Build monitor, on a steeeck!
Build monitor, on a stick.