Silos – Agile Anti Patterns

Silos – Agile Anti Patterns

Last time I wrote about Work in Progress and why you should managed it to avoid creating bottlenecks. Silos cause bottlenecks. So this time I want to share my thoughts about why you often fine silos in a software delivery teams.

Organising a team around technology

In large organisations you might find IT functional units such as UX teams or database teams. The company organisation treats IT functions like business units. This might make sense on paper, but delivering software requires much tighter collaboration.

This results in each team focusing are their own work. The system they are part of is a secondary consideration. They tend to optimise themselves around their own workload. When this goes bad, the rest of the system becomes a customer.

You want this database changing?

Fill in this service request and we’ll get to it when we are ready!

Pressure to keep everyone in the team busy

Lets face it. Often the people who deliver software projects are not always seen as people. They are resources that need to optimising. You might feel a pressure from above to “make sure everyone is busy”. You must find new work to keep “resources” busy. The pressure comes from people striving for efficiency over effectiveness. Good results can only come from busy specialists because they “cost” the most. The motivation is financial rather than delivering a quality product.

What this does it increase the team’s work in progress as your specialists start more work to “get a head start”. When they finish their part, the value is not realised as the rest of the team catch up. The work stacks up unfinished.

We have to accept that people are expensive. A desire to maximise the return on investment will never go away. The answer, if you care about team effectiveness, is not to shape the work to fit the skillset in the team. Instead the answer is to fit the team to the work by encouraging generalists.

Lack of Definition of Done

The definition of done is a key elements in fostering team focus. It counters the individualism you find when a bunch of specialist come together. Without it you may experience the “Many type of dones”. The most common is development done where work is “thrown over the wall”. It is even more frustrating when the wall is a desk partition.

When a team of generalist form, and they focus of collaborating on a prioritise backlog. As the do so, the silos seem to evaporate. Everyone is busy although they might not be working on their preferred tasks. Project managers are not required as the team organises themselves around the work. There is no need for someone to move the work from person to person. And those that hold the purse strings have a warm feeling. The feeling that comes from predictability and knowing that no-one is under utilised.


Ignoring Work in Progress – Agile Anti Patterns

Ignoring Work in Progress – Agile Anti Patterns

What does Work in Progress mean?

Work in progress is a measure of the amount of work at a particular state of a workflow. What this means in practice is the amount of individual development tasks currently being undertaken by an individual or a count of how many widgets are at the same stage of a manufacturing process concurrently.

Why is it important and why shouldn’t it be ignored?

As it turns out, Work in Progress is a very important element in understanding the effectiveness of a team.  Experience shows that optimising the Work in Progress by determined an appropriate limit for each stage of a workflow can increase the rate at which work can flow through a workflow and therefore increase the effectiveness of a team. In effect, it encourages a team to focus on what is important at any given moment and reduces the impacts of context switching by limiting the amount of work being done at any point in time. Observing work in progress highlights blockers and bottlenecks clearly and obviously.

Given that this should seem like common sense to you it isn’t surprising that minimising work in progress features prominently in most “Agile” frameworks.

However, in practice common sense is not always applied.  In the wild you’ll often encounter teams that don’t manage their WIP limits. They will argue stubbornly that what you are suggesting, having the whole team focus of a small number of activities, is some sort of snake oil that will limit rather than improve the team. What you as an outsider will observe will include some or all of the following

  • Individuals seem busy but feel like they are never achieving anything. You will hear statements such as “I have not got any real work done today”. In fact they spend most of their time context switching.
  • Many pieces of work will be started but a lot less will be finished. There might be a sense that the team is on a death march and never gets to do the improvements that are desperately called for. Worst the team cost effectiveness may be in question.
  • Sometimes the result of a team rather than individuals dealing with too much work is a bottleneck later in the process. I often see this happens when there is an imbalance between development and testing. In order to eliminate the backlog of development work ready for testing stacking up, the work in progress limit for development should be reduced. In a multi skilled team this might free up enough people to clear the backlog quicker.

The first step towards minimising work in progress is to understand whether you have a problem. The simplest way to do this is to use a Kanban board and measure the number of items at a given status at any point in time.  It should be immediately apparent where the most work is occurring. It is then possible to set limits to ensure that the flow of work through the team’s process is maintained. Once you are comfortable with that a cumulative flow diagram provides a deeper view of the work flowing through a process over time. This might show for example that a Scrum team has too much work stacked up in “design” at the start of a sprint, and the same thing happens to work in “testing” at the end of the sprint. Using this insight, the team may self-organise themselves to limit these bottlenecks which results in a better flow overall.

Kanban CFD_1

That is the easy part. The hard part is getting the team to redistribute themselves to get the work flowing through and to avoid management types from thinking the key to optimising flow is by playing convoluted games of Resource Planning Tetris.


Backlog Black Hole – Agile Anti Patterns

Backlog Black Hole – Agile Anti Patterns

If I create a story and add it to the backlog, it will be lost forever and will never get done

A backlog is a prioritised list of work that needs to be done. The important stuff is at the top and the least important stuff at the bottom. If you find that work is “disappearing” in your backlog what could be the cause

  1. The backlog is not being maintained. The backlog is a living thing and as such needs feeding and watering. By that it needs near constant refinement. New work is discovered it gets added to the backlog. But what is happening to the existing stuff. All stories need to be reviewed not just the new ones. They need updating based on current knowledge. That might mean that the story is not required any longer and should be removed. Some people go so far as purging stories that have not been delivered based on their age. The thinking is that if a story gets to being 3 to 6 months old without being delivered then the chances are it will not be delivered in its current form at all.
  2. The newest work is the highest priority. Just because you have thought of the next killer feature it doesn’t automatically mean delivering that work is the highest priority. It should be assessed based on all the work in the backlog. If new work is always added to the top this starts to push older worked down, often meaning the team never get a chance to work on it.
  3. The work is not well defined. In order for someone to understand the work involved in a story it must be clear. If you are going to the trouble of adding work to the backlog that you think needs to be done you should also put in some effort to describe it. I’m not saying that you need to write “War and Peace” but you do need to represent the work to the Product Owner in ceremonies such as backlog refinement. In some circumstances, there are benefits to be found by having a triage process for new work. This provides a chance for the work to be reviewed by the necessary parties to ensure that it is understood, be prioritised and actually needed.
  4. You don’t have the right tools. A small team might get away with managing their backlog with sticky notes on a board. Large teams may need some tooling. Tooling can be an inhibitor as well as an enabler. So perhaps a tool has been implemented that is hard to use or that requires the team to be trained on. This might make it hard to find stories when you need them. Often it is possible to configure tools to provide reports of stories added in the last week or to enable integration with messaging tools such as slack so you have a constant stream of messages indicating new work entering the backlog.

Up to now this discussion has focused on the negative position that it is a bad thing that work is “being lost” in the backlog. However, when you think about it this may be a sign that you are doing the right thing. The work coming in may be aspirational or simply a wish list which is not what your customers really need. If you have an effective feedback loop you’ll be reacting to your customer’s needs rather than focusing on the things that they don’t care about.

Therefore, if you are the one coming up with the ideas that are not making it into the system you need to understand why. You can’t be precious about the work because it was “your idea”. This is looking at the product you’re building from a personal point of view and not considering how the product is used in reality. Perhaps you don’t understand the product as well as you think you do.

Finally, it is worth making a point around continuous technical improvement. My point of view is that for a product to be successful over a long period of time the technology it is built with needs to continuously evolve. Whether you call this technical debt or something else the point is that there will always be technical work that needs to be done that may not have direct value for the customer. The value is actually to your business as you’ll be able to continue to serve your customers in the future.

How you deal with this depends on the organisation. Often people implement a capacity tax that says that a given percentage of the team’s capacity goes towards technical improvement. This way the team are not asking for permission to improve things but there is a still need to document and prioritise the technical work that needs to be done. This is still a backlog. In other situations where the product owner is technically savvy and understands the relative value between delivering new features vs technical improvement, technical stories can be treated as any other work in the backlog.

Whichever way you look at this, it boils down to the fact that there is a pile of work that needs to be done. The work needs to be prioritised and each work item will have a different potential value to your customer and your business. And their needs to be way to make this work visible and transparent in an efficient manner.

Never ending stories – Agile Anti Patterns

Never ending stories – Agile Anti Patterns

The Never-Ending Story was a 1984 fantasy film about a boy who reads a magical book that tells a story of a young warrior whose task is to stop a dark storm called the Nothing from engulfing a fantasy world.  Apparently it was quite good but all I can remember about it is the large white flying dog like creature.

The Never-Ending stories I am more familiar with are those in Agile projects that start out as something that is assumed to be quite straight forwards but then generates much more work than was expected and before you know it, not only are you carrying it forwards into the next sprint but it starts reoccurring in multiple sprints. So, what are the characteristics of a never-ending story.

A story really represents a feature and as such becomes a bucket for all areas of functionality related to that feature

Agile is about reducing feedback loops in order to build confidence that you are always working on the most important thing at any given moment. Often never-ending stories internalise that feedback loop. Maybe the story represents a new area of a solution. You don’t what to do too much up front design or suffer from analysis paralysis so you have a story that enables you to try a few things out. This is reviewed with the Product Owner and the rest of the team which generates new ideas which is new work.

There is nothing wrong with this description so far, however a never-ending story emerges when that new work is added to the original story. Often this is coupled with a feeling that no one really knows what good looks like. The team starts to feel like they are working towards a moving target. It will only be a matter of time before the “When will you be done” questions start. Think for a minute about how it is possible to formulate an answer. Every time the team asks for feedback, more work is generated and so the goal has changed. Yet they are asked to say when work, which they may not know about yet, will be complete.

Work is blocked by external dependencies

Sometimes someone outside the team has a stake on when a story is complete. It may be that you are integrating with a third party and they have onboarding activities. Perhaps there is an external customer that has a final say on whether the work has been delivered to the necessary standard. The important thing to realise is that you cannot control this decision making and it is highly likely that your timescales will not align. The best thing to do is to accept it and then put in controls to minimise the impact on you.

There are a couple of things you can do to take control.

  1. Understand the requirements of the third parties upfront and ensure that this is factored in when creating the stories in the first place. This may feel like up front design but you are doing just enough to mitigate the risk of a delay in the future.
  2. Don’t “leave the bonnet open” on work whilst it is outside your control. Deal with feedback whether that is good or bad as new work coming into the backlog and deal with it on a priority basis.
  3. Minimise external dependencies by only having them when you absolutely can’t avoid them or where they add value to solutions you deliver.

Quality defects stopping work being completed

In this situation, the work has been done and the team’s tester is working on it only to find a quality issue. The story goes back to the developer to be fixed. Subsequently the tester picks up the work only to find more issues and repeat.  When this has been happening for some time, e.g., it is a reoccurring theme in multiple stand ups, the team needs to try to understand why things are bouncing around between two team members. Are the developer and tester working together on the story or is the developer “throwing the work over the wall”?  Are the defects being raised related to the changes being made under the story in question or are they coincidental. Are the developer and tester on the same page as to the quality metrics for the story?

So what?…

Never-ending stories are bad because they harm your team’s predictability. They are a black hole, they consume effort and people. No-one in your team is really sure when they will be complete. If you are working on one it can be very demoralising. Personally, I am motivated by finishing things but never-ending stories can really feel like a ball and chain, never allowing you to finish and never enabling you to move on to other things.

When you look at the examples I give there are a couple of ways to avoid never ending stories

Firstly, ensure that you are aware of the work that is being created. Whilst it is normal to create some new work when delivering stories, you need to decide the point at which you need to call it out, surfacing it as new work in the backlog and letting the PO prioritise it rather than absorbing it. If you are not doing this during a sprint, maybe when the story is carried over at a sprint boundary, this would be a good time to ask yourself if the story should be broken down.

Secondly many of the problems with defining the scope and boundary of a story could be resolve by investing time in defined acceptance criteria at the start. You may use a Story Kick-Off for this. The acceptance criteria could describe how the story should function, they define the quality criteria for the story and the expectations of third parties. And let’s not forget that we should be aiming to avoid large stories, breaking them down along their natural seams in order to keep your team predictable and high performing.

Busy Time – Agile Anti Patterns

Busy Time – Agile Anti Patterns

Every development team is different. Sometimes you walk onto the floor where the developers live and it is a hive of activity. There might be a few people clustered around a white board, whilst others are having an animated discussion around a screen.  Often the workspace is set up to enable easy communication with no barriers, and break out areas where people can go to focus on a particular challenge. Through that noise, movement and activity you get a sense of collaboration.

Other times you can be deafened, deafened by the clattering of keyboards… and nothing else. People are obviously busy and getting things done but what they are doing is not immediately apparent.  Sometimes conversations will spark up but they are often muted with people wanting to get back to the task in hand. If you observe longer you might find people getting frustrated with each other, annoyed about interruptions and thinking that time collaborating in meetings is a waste of time, time away from getting real work done.

I’m going to suggest that the way success is measured in these two extreme examples is different. And that may be driving behaviours.

In the first example I would guess that the team is motivated by flow. They want to minimise the cycle time of getting work completed. This is driving collaboration and the desire to minimise work in progress. Whilst it might seem from the outside that all the noise and communication is wasteful and yes it might be less efficient from an individual’s perspective, it is effective in getting the work in hand done.

In the other scenario, you could walk up to any person and ask “are you busy?” and you would get a resounding confirmation. People in that team may feel maxed out. And it is likely that their managers would be very happy. After all, each member in that team is being fully utilised writing code. What’s not to like? The managers may even be reaching out to the recruitment market to ensure that the organisation’s delivery commitments are made.

What is happening is the two organisations are optimised around a different dimension. Team 1 are flow optimised and team 2 are resource optimised.

If you buy into lean thinking, then the focus should be on flow and not resources. Results for a business are achieved when ideas are processed by the development team and release to customers as quickly as possible.

Running people at 100% capacity is risky. By definition something at 100% capacity means nothing else can be taken on. Upstream that means work can stall. It means there is no contingency for the unexpected. Busy people become bottlenecks. If you are a bottleneck in a resource optimised organisation you become the focus of attention as you are critical to getting things done. Some people like that but not everyone does. And what happens in resource optimised organisations if one of the bottlenecks decides to leave?

In flow optimised organisations people tend to avoid being bottlenecks. Instead of being a doer people in these organisations are enablers. It might seem like a subtle difference but it is a fundamental in organisations that are routinely delivering new features to their customers as oppose to continuously putting out fires.


Long Hours Culture – Agile Anti Patterns

Long Hours Culture – Agile Anti Patterns

If you have worked in IT for a while you see the same things happen over and over again. When you have worked in IT for a few more years you start questioning the things you see and wondering if there is another way. Given a few more years you’ll be the one shaking things up so the old norms are consigned to the dustbin of history.

The topic I’m touching on today is working practices for software developers or more precisely it is a challenge to the long hour’s culture often seen in software development.

There was a time (and it probably still happens today) where you simply didn’t leave work until after the boss did. They leave at 8pm so you’ll be leaving at 10 past. People would work 10, 12, 14 hour days so they looked like the important one, the busy one, the one that could be depended on. How many times have you heard people recount the 50 or 60 hour weeks they have worked and hold it aloft as a badge of honour.

Using these old terms, I might be considered a boss. Well I’m usually the oldest on a team but I certainly don’t consider myself a boss. Do I expect people to leave after I do? – no I don’t! Instead I judge people on results and how they interact with their team mates. Not on how much stamina they have.

I have likened the mental effort of software development to doing an exam. You’re doing complex mental puzzles for many hours each day. Having done this myself, I know that if I work flat out for 6 – 7 hours I’m mentally spent. After that I’m in neutral. I might be trying but nothing much is happening. Many studies have demonstrated that most people’s mental capacities are similar. If you tell me that you have been doing 12 or more hours a day I question what you actually have been doing in that time or whether that time would have been better spent recharging for the next day.

Lean and Agile tell us to use prioritisation and work in progress limits to set a predictable and sustainable pace. This means that the 6 – 7 hours mental capacity each developer spends each day is always on the most important thing. There is no stress and no pressure. You are always working on the most important thing. I’m not saying you only work 6 – 7 hours. Instead you are free to use the time outside of that for what you want, whether that is a side project, training or otherwise keeping yourself sharp.

Unfortunately, development teams don’t work in a bubble. There are often arbitrary deadlines which need to be dealt with. Before I move on to that, it is worth digressing for paragraph to consider why Agile teams find themselves with deadlines.

Too familiar?

When these deadlines come up it is important to recognise where they are coming from. In my experience, it is always someone outside the team, not working in your sustainable pace each day. It might be a sales person who has committed to a customer that feature X will be ready by Y. It might be an external supplier who is expected to integrate with your software by a given date. The commonality is that they don’t know what the pace of the team is and they have plucked a date from thin air without any further context or understanding. They have made a commitment on your team’s behalf.

So, the team has to knuckle down and get on with it. The first thing to do is reprioritise. Focusing on the new commitment might still be achievable if all other things in the backlog are delayed. If not than the team should agree between themselves what the new working arrangement should be. It is a team decision and they should resist the urge to be dictated to from outside. This can be very emotionally charged as the external stakeholder may see the team not caring. What is really happening is the stakeholder has put their neck on the line and now they what everyone else to pull out all the stops to save them.

Everyone should recognise that as hours increase quality suffers. Notice I didn’t say “may suffer”. Quality will go down. Mistakes will be made. One strategy to minimise this is to ensure each team member has some opportunity to rest and recover. Not only is it important for their own mental state, a fresh pair of eyes are more likely to identify mistakes early.

Doing this however it a symptom of a wider problem. Ideally resolving it might be in your control but more often than not, it is cultural and is much harder to change.

Regardless, my mind is set. You can keep your evenings fuelled by pizzas and your days off in lieu for weekend work. All I want is a sustainable pace.




Context Switching

Context Switching

I try to avoid Absolute Thinking but I’m going to make an exception.

Context Switching is never a good thing.

I guess I’m going to have to justify that statement

The Problem with Context Switching

Whenever you read about the problems with context switching in IT it isn’t long before you find this 2001 article written by Joel Spolsky, called Human Task Switches Considered Harmful. It describes the complexity of software development where the developer has to juggle many abstract concepts in their head. It takes time to build up this mental model. Building it has a cost so it makes sense to hold on to it for as long as possible. The inevitable context switch means that a mental bookmark has to be used to keep your place, do the task that caused the interruption and then direct focus back to the original task.

However, it is not a simple as flicking back to your bookmark and continuing where you left off. You have to pay a cost to switch back. The state of mind at the point of switching has to be retrieved from memory and then the mental model reconstructed. It is a bit like the process of deserializing an XML or JSON file into an object graph. For each context switch you have to pay the price twice so it follows that the more you context switch, the higher the cost. The expense of context switching should never be underestimated.

It is often that the people that have to context switch heavily are the people that you rely on the most. There is a tendency to want to spread these people across as many things as possible, adding their magic to multiple projects. Let’s not forget they will be aware of that expectation, they will have no choice but to except the context switching, but they’ll feel that they can’t get anything done.

Why people think they are good a context switching

The mind is a wonderful thing, it adapts to your situation, it finds short cuts and efficiencies. All this means is that you are reducing the cost of context switching to some degree, but it is not completely removed. This is what people mean when they say that they are good context switchers. They mean that the cost of switching is slightly reduced – but it is still being payed.

Generally, people get used to context switching and accept it. The feeling of not achieving anything becomes so constant it fades into the background. How many times have you failed to tick off your daily to do list because you have been doing something else that was unexpected? How often have you thought that the day has been a write off and you haven’t achieved anything? How many times have you detected that sigh, when asking that same subject matter expert yet another question?

Strategies: Avoidance

The most obvious way to avoid the cost and mental energy drain of context switching is to take steps to avoid it all together. Try to focus on one task and see it through to the end. The reality in most organisations is that avoidance is not possible. Being able to focus on one task may not even be in your control.

You have to communicate, we are bombarded with notifications, modern UI interfaces are designed to distract and then hold your attention and modern software development encourages more communication.

Knowing that you have a problem with the inefficiency of context switching is the first part of the solution. You now are aware of the drag that taking on that extra project will have. Turn email, phone and other notifications off when you need to focus. Often the universal “I’m busy” signal can be a wearing a pair of headphones. This is often enough to filter out the minor distractions but the important ones will still get through.  This needs to work both ways. As you set aside time to “get on with stuff” you also need to make time to deal with distractions.

What happens when the distractions are regular and constant?

Strategies: Acceptance

When people from a technical background find themselves in a position of leadership or authority they often feel like they are not achieving anything. They had been creating working software but now find themselves in meetings or helping other people in their team.

However, you are achieving something, but the goal posts have moved. When you lead that design workshop, you were helping your organisation meet its business objectives. When you were mentoring members of your team, you were helping them develop high quality software.

The point is that you may simply have to accept the context switching that comes from this role. You have to accept that you are helping your team be productive rather than yourself. It is just about setting expectations.

Again, you are able to manage it. If you are constantly getting asked the same question, create a FAQ. Get in early to give yourself some “me” time. And if you really can’t scratch that “need to achieve” itch you might be able to find side projects outside of your working time.

Why Agile Ways of Working help?

Frameworks such as SCRUM designate communication and working time. Ceremonies such as planning, stand-ups and retrospectives formalise communication and do so at a regular cadence. Doing this regularly sets an expectation that this is time to organise. It greats a habit. Outside of this is worktime. Worktime is intended to be calm and a time to minimise external distractions. The team still has to communicate but they can reduce distractions by focusing on a small number of things keeping context switching to the minimum.

  • You plan together as a team to a single goal.
  • Each day the team synchronises to ensure that everyone knows what they are doing but help can be apportioned when needed.
  • Work in progress is minimised to avoid having to switch focus.
  • A role such as a SCRUM master protects the team from anything that would cause them to lose focus.

Sounds like a good idea!