The state of “Not Invented Here Syndrome” in 2017

The state of “Not Invented Here Syndrome” in 2017

Development teams often build up high levels of trust internally due to the nature of the constant collaboration between team members.  Whilst that internal trust increases and increases, it can cause a lack of trust of outsiders whether that be 3rd parties or even other internal teams. So, when there is a genuine case for reuse there is often a strong argument against it. A common one is that the high-quality standards of the team can only be assured if code is written in house.

And why not. Developers like writing code, therefore given the chance they will write “all the code”. But code has a cost in terms on maintaining a solution over time. And we will have to support the solution because software isn’t written once then forgotten about, it continuously evolves. And let’s not forget that writing scalable, reliable and adaptable distributed systems is hard.  Who really wants to be debugging a custom load balancing solution when your system is on its knees and customers are beating down your door. Why invest the next couple of months building yet another custom security solution when your competitors seem to be releasing new features every few weeks.

The IT industry is seeing trends that will hopefully consign that old insular mindset to the history books.

Cloud computing offers, amongst many other advantages, the opportunity of offloading complexity on to some other party. Why worry about heating and air conditioning in a custom data centre when all you really need to do is build a website? Economies of scale means that costs are substantially reduced but you need to remember that cloud offerings are built for the masses and if you don’t fit then you may not get the benefits you expected. Cloud solutions such as Azure and Amazon Web Services practically offer a menu of services that you pick based on your requirements for ease of use vs the flexibility and control that you need. At the extreme, serverless computing promises that you can deploy and run code in the cloud without ever worrying about how the underlying infrastructure will be scaled to meet demand.

There is a trend where many companies are reinventing themselves as tech companies – Netflix and Amazon are just a couple of examples of companies that in order to be disruptive in their particular marketplaces transformed themselves into technology companies. Over the last few years this has reached a tipping point and now many organisations are trying the same thing and expecting the same results. Whilst it is true that IT is fundamental to many business models and being technically savvy as an organisation has a key role to play it is unlikely that everyone needs to code their IT from the ground up.

By looking at the first movers in that space you see technologies being developed in house to solve a particular problem and then shared back to the community. Google created AngularJS and Facebook the Cassandra NoSQL database. Today anyone can pick up these projects for their own use and perhaps more importantly they can contribute to them allowing them to evolve independently.

So, my vision of a team that is successfully avoiding NIH Syndrome in 2017 is one that

  • Has a wide understanding of the technology landscape
  • Does not exhibit siloed thinking about technology stacks, particular products or architectures
  • Has the time and space to try new things
  • And is encourage to contribute back into the community that they take from.

Reusing open source software is not like picking apples from somebody else’s orchard. It is a two-way proposition. You use an open source project to enhance your own product – usually to save cost and time. Therefore, you should invest some of that time back even if it is to simply fix bugs or answer questions on Stack overflow. And here in lies the challenge. Many organisations do not yet see the value of reinvesting in the community that bootstrapped them to where they are today and are so single minded they cannot see beyond their own immediate business pressures to deliver more features. Whether this approach is sustainable – I’m not sure. But as more and more companies transform into technology companies the cream of the development world will come to expect certain values from their employers and as you know the cream rises to the top.

 

 

 

 

Advertisements

Permission, Ability and Skills

Permission, Ability and Skills

It could be argued that the primary purpose of a software developer is to provide solutions to problems. Problems are presented in the form of requirements or user stories and it is the software developers job to provide a solution. They provide a solution with the tools that are available and within the constraints that exist within the organisation in which they work. Normally this is done unconsciously and we don’t stop to think about it. We open of our IDE of choice and start coding.

It is only when we start to encounter what might be considered more difficult problems, we start to see the limitations of this approach. What happens if the problem is that of building up a continuous delivery approach for your team or providing a level of disaster recovery for your product’s infrastructure.

Torturous Analogy Time.

Let imagine that the problem is that I need to travel from London to New York for conference. I have permission from my organisation to do this but I have no other help.  Let say I work for an aircraft manufacturer and they have a plane sitting outside ready to be used. They don’t want to waste money on buying a standard airline ticket when they have a £multi-million asset sitting outside. (I told you this is torturous!). So now I have the ability to get from London to New York but that still isn’t helping me because I can’t fly the plane, I don’t have the necessary skills.

In order to solve the problem I need the permission to tackle it, the ability, through technical support and the skills to use that technology to solve the problem at hand.

In software development we’ll nearly always have the permission and tools but that is not always enough to solve the big problems such as those a mentioned above – building up a continuous delivery approach for your team or providing a level of disaster recovery for your product’s infrastructure.

Solving the big problems often requires organisational transformations which require at least three aspects to change – people, process and tools.  Simply having the tools is not normally enough.  It like having the plane without the skill to fly it. As technologists, we want to believe that it is the tool that provides the solution to a problem, but it isn’t.

We are doing ourselves a misjustice really. We provide the skills that ultimately solve the problem. The technology simple provides the ability to bring those skills to bear.

Busy Time – Agile Anti Patterns

Busy Time – Agile Anti Patterns

Every development team is different. Sometimes you walk onto the floor where the developers live and it is a hive of activity. There might be a few people clustered around a white board, whilst others are having an animated discussion around a screen.  Often the workspace is set up to enable easy communication with no barriers, and break out areas where people can go to focus on a particular challenge. Through that noise, movement and activity you get a sense of collaboration.

Other times you can be deafened, deafened by the clattering of keyboards… and nothing else. People are obviously busy and getting things done but what they are doing is not immediately apparent.  Sometimes conversations will spark up but they are often muted with people wanting to get back to the task in hand. If you observe longer you might find people getting frustrated with each other, annoyed about interruptions and thinking that time collaborating in meetings is a waste of time, time away from getting real work done.

I’m going to suggest that the way success is measured in these two extreme examples is different. And that may be driving behaviours.

In the first example I would guess that the team is motivated by flow. They want to minimise the cycle time of getting work completed. This is driving collaboration and the desire to minimise work in progress. Whilst it might seem from the outside that all the noise and communication is wasteful and yes it might be less efficient from an individual’s perspective, it is effective in getting the work in hand done.

In the other scenario, you could walk up to any person and ask “are you busy?” and you would get a resounding confirmation. People in that team may feel maxed out. And it is likely that their managers would be very happy. After all, each member in that team is being fully utilised writing code. What’s not to like? The managers may even be reaching out to the recruitment market to ensure that the organisation’s delivery commitments are made.

What is happening is the two organisations are optimised around a different dimension. Team 1 are flow optimised and team 2 are resource optimised.

If you buy into lean thinking, then the focus should be on flow and not resources. Results for a business are achieved when ideas are processed by the development team and release to customers as quickly as possible.

Running people at 100% capacity is risky. By definition something at 100% capacity means nothing else can be taken on. Upstream that means work can stall. It means there is no contingency for the unexpected. Busy people become bottlenecks. If you are a bottleneck in a resource optimised organisation you become the focus of attention as you are critical to getting things done. Some people like that but not everyone does. And what happens in resource optimised organisations if one of the bottlenecks decides to leave?

In flow optimised organisations people tend to avoid being bottlenecks. Instead of being a doer people in these organisations are enablers. It might seem like a subtle difference but it is a fundamental in organisations that are routinely delivering new features to their customers as oppose to continuously putting out fires.

 

The continuous pursuit of learning

The continuous pursuit of learning

I often think the biggest challenge of working in IT is learning. Not only is the technology that we use relatively complex, it is evolving rapidly. The tools and languages you are using now are changing at an ever-increasing rate. You can no longer expect that what you are working on now will be current in 12 months’ time.

On the one hand, you could be working on something long term. Maybe working on a product, adding features over months and years. The fundamental technology stack of the solution is less likely to change so you’ll be using the same languages and tools each day. If you keep an eye on what is happening in the industry, you might see new things. Often, they are things you will not be able to use. Over time you might become more distant and start thinking you’ll never use the “cool” stuff.

How do you get to play with the good stuff if you have little opportunity to use it in your day job?

On the other hand, you might regularly move from job to job, whether that be as a freelancer or working for an IT consultancy. Here the challenge is not the lack of exposure to different things it is quite the opposite. There are so many things you are expected to know, how can you keep up to date, let alone be an expert in so many things?

How do you know enough about something to be credible in a new job?

How hard is it really? Let say you’re a .NET dev. You need to know your way around the .NET framework namespaces covered in this diagram.

NET_35_Namespaces_Poster_JAN08 (1)-page-001.jpg

There is quite a bit there but with a training course, some hands-on experience and a bit of background reading you can become quite effective.

Suppose now your application is running on Microsoft Azure and you are asked to evaluate if you are using the “right” Azure services and that your organisation is getting value for money.

Azure is this big, with each service have lots of hidden details and some being as big as the entire .NET framework.

Microsoft-Windows-Azure-Overview-Poster-page-001.jpg

And the questions keep coming

  • Which is better Amazon AWS or Microsoft Azure?
  • Can we use React or AngularJS for this new front end feature?
  • We should be writing Microservices, can we use containers here?

How are you supposed to provide credible responses to these questions? You are being asked because you are considered a “technical” expert, but if for don’t have a point of view can you remain an expert?

At this point I’m not going to wave a magic wand or pull a silver bullet out of my pocket that makes this all easy. This problem is not going away. Instead it needs to be addressed head long.

The first stage is acceptance. You need to realise that it will take effort to stay current. If you are employed and not freelance, your employer will be expecting you to stay current too. As a freelancer you have to keep up to date to continue to pay the bills. Employer may offer some sort of training budget but frankly this is never enough, often hard to justify, in terms of time away from clients and projects, and in most cases, you are not given a free choice on how this time is spent.

Once you have the epiphany that this is in your hands and no-one will do it for you, the next step is to broaden your horizons. Seeing what other people are doing can open your eyes. Sometimes you can see what other projects are happening in your organisation but this is just the start. We are lucky that we have the Internet at our disposal where you can find out about anything. However, curated sources may be better to avoid information overload. Services like Flipboard and Medium offer to collate articles on certain subjects. I personally prefer podcasts. I can listen to them in dead time such as during commuting, they tend to cover a broad range of subjects and they often have further reading lists.

Getting this far, it may be dawning on you that this will involve time. That is true: you’ll will have to make an investment in time to make this happen. This is true in any profession. Time always needs to be invested for self-improvement. IT is no different.

How much time should you invest? Think about all the commitments you have, both professionally and personally. If you are already working long hours and have a large family, then you are not going to be able to invest many hours each week. Perhaps, you’ll only be able to use some of your commuting time and maybe one or two lunch breaks a week. Other people may find much bigger chunks of time. The important thing is to be realistic. There is no point over committing. You’ll find it too much of a challenge and be very likely to stop altogether. The aim is to make it routine and easy.

Once you have carved out some time you should think about what you are going to achieve and how you are going to do it. Are you going to use that time for reading articles or listening to podcasts? Do you want to do online courses such as those provided by Pluralsight? Maybe you’ll be setting aside time so you can work on side projects or even contribute into some open source projects. What you end up doing will be personal and will relate to your overall goals. Personally, I try to do a few of these. Sometimes, like on my commute, Podcasts are my only options but other times it easy to open of the laptop to either read something or open up an IDE.

Happy learning,

Long Hours Culture – Agile Anti Patterns

Long Hours Culture – Agile Anti Patterns

If you have worked in IT for a while you see the same things happen over and over again. When you have worked in IT for a few more years you start questioning the things you see and wondering if there is another way. Given a few more years you’ll be the one shaking things up so the old norms are consigned to the dustbin of history.

The topic I’m touching on today is working practices for software developers or more precisely it is a challenge to the long hour’s culture often seen in software development.

There was a time (and it probably still happens today) where you simply didn’t leave work until after the boss did. They leave at 8pm so you’ll be leaving at 10 past. People would work 10, 12, 14 hour days so they looked like the important one, the busy one, the one that could be depended on. How many times have you heard people recount the 50 or 60 hour weeks they have worked and hold it aloft as a badge of honour.

Using these old terms, I might be considered a boss. Well I’m usually the oldest on a team but I certainly don’t consider myself a boss. Do I expect people to leave after I do? – no I don’t! Instead I judge people on results and how they interact with their team mates. Not on how much stamina they have.

I have likened the mental effort of software development to doing an exam. You’re doing complex mental puzzles for many hours each day. Having done this myself, I know that if I work flat out for 6 – 7 hours I’m mentally spent. After that I’m in neutral. I might be trying but nothing much is happening. Many studies have demonstrated that most people’s mental capacities are similar. If you tell me that you have been doing 12 or more hours a day I question what you actually have been doing in that time or whether that time would have been better spent recharging for the next day.

Lean and Agile tell us to use prioritisation and work in progress limits to set a predictable and sustainable pace. This means that the 6 – 7 hours mental capacity each developer spends each day is always on the most important thing. There is no stress and no pressure. You are always working on the most important thing. I not saying you only work 6 – 7 hours. Instead you are free to use the time outside of that for what you want, whether that is a side project, training or otherwise keeping yourself sharp.

Unfortunately, development teams don’t work in a bubble. There are often arbitrary deadlines which need to be dealt with. Before I move on to that, it is worth digressing for paragraph to consider why Agile teams find themselves with deadlines.

pizzas
Too familiar?

When these deadlines come up it is important to recognise where they are coming from. In my experience, it is always someone outside the team, not working in your sustainable pace each day. It might be a sales person who has committed to a customer that feature X will be ready by Y. It might be an external supplier who is expected to integrate with your software by a given date. The commonality is that they don’t know what the pace of the team is and they have plucked a date from thin air without any further context or understanding. They have made a commitment on your team’s behalf.

So, the team has to knuckle down and get on with it. The first thing to do is reprioritise. Focusing on the new commitment might still be achievable if all other things in the backlog are delayed. If not than the team should agree between themselves what the new working arrangement should be. It is a team decision and they should resist the urge to be dictated to from outside. This can be very emotionally charged as the external stakeholder may see the team not caring. What is really happening is the stakeholder has put their neck on the line and now they what everyone else to pull out all the stops to save them.

Everyone should recognise that as hours increase quality suffers. Notice I didn’t say “may suffer”. Quality will go down. Mistakes will be made. One strategy to minimise this is to ensure each team member has some opportunity to rest and recover. Not only is it important for their own mental state, a fresh pair of eyes are more likely to identify mistakes early.

Doing this however it a symptom of a wider problem. Ideally resolving it might be in your control but more often than not, it is cultural and is much harder to change.

Regardless, my mind is set. You can keep your evenings fuelled by pizzas and your days off in lieu for weekend work. All I want is a sustainable pace.

 

 

 

How long is a piece of string?

How long is a piece of string?

This is a post about prediction.

In the past, estimates were king. If you needed to know how long something would take, you asked for estimates.

However, estimates are personal. It is an individual’s opinion given their experience and knowledge of how long something might take.

As a developer I would hate being asked for estimates.

I knew I would be held to any number I provided. Often given a lack of available information and the pressure for a number an estimate would be pulled out of the air. I then would suffer the indignity of a team leader halving or doubling my estimate depending on who they were talking to. It was clear to me that estimates were never correct (whatever that means) and they generally caused stress and pressure all around.

Now I am frank when I talk to people about estimates. They are a guess, pure and simple! When people ask me for estimates I am vocal that they are asking me to guess. That can make people feel uncomfortable but I’d prefer getting the cards out on the table from the outset rather than everyone pretending that an estimate is something empirical, something tangible or even scientific.

Project Manager (PM): What is your estimate for X.
Developer : Based on what little I know – 2 weeks.
PM: You don’t sound sure – how confident are you?
D : About 60%.
PM: Okay I ‘ll say 4 weeks to be sure.

So the developer guessed. Then said they weren’t that confident. And then the PM doubled the original number. A guess of a guess of a guess!. We use terms like mandays and confidence factor to remove that uncomfortable feeling that we are just making things up. But the result of this guessing process is a number on a project plan. That plan becomes a commitment, often financial, and suddenly you are being asked why you can’t deliver something in the time you estimated.

Instead of feeling bad about it we should all get comfortable guessing!

Looking at chunks of work in terms of their size and complexity instead of the time taken to do them is a good start. You are relatively comparing two things within the same context – A is bigger than B but it is smaller than C. This is how story sizing works. A team uses past experience to size work. Empirical evidence is used to provide a forward projection of how much work can be achieved in a fixed time. The team delivered X units of work in Y weeks so it is likely that if everything stays constant then X units of work will be delivered in the following Y weeks.

The group aspect is often over looked. The work is sized as a group. The work is delivered as a group!

The sizing of the work is independent of who does the work, well it should be if you want your team to be as effective as possible. Different people will have different views of how the work can be completed so it is important to get a group perspective of what might be involved. Do not spend too much time on this. Even when you are sizing as a group it is still a guess. The process can also be incremental. Sizes can be changed when more information is available.

So given the history of the misuse and misunderstanding of estimates I can sympathise when there is a reluctance to undertake any exercise that would provide some level of predictability.

It is naive to think that a team can work in a silo and there will be no one wanting to know when the work is completed. If no one cares enough to want to know when a feature is completed or a bug is fixed then that probably mean that no-one cares about your product. If so why are you doing the work?

So let’s assume that you are doing the work because someone does care, be that a customer you’ll never meet, or a more Enterprisey customer that has made business commitments, or someone who is payrolling your team and wants to see a return on investment.

We are accepting sizing is a guess but that guess can be useful. By comparing your guess against how the team actually performed you can use that to become better at guessing and so better predicting when a feature might be completed or a bug fixed. No one is being held to an estimate. If a piece of work takes much longer than its sizing indicated then this is fine. Use it as an opportunity to reflect, understand what happened and improve the team’s predictability going forwards.

All this is gained from a guess!

Armchair Psychology for the Agile Practitioner

Armchair Psychology for the Agile Practitioner

Modern software development is as much about people and relationships as it is about programming languages and tools. A single developer, no matter how good, is never going to be as effective as a highly performing team. Over years working in such environments I have built up a rudimental understanding of the psychology behind teams, good and bad.

Confirmation Bias

People like to work within a framework of beliefs. We use this framework to make decisions about what is right and what is wrong, to predict what will work and what won’t. We favour information that confirms our beliefs and discount information that doesn’t. The difficulty is that everyone will have a different set of beliefs. When we are confronted by opposition the natural instinct is to assume that you are right and the others are wrong. People don’t like to think they are wrong, and so filter out information that does not confirm their pre-existing beliefs. This is confirmation bias.

Confirmation Bias is often behind resistance to change. It can be why organisational transformation is difficult. It is the PRINCE2 advocate that tells you all Agile projects fail because they heard about one Agile project that went wrong and they have too much invested in their certifications. It is the Microsoft .NET developer with 10 years in the stack who refuses to work with Javascript.

It is possible for people to change their beliefs but there needs to be a reason. Usually it is down to self-preservation. If there is an obvious benefit to change, they will change. It is just a case of finding the right motivation.

Sometimes it is possible to integrate both beliefs. The militant .NET developer may consider using Javascript if they moderate their views. Again it requires the right motivation, so if they see a person or organisation they hold in high esteem change, they might change too.

This sums up confirmation bias for me

Bandwagon Effect

The probability of one person adopting a belief is proportional to the number of people who currently hold that belief. Ever feel like you are swimming against the current, when everyone is against you and you’re the only person talking sense. This is the bandwagon effect.

The bandwagon effect is why it is hard to argue your point in a group of three when the two others hold the same opposing views. It is also why you can influence the direction of a team by ensuring that you have a core set of people who share your beliefs.

Information Bias

This is the assumption that the more information you have; the better decisions you will make. This is not always the case because information can be contradictory or bring its own bias to the table. Often people can make acceptable decisions with a lot less information. It is usually better to make a decision using the information you have, rather than deferring it.

People who suffer from information bias can struggle in Agile environments. Generally, user stories are brief and are likely to be missing “all” the information. Analysis paralysis ensues as the team strives to obtain a complete picture. Successful Agile projects tell us that all the information is not required, or at least it is not all required up front. A successful outcome is achieved through feedback and collaboration, experimentation and testing rather than amassing all available information at the start.

Overconfidence

Many of us are so confident in our abilities we will make commitments on behalf of ourselves and our teams without assessing the risks. Often it is the experts that are the over confident people as they are convinced they are right. How many times has an expert in your project under estimated the complexity of a feature, made a commitment and then the whole team finds itself under pressure.

Sunken costs fallacy and the impact on Agile teams

A sunk cost is any past cost that has already been paid and cannot be recovered. In IT this may be the time (and therefore resource costs) of gathering requirements, or doing an architectural study or refining a design. This time and effort is gone and cannot be recovered.

However, many people look at this differently. “It cost 20% of the project budget coming up with the design so we need to implement it otherwise we’d have wasted our money”. Did anyone stop to think if the design was appropriate or whether the requirements had changed or whether the product was even needed anymore.

Upfront planning is a common cause of sunk costs. The longer and costlier the planning phase, the more temptation to see it through. Agile projects go some way to address this by doing regular planning and therefore minimising the perceived costs lost if the direction of the project changes.

In large scale Agile projects, you may have a team of people working a sprint ahead, getting stories ready for the next sprint. This team will be fleshing out the details of stories, decomposing them, working out dependencies and even defining the order in which they need to be delivered. This looks good on paper but in this model without the SCRUM team’s involvement not much of the planning will remain intact when they do get involved. Sunken costs come into play because the “design team” are reluctant to throw away all that effort. The SCRUM team then become disengaged because they are simply implementing someone else’s design. All the Agility you have tried hard to gain is lost.