Let’s say you had to choose a default strategy for developing software:

• bureaucracy, waterfall development, centralised decision-making and micromanagement?
• Or a culture of trust, autonomous cross-functional teams, continuous delivery and iterative development?

Seemingly, people in the IT-industry now have come to recognise the latter as a more successful strategy. We know that agile works. Even in the public sector. Then why is the transformation from slow, rigid and expensive processes to a more successful, cost efficient way of generating better value so hard?

Sure, there are obstacles in factors like these:

• Budgets
• Contracts
• Architecture
• Organization
• Legacy systems
• Tools and technologies

These are certainly real barriers for an agile transition. However, they are all tangible, visible and, dare I say it, easier to deal with than the more complex, darker forces lurking in the shadows of the human mind (they are not really dark forces, but a normal part of our human nature and survival strategies).

In this post I suggest five forces I see working against a better way of developing software. All of them might not be relevant in your context and some are stronger or weaker depending on your context. If none of them are familiar, consider yourself lucky.

1. Individuals resist being changed

If you work in a startup or with innovation, you might be familiar with the diffusion of innovations. Only 16% of people are considered Innovators and Early adopters. Humans have a history of surviving by staying with the pack and following the majority, and most people need serious consideration to accept new ideas. For organizations, the adoption to new ideas are even more complex.

2. People enjoy power

Agile is about autonomy, learning by doing and letting the team decide how they solve the problem. This requires a large dose of trust, not just inside the team but from management and leaders. Agile does not work very well in a command and control environment.

Unfortunately, people who are used to being in control and exercise micromanagement will naturally find it hard to let go of their control. It requires a shift from Theory X to Theory Y, where not only leaders learn to trust but the team member stop playing the blame game and learn to take responsibility.

3. “We would rather be wrong than uncertain”

Agile are optimized for changing requirements and uncertainty. Humans are optimized for categorizing data and generalizing problem solving. This gives us the warm and fuzzy illusion of control. We are extremely bad at dealing with uncertainty. As Dan North says it: We would rather be wrong than uncertain.

The irony is that agile sceptics often mistake agilists for being daredevils and risk takers, when in fact agile is handeling risk in a much safer way than what carving assumptions into stone is.

One consequence of this is that we are tricking ourself into believing we can predict the future with mental images based on a weak analogies.

The House Analogy

One often used analogy for software development is building houses. This gives us the illusion that we are building something concrete which we should plan in great and accurate details up front, and it will in fact turn out that way. And the bigger the better. It also implies that demolishing the house and starting over is expensive, when in fact, deleting code and starting over is literally done by a keystroke.

The Elephant Analogy

Another analogy we often use is comparing projects to elephants. “How do you eat an elephant? One bit at a time.” This gives us the illusion that we know exactly how the end result is going to turn out, and we can just split the problem into 100% accurate increments and assemble all the pieces perfectly together at the end. When in fact we are trying to grow a new creature that doesn’t exist yet and no one has ever seen before.

4. Agile threatens The Establishment

This is an often overlooked consequence of following the agile principles. Delivering value early and frequently has a significant impact on how we are organized and what skills are needed. Our goal is to get real feedback early. “Working software is the primary measure of progress”. Instead of getting written specifications handed over, developers are working face-to-face with business people and user representatives. We are in the complex domain of the Cynefin framework and apply a probe, sense and respond approach. This greatly reduces the need for detailed planning, estimations and comprehensive documentation up front. Consequently, it reduces the need for coordinators, managers and administrators.

It can then seem like a contradiction when we choose to outsource programming, making it harder to collaborate through geographical distances and cultural barriers. At least it generates more work for management. Christin Gorman has an excellent point about this.

5. Agile is small and cheap

Agile is in a sense about keeping it small and simple and eliminating waste. Simplicity and downsizing does not generate the same amount of need for middle managers, coordinators, documentation, meetings and administration. Yes, we are saving costs and that is a positive force for the company paying for the IT related services. But on the other side we are also cutting income for the people making money out of those very activities that does not generate direct value.

Agile generate less income for management consultancies than the waterfall model (unless you are selling agile coaching, courses and certifications). Unfortunately, money is power and this is a strong force I think often is overlooked by somewhat naive techies. Budgeting and economy is not our favourite topic (and economists  on the other side don’t bother with understanding technology either when they outsource core business to India hoping for cost savings).

And let’s be honest. For humans in general, smaller projects are not as impressive on our resume.

“Don’t fight stupid – Make more awesome”

Introducing and sustaining agile can sometimes seem like a dark and long path through frustrating and complex people issues. Introducing change to a business requires great patience and a positive attitude. But it is also necessary to understand the challenges and unfavorable conditions working against you when all your best effort, logic, reasoning and patient are not getting you any closer to your goal. Recognizing the human drivers at play can make them easier to tackle and to know when to stop banging your head against the wall. Maybe you will even find a way to use them to your advantage?

Taking a deeper look at these human forces will also make you realize that all of them are a natural part of our nature and not as evil as they first might appear.

Fortunately, there are lots of techniques for overcoming people issues. Fearless Change (a wonderful read) by Linda Rising and Mary Lynn Manns contains powerful patterns that will make your mission towards change more fun and more efficient.

Besides, being the most adaptive creature on earth, humans have the capability of learning new skills and adopting to change. Bosses are becoming leaders, test managers are becoming an integrated part of development teams and project managers are providing their skills as coaches and scrum masters. And as I have learned from coaching theory and from reading Fearless Change: In order to introduce change, you have to genuinely believe in the hidden abilities and good intentions of people.