The State of Agile in 2018

I just finished reading Martin Fowler’s recent blog post, “The State of Agile Software in 2018.” His final three recommendations are real crowd pleasers:

1. Get rid of the Agile Industrial Complex and the idea of imposing stuff on teams. Let teams work out the way they should work themselves.
2. Raise the importance of technical excellence, and never forget that when writing software, the technology side is really vital, and…
3. Organize around products

I basically agree with his stated observations. Although his recommendations are easier to read than to execute. Furthermore, I find it unsatisfying that he doesn’t present any explanation for why these issues exist. Working as a consultant, I share his observations and I have a working model that explains them.

So in this post, I will share my own perspective as to why agile software suffers from the issues that Martin has called attention to — with most attention applied to the Agile Industrial Complex. I’ll follow that up with what I think teams should do to improve their own software development practices (and in so doing, become part of the Agile Industrial Complex 😎).

All Software Development Teams Have a Process

Maybe your process is sort of like this?

In the context of software development, a process is just a model of what’s happening in a system of people, described in terms of activities, events, and artifacts. Even if the process changes every single sprint and moves between agile and waterfall, there is a still a sequence of events and activities that are happening within that system of people. Even total chaos can be modeled as a process after it has been inspected well enough to be understood.

A Process is a Tool to Solve Problems and Address Priorities

Just as you would design a program to solve a problem or meet technical requirements, a process can be designed that will solve your problems and address your business priorities. A process should not be used that is not understood by the team using it. Some teams are not aware of their own process and sometimes that’s okay. The purpose of having a retrospective (and also postmortems) is to better understand and refine the team’s process in order to improve their performance and solve their problems.

In some domains (ie. construction, banking), a process can take so long to design, implement, and test that it is considered to be a valuable asset. It may represent a competitive advantage by mitigating risks or avoiding costly mistakes. In many of these same domains, standard business processes have been devised just as standard protocols are created in the world of tech. People who come from those domains may attempt to apply the same mental model to software engineering. I have witnessed jaws drop when people from construction engineering learn that no industry standard software engineering process exists.

The Problems and Priorities That You Have Should Determine the Process That You Use

I can’t tell you how many times I’ve observed a person who hears about something that another company does and immediately attempts to adopt the same practice without first critically thinking about what they will gain (or lose!). Every company is a little bit unique. As such, every company will have some problems and some priorities that are not shared by other companies. I could say the same thing about teams: every team is a little bit unique. That’s because companies and teams are both composed of (complex) human beings and we’re all a little bit different from each other.

Some Problems and Priorities Are Common and Thus Some Processes Should be Common

Even though every team and every company is a little bit unique, they also tend to share a lot of common pressures that are borne from common operating parameters. For example, just about every team is trying to be efficient with their time and resources. Every team is usually under pressure to deliver value to end users and/or customers. Every company is trying to grow. Every good engineer has seen the impact of shoddy code or poorly designed programs and that’s something they’re trying to avoid.

This commonality of problems and priorities suggests that some prototypical agile framework should work pretty well as a boilerplate for most teams. However, as with actual programs, the devil is in the details! In my opinion, every team could probably realize productivity gains by incorporating continuous delivery practices. But does every team need to integrate UX design? No, not if you don’t have user interfaces.

The genius of the Agile Manifesto, is that it does not dictate a process but instead it details key principles for designing a process that will help any team deliver value. Delivering value (by solving problems or addressing priorities) is the most basic and common mandate of any team that can be assembled.

Now with all that behind us, the reason the Agile Industrial Complex exists is because the Agile Manifesto doesn’t dictate a process. And because there will be problems and priorities shared by various teams across the industry, there will be some teams that have a better or more mature process than others. Those who have observed the positive impact of a good agile development process are usually more than eager to share it.

I would advance Martin Fowler’s first recommendation, with the following: When reading about another company’s practices, always consider how similar that company is to your own. Are they a competitor? Are they similar in size? Do they operate with the same business model? Is their culture similar to yours? If the answer to any these questions is “no” then maybe what works for them won’t work for you. The best agile development process for you will be determined by iterating on it. So ensure that you are consistently carrying out retrospectives and using those to optimize your own process.

Products vs Projects

I’ll end with a final word on projects. It seems that today, nobody wants to work on a project, they only want to work on a product. Products are sexy and projects are stinky poop. But not everything can be a product and thus projects are real and they are the right way to structure some types of engineering work. An example of this might be a platform migration (moving from AWS to GCP). Projects can still be managed effectively using agile principles. I’ll dive deeper into agile projects in my next post.

Stay tuned!