Implementing Agile Values and Principles: Chapter 5

A few years ago I wrote Understanding Agile Values and Principles (https://www.infoq.com/minibooks/agile-values-principles/). I’m working on a follow-up book about implementing them entitled Implementing Agile Values and Principles. I will post draft parts of the book on Medium.com as I finish them. I would certainly be interested in hearing people’s opinions about the draft material which I will certainly consider in making revisions as I work toward the final version.

This post is the draft of the 5th chapter on Responding to Change.

Chapter 5: Responding to Change

In UAVP I suggested that this value perhaps represents the common dictionary definition(s) of agility more than any other value. All such definitions imply minimal effort to make changes. I was influenced early on in my thinking about agility due to an article by Jim Highsmith comparing software development to climbing.

Alistair Cockburn described this relationship saying that in both climbing and product development:

• Technical skills matter but can be learned and, though some people are better at it, people can improve.

• Not just any solution will do, so planning is required but improvising is needed.

• Tools are required. You can’t get far without them, but they don’t have to be “fancy”. [In fact, simpler ones can be better. Consider the first Agile value.]

• Usually done in teams as solo efforts are a minority.

• It’s fun and challenging while, at the same time, dangerous (e.g., physically in climbing, failure in product development).

Another interesting take on this has been provided by Miki Tebeka who offered the following ideas on climbing and development:

· While strength has it’s place, technique matters more so take time to learn new techniques.

· Tools matter but they take time to master, so give yourself time and practice to do this.

· Learning how to fail and being willing to do so can open you to different approaches, but reducing the impact of failure is necessary.

· Learn from others and share what you know to gain more techniques and expand your learning.

· Sometimes being safe is wise but the fear of failure can cause you to miss a better solution.

· There will be progress and setbacks so commit to a long view and take care of yourself as burning out in the short-term will lose progress you have made.

· Step away from a problem; rest; sometimes you need that to find the way.

And, as mentioned in Chapter 1 when discussing agile definitions, balance and stability are important for both as well.

An important realization is that agile methods expose the true nature of project work. Work does not move neatly from one phase to another as the waterfall model would hope happens. As noted below, an iterative approach has been what people have used since at least the late 1950s. However people have tried to appear to follow the accepted waterfall idea though waste occurs and there is loss of useful communication because of that.

Change Response

The goal of change in product development is to produce the most satisfactory result for the customer. The Agile view is that change (and decisions in general) should be based on the best information you have at the time such change has to be made. Kent Beck has pointed that this isn’t a license to do something when “You should know better.” (More on this in Chapter 15 when we get to Technical Excellence.)

The iterative and incremental approach advocated by Agile approaches is nothing new, however. But it has not been as widely believed and spoken of until the Manifesto was created. As I noted in UAVP, Victor Basili and Craig Larman published “Iterative and Incremental Development: A Brief History” in 2003 in IEEE Computer showing how the idea, in some areas of development, “had been accepted 13 years before the waterfall model that Winston Royce defined in his 1970 paper” entitled “Managing the Development of Large Software Systems.” People fixed their thinking on that waterfall model, seemingly ignoring what Royce said about it. Royce’s son noted that his father felt a waterfall approach “would not work for all but the most straightforward projects” and that “the implementation described is risky and invites failure.” The bulk of Royce’s paper talked about an iterative approach being better. (He did not offer an incremental delivery approach, however, sticking with the overall waterfall (phased-sequential) model, envisioning iterations just between adjacent steps in the flow.

This is Royce’s third figure from his paper showing how he envisioned iterative development would ideally happen. He did follow this diagram up with another diagram saying “Unfortunately, for the process illustrated, the design iterations are never confined to the successive steps.”

Change, however, doesn’t just occur in product requirements, though other changes generally impact a project’s requirements/goals in some way:

· Changes in an organization’s market will impact the need for change in product plans/expectations as responding to competitors or consumers is needed.

· Technology changes will impact considerations for how a product might need to be implemented or cause the customer to feel they want to adopt such technology in the products they use.

· People will change as they move on to other roles, jobs, or companies. People who fill in behind them will have different ways to envision what needs to be done or how it will be done.

Even smaller changes can affect and require redirection of work that had been planned. Agility can be more easily handled if people involved in a project keep goals, rather than plans, in mind since the former should not change significantly and represents a value focus that plans may not as they are more likely to have to be changed. Customers and development teams can respond to that change without abandoning the goals.

Sometimes goals will change if the forces that require changes in plans are urgent, but we might think of that level of change as a “disruption” rather than a “change.” If this does happen, current work may have to be stopped and replanning will have to occur. The subject of descoping may have to be discussed between the development team and the customer about what content may need to be changed to accommodate the new, urgent change.

(I’ll come back to the topic of embracing change in more depth in Chapter 8.)

Following a plan

By pursuing agile planning, organizations must face the shock that they will not have a clear, though misleading, view of their project. As noted above, accepting this avoids the waste in trying to assume and behave in this manner.

However, regardless of being adaptable to change, at some level, people involved in the work need some basic sense of what needs to be done, at least in the near future. How far in the future people must know this is their “horizon.” For some people, this horizon is the entire expected amount of time to accomplish all the scope. This is the classic waterfall approach to planning. The Agile approach encourages establishing shorter, more incremental horizons so the work accomplished in one such period will better inform future work based on the learning gained to that point.

From a team’s perspective they have three such horizons:

· Each day, using the daily meeting to consider the state of their work and adjust their plans for that day and, perhaps, the rest of their iteration.

· Each iteration, using the iteration planning meeting to decide what they will do in the next few weeks.

· The “rolling window” of a few iterations used by the team and customer (or customer representative) to look ahead and do backlog review.

The customer can participate by taking a long horizon view (e.g., a year) and break it into smaller horizons such as quarterly. The idea of a good horizon is that it be useful in assuring the work to be done can be relatively stable. From a development perspective that may be no more than a month (a 4 week iteration) while the customer may feel that’s too short, at least to start. The basic idea is that the horizon allows for handling change easily by doing do as soon as possible.

I want to repeat some views on planning which I mentioned in UAVP. One example is von Moltke’s statement that “No battle plan survives contact with the enemy.” (He really said, “No operation extends with any certainty beyond the first encounter with the main body of the enemy.” The terse version seems a bit easier to grasp at a glance, though.) We also have Eisenhower saying, “In preparing for battle, I have always found that plans are useless….” The ellipses, though, represents the remainder of Eisenhower’s quote, which is “but planning is indispensable.”

Eisenhower’s full comment emphasizes how planning, if done collaboratively, establishes relationships that can deal more easily with changes that have to occur in the plan. People can build understanding of and confidence in one another that makes this possible.

Many aspects of Agile work help in planning and changes to planning:

· The entire iterative and incremental planning approach provides the short(er) horizon which, in turn, makes changes less risky and less costly.

· Having team members feel accountable for all the work to be done expands the capability of the team to get everything done to meet the plan or respond to changes in the plan with agility.

· Taking a lean/Kanban approach of finishing one thing before starting another, limiting multi-tasking and work in progress, means that when changes occur, fewer things in the midst of being done will be impacted by the change.

· And there are some purely technical practices which make change, at the product level, easier such as:

o Refactoring intended to improve the ease with which product changes can be made;

o Any forms of testing which can make people feel safer in making changes because they can be verified easily;

o As much automation as possible to ease the process of making changes (e.g., build and test, pushing work into and taking it back out of production, working in pairs to surface issues immediately at low cost and without allowing a problem to remain hidden until the change needed is more difficult).

It is truly difficult to plan complex work predictably far into the future. There is too much to consider, too many variations in what may happen. Even the fundamental task of writing requirements which drive the planning is difficult and then the plans and requirements change as the customer gets an idea of what they will actually be getting.

Traditional project management approaches create PERT or Ghant charts of the whole project since it this seems to provide control over the project by tracking tasks and checking them off as they are reported complete and comparing planned to actual dates for these to occur. However, using an Agile approach with collaboration between teams and customers will produce useful changes making such charts out of date too frequently. Then, the effort to try to keep them updated will increasingly be wasteful.

Shorter horizons for detailed plans can reduce that planning waste. Once a detailed plan is created there can be a lot of momentum and commitment to meet that plan. If the horizon is short, that momentum and commitment is a good thing because change isn’t as hard for things beyond that horizon.

The traditional approach to planning may be viewed as “target-driven.” You plan task goals and milestones, presumably related to achieving customer value, but the tracking is focused on the task targets. The Agile approach to planning may then be viewed as “value-driven.” Where action is focused on a consideration of the value being achieved. Changes in product or process are then made considering the impact on value delivery. (Chapter 13 will address this further.)

Sanjiv Augustine explains (in Managing Agile Projects), his ideas on agile project management which he says that APM is “the work of energizing, empowering and enabling project teams to rapidly and reliably deliver business value by engaging customers and continuously learning and adapting to their changing needs and environments.”

He speaks of “transitions” that (project) managers must go through to support such an approach to managing projects:

· Instead of the typical Work Breakdown Structure for planning take a Feature Breakdown approach.

· Admit there is no plan that will not have to change, so embrace adaptive, rather than predictive, planning.

· Related to such planning, take an adaptive rather than corrective approach to change.

· Focus on the effectiveness of day to day work rather than strict adherence to the plan.

· Realize that people are the loner term focus, not the plan.

· Traditional management tools can still work, but the key is how they are used in an Agile project’s evolutionary approach compared to a traditional one.

· The biggest transition is to give up the idea of command-and-control as “people do what they want to do.” Find “simple ways to work with this reality.”

Regarding planning, a few notable people have said:

· “Everybody has a plan until they get punched in the mouth.” — Mike Tyson

· “No battle plan survives contact with the enemy.” — a paraphrase of Helmuth von Moltke the Elder

· In preparing for battle I have always found that plans are useless, but planning is indispensable.” — General, and then President, Dwight D. Eisenhower

All of these people, and many others, recognized that despite all the time you spend planning, you cannot think of everything and not everything you thought of will stay the way you expected/hoped. Replanning, therefore, must be expected and understanding the overall objective(s), rather than just individual steps, makes that replanning easier. (Eisenhower’s comment says that the act of working together to plan, is more important than the plan. Having learned to work well together during the initial planning makes changing the plan easier because of the relationships built.)

Planning’s Focus

“The problem with specifying the method along with the goal is one of diminished control. Provide your people with the objective and let them figure out the method.” — L David Marquet (former US Navy submarine Captain)

Having carefully created a useful Product Backlog, it’s hard not to view it as the focus of iteration planning since much time has been spent trying to refine its contents to be clear to the Agile team.

This could be a mistake. It may distract attention from the larger objective(s) desired. You may have heard the phrase “Outcomes over outputs.” The Backlog contents are those defined outputs. The outcomes, however, are the true end for which we develop the outputs.

We have to ask what the objective(s) we, and the customer, have for developing the requirements in the Backlog. The iteration goal represents this intent which is why having a goal for an iteration is recommended.

To quote another well-known individual: “Goals transform a random walk into a chase.” — Mihaly Csikszentmihalyi (Hungarian-American psychologist who identified the focused mental state known as “flow”).

Planning with the iteration goal in mind will allow the Agile team to select those Backlog items which will achieve that goal. Such a goal will help the team:

· identify the important things to focus on during the iteration.

· use that focus to promote teamwork in achieving that goal.

· feel empowered to make effective decisions.

· demonstrate flexibility on what to implement to meet the goal.