ASQ Software Division’s ICSQ 2009 Conference Notes

I used to attend Agile and quality related conferences years ago and have saved notes from many of them. While some topics have become overtaken by events since then, there still seem to be some useful ideas that I’d like to pass along.

Keynote — Bill Curtis on “Quality in Multi-Tiered IT Applications”

Bill Curtis has been a researcher, practitioner and chief scientist in software methods and individual performance for many decades. He has worked at ITT, MCC (Austin, TX research consortium), SEI (as head of the Process program), TeraQuest (process assessment and improvement), and now at CAST Software. I have known Bill over the years during his time at MCC, SEI and TeraQuest, in particular coordinating (and applying the results of) his research activity in software design expertise for the company where I was working at that time.

Curtis started saying, “We’re moving beyond just what smarts and knowledge can handle.” By this, he meant the systems and their interactions have evolved (and continue to evolve) to where product (code) quality ideas are not enough to manage the desired results. Expertise in application level quality, i.e., how all the components interact, is what has the largest impact on system quality today. Quoting an ACM article (Jackson, “A Direct Path to Dependable Software,” CACM v. 54, no. 4), “The correctness of code is rarely the weakest link.”

Curtis pointed to problems with design choices that “pass (functional) tests,” but are (at best) inadvisable practice when scaled and must address non-functional production requirements. Presaging the end of day keynote by Joe Jarzombek, Curtis said that we need to be able to make dependability, assurance, etc. “cases” about our systems. That is, we should be able to provide evidence to support arguments that justify belief in claims about such non-functional requirements.

Curtis offered a few other ideas such as:

· addressing people’s ability to understand a system when changes must be made since he said 50% of the change effort in maintenance is devoted just to figuring out what the system, not an individual module, is doing;

· allowing (and training) testing groups to do true QA, indeed do Quality Engineering, which would require a broader involvement of personnel from testing organizations in the full lifecycle of work as well as not regarding test groups as “entry-level” positions;

· understanding the COO’s view on the need to standardize ways of doing things an organization does not compete on.

Finally, Curtis mentioned the Consortium for IT Software Quality “sponsored by a partnership between the Software Engineering Institute (SEI) at Carnegie Mellon University and the Object Management Group (OMG) to combine their industry-leading strengths in developing software-related standards and appraiser licensing programs.” As one of its activities, Curtis said, it will work to create more operational definitions of software (non-functional) quality characteristics (i.e., the “ilities”). The ISO 25000 series, which supplanted ISO 9126, has definitions, but the CISQ’s work suggests they are not viewed as operational enough.

Tom Roth — Psychology of Software Quality

Roth’s background is in embedded avionics software quality assurance where he is in charge of overseeing reviews, internal audits, testing, etc.

Roth started by saying we should “think of QA as people trying to influence the behavior of other people developing software.” Hence, his talk was about influencing people with regard to quality and the importance of QA knowing how to develop trust in developers since not everything can be reviewed. (Interestingly, an article in the ASQ’s main magazine, Quality Progress, for this month, is entitled “Trust, But Verify.”) But Roth cautioned against “enjoying the power you have” as an arbiter of quality, using knowledge of psychology to establish a collaborative relationship with development groups/management.

In discussing inspections and reviews, Roth noted that software engineering formalities, such as Fagan inspections, impart a level of discipline and, potentially, a group sharing of responsibility, which may not exist with individuals alone. Indeed, inspections turn the deliverable being inspected over to the group and the individual does not bear the full brunt of making sure quality is present in that deliverable. From an Agile perspective, I was thinking that, after a while, such discipline should become more internalized and less dependent on external rigor.

Some of the things Roth touched on were how:

· in relationships, differences often attract while similarities comfort, but trying to force sameness can destroy the attraction;

· inappropriate habits exert tremendous influence on further (and, perhaps even worse, expanded) inappropriate behavior;

· we are not 2, 3 or 4 different people, despite external appearances under different social circumstances, i.e., there is a congruence in behavior which, especially under stress, will reveal itself;

· people working alone can spend 2/3 of their time evaluating alternatives and 1/3 implementing a chosen alternative while two people working together reverse the balance, effectively quadrupling the productivity of one person alone;

· behavior [engineering] leads attitude [morality] — you can tell people what to do but not how/what to think, so work on behaviors/practices and allow the thinking to come along on its own.

The last two struck me as quite interesting, of course, from an Agile perspective.

Siegfried Zopf — Pitfalls in Globally Distributed Projects

Zopf is from Austria and discussed issues and problems he has observed working in multi-national, multi-cultural project situations.

Zopf began by making a distinction between distributed and outsourced situations, then, between minimally and large-scale outsourcing. Overall, it seemed as though he was emphasizing awareness of the level of appropriate control to apply to the different combinations of situations. For example, in a minimally responsible outsourcing situation — the work is confined to low-level design and coding — risk is low and pulling the work back in-house is not a problem, but the financial savings are lower. On the other hand, there is great financial advantage in outsourcing greater parts of the work, but much more local project management required in the outsourced locations. Zopf suggested allowing for 15–20% cost for such a “beachhead” operation.

Zopf also, in connection with any larger outsourcing effort, noted how planning must account for extra costs in project management, communication, travel, documentation, and knowledge transfer that would not exist in a fully local project. Thus, a company cannot take an estimation done assuming in-house work, then simply portion out parts of the work, “transplanting” the estimate and plans without adjusting for the differences.

And, for distribution matters in general, whether outsourcing or not, there are still issues related to national and cultural differences regardless of whether or not it is the “same” company. A couple examples of what he discussed are:

· two groups, for each of whom English is a second language, and the problems they can have trying to communicate using English which are not the case when at least one group is native English speakers;

· monochronistic and polychronistic cultures where the former view time as linear, compartmentalized, and value punctuality and the latter view time as more fluid, schedule multiple things at the same time, and even expect people/things to be late.

One final point in distributing work (with or without outsourcing) is process mismatch. Specifically when working with a high maturity (using the CMMI scale, Level 5) organization, that organization will find it difficult working with another organization that is not at least Level 3. In the reverse direction, a low maturity organization may find it frustrating working with the expectations and pace of a high maturity one.