Software Engineering Process Group (SEPG) 2000 Conference Notes

This Conference started about 13 years ago and has grown steadily over the years. This year, it drew ~2200 attendees from around the USA, Canada, Mexico, Europe, Asia, Australia, and South America. Both tutorials (half and full days) and presentations (45 minute sessions) are offered. I found this year’s Conference better than last year’s in both logistics (all sessions were within a short walk from one another) and session content.

What follows below are highlights of the tutorials and sessions I attended. Unlike the SM/ASM Conference, no paper proceedings were provided. Instead, handouts were available at each session. I collected quite a few, even from sessions I did not personally attend. Also, this year I was a speaker at the Conference on the topic of achieving ISO 9001 registration and CMM Level 3 assessment concurrently in large and small organizations.

Tutorials –

Each year, the Conference begins and ends with tutorials and I attended four. I also collected the materials from a couple others on SEI’s process planning recommendations and their work with high maturity organizations. The former I mention briefly at the end of the Tutorials section while the latter is mentioned in conjunction with a regular parallel session given on the same topic.

Action Planning for SPI (Kim Caputo, Unisys)

The object of this tutorial seemed to be to distinguish between action items and actions plans and to describe how to develop appropriate SPI plans for the various lower organizational maturity levels. Caputo characterized the levels as follows regarding their belief in SPI:

· Level 1 — A “fast-cycle” organization that cannot afford the “additional and unnecessary overhead” of SPI planning.

· Level 2 — An organization that “doesn’t have time to do it, but can’t afford not to” either.

· Level 3 — An organization that feels it must “improve faster and make it easier to improve.”

For the first of these, she recommended a plan that would distinguish between:

· desired results (and the source of the goal/objective), prioritized by impact;

· what is needed to achieve the results, prioritized by urgency; and,

· activities (with appropriate milestones), prioritized by feasibility.

At the next level, she recommended adding risk analysis for starting out, dealing with mid-point crises, and achieving closure. She described some typical concerns heard and behaviors observed at each of these points in an SPI effort:

Caputo also noted people’s desire to gravitate toward where the attention is being placed. Hence, she recommended making sure senior management paid attention to successes, dealing with criticism of failures less visibly. Caputo was quick to note that this did not mean ignoring the problems or sweeping them under the rug. She did say the point was to get people to move to something new based on the positive attention things get when they work rather than the difficulties being encouraged while trying to make them work.

At the higher Level, the following chart was used to suggest how different parts of SPI activity would interact.

At this Level, Caputo stated that a more detailed plan, covering the following seven items, should be pursued since the organization wants to change and is really seeking to improve how they change.

1- Overall Objective (What problem are we trying to solve?)

2- Desired Results (What do we want to accomplish? What needs to change? How will we know we’re done?)

3- People (Who will be affected and how?)

4- Change Factors (What concerns will people have and how will we address them? What leverage can be built?)

5- Scope Boundaries (What issues must be addressed? Which ones should not be included in this plan/effort?)

6- Deliverables (What has to be an output of this effort? Who will produce these? How?)

7- Actions (What milestones are needed? Who will carry them out? When must they be done?)

Interestingly, Caputo argued against what some others at past SEPG Conferences have suggested. Others seem to have felt that assigning a middle management “champion” to each distinct process area makes sense. Caputo suggested that, at least in less mature organizations, this could create competition between champions to just get their goals accomplished without understanding the consequences to others’ goals. Instead, she suggested spreading the responsibility out for each SPI effort so middle management would work together.

Building a Collaborative Software Quality Assurance Function (Ken Dymond, Process Transition International)

Dymond is the author of one of the most popular texts on the CMM — after the “official” text from the SEI. Because he uses diagrams and stick figures liberally to illustrate the CMM key process areas (KPAs), his book is known by the CMM community as the “dummy’s guide to the CMM.”

Dymond began by quoting the CMM on the SQA function as having “the freedom to be the ‘eyes and ears’ of senior management.” This contrasts how the CMM, and most formal process structured views of software development, regard SQA. Most of the industry looks at the term and thinks of testing as quality assurance. However, the CMM views SQA as the effort to objectively verify that software development activities and products meet accepted organizational standards, procedures, and requirements. In some organizations this is largely an audit function; in others, it is more consultative. Dymond’s presentation focused on the latter treating the audit aspect of SQA as its information gathering mechanism.

Dymond began by describing the need to “engage the project,” explaining the role of SQA as trying to locate problems in the process, identifying what improvements might be needed, and noting what SPI activities could be conducted. Though the SQA function would not necessarily be where SPI initiatives are located, their role did include helping to identify opportunities for improvement, not just determine compliance or lack thereof.

The rest of the session involved examples of SQA planning and verification (audit) activities.

Dymond did stress that there was no justification for compliance auditing if there was no clearly accepted and understood standard against which to perform such audits.

Influencing and Working with Middle Management to Implement SPI (Chuck Myers, CRM Consulting)

Last year, I attended a Birds-of-a-Feather session given by Myers on the topic of working with middle management. He indicated he was submitting a tutorial proposal for this year, so I wanted to be sure to take it based on last year’s brief discussion.

What interested me so much in his topic was that I had identified the middle management “bottleneck” with regard to conducting SPI activities during my industry visits/studies for Bellcore about 10 years ago. As Myers noted in his talk, middle managers get to be where they are and are rewarded for successes using behaviors and approaches to project work which many SPI efforts seek to change. Thus, middle management often views SPI work as not having much in it for them, indeed, may view such activity as a threat to their positions and effectiveness.

Myers’ tutorial was about recognizing and dealing with root causes of resistance. That is, recognizing why it is that middle management seems to resist process improvement. Some possible explanations given were that middle managers

· are simply overwhelmed with demands from their management, customers, product development issues, crises of all kinds, peers, administrative duties, their own career concerns, their families, personnel matters (their own subordinates), and a variety of change initiatives that compete with those SPI proposes.

· are in favor of the SPI ideas, but are just not effective sponsors of the work [Myers made the distinction between the roles of managing, leading, and administering, not all of which every manager is going to excel at].

· are really resisting something other than the SPI effort such as the people implementing it, i.e., they are not folks whom they respect/trust.

· truly do not understand (or misunderstand) what SPI is all about, perhaps due to prior negative experiences with changes related to methods/process.

· understand, but see little value for the effort invested in SPI work [Myers discussed extrinsic and intrinsic motivation, noting that “If you can’t get commitment (heart and soul), then compliance (the posterior) may have to do.”]

· are just pessimistic about the whole SPI effort, i.e., they’ve seen it all before, learned the buzzwords and how to use them in appropriate contexts, made what they do seem like what was wanted by the SPI effort, and just waited out the program until the next management reorganization.

A good bit of Myers’ advice (and tutorial exercises) had to do with how to put yourself into the place of the middle manager to understand what they are up against. On the other hand, he made it clear that part of the responsibility of senior management, if they are serious about improvement, is to make it clear they expect middle management to come to understand what SPI is trying to achieve and work constructively toward it.

Keynotes –

Each day of parallel sessions started with two keynote addresses and included an invited speaker at lunch as well. The first of the keynotes was on Boeing’s commitment and efforts for SPI. The next was by Steve McConnell on the culture of software development and the emerging efforts to create a formal professional status for software engineering. The first lunch speaker was Stan Rifkin, formerly of the SEI, who spoke about corporate culture and three ways to focus that culture. The next two keynotes were by EDS and Tata Consulting Services (India) on their respective SPI efforts. (I did not attend the other lunch keynote.)

Boeing’s SPI Efforts (Scott Griffin, Boeing CIO)

Boeing was one of the key sponsors of this year’s Conference. For example, the Conference industry chair was John Vu who heads up Boeing’s SPI efforts throughout the organization both on the military and commercial sides. Griffin is Boeing’s CIO and spoke about the ongoing efforts at Boeing to grow their process maturity. Besides the CMM, Boeing is also engaged in trialing the Personal Software Process (PSP) and Team Software Process (TSP) at various organizations within the company. The PSP takes CMM principles which individuals can practice and shows them how to do so to improve their personal development process knowledge and skill. The TSP applies the CMM concepts to groups of 3–12 people (who have been exposed to PSP already).

Griffin also discussed some data to illustrate the impact SPI efforts have had. For example, employee satisfaction improved from 74% before SPI efforts to 96% afterwards from a mean of 5.7 (Neutral to Not Quite Satisfied) to one of 8.9 (Very to Highly Satisfied). Defects were lowered by 75% and test time was reduced 94% among PSP/TSP trained groups while working on ever larger systems (up to 2.36 times larger as measured by lines of code).

As with many other speakers, Griffin emphasized the importance of the visibility of his support for SPI as a senior manager.

Software Development Culture and Professionalism (Steve McConnell, Construx Software)

McConnell is the author of several well-known “guides” and “handbooks” on practical software development and process change matters. He has consulted with commercial software development organizations, such as Microsoft, and is the current editor of IEEE Software. His talk used the history of the California Gold Rush as a metaphor for the current state of software development practices in the USA.

McConnell was pointed out that it is usually the big successes that we hear about, just as with the big gold finds. He stated that, “gold rush approaches to software development insulate companies from the natural consequences of using inefficient practices (for a while, anyway)” with the effect that “occasionally these practices work.” But best practices in software have been known for many years, decades in some cases. Why aren’t they used? Again, the short-term belief that “good people” can overcome all other obstacles, even when this tends to limit their true effectiveness. It becomes, indeed, a “red badge of courage” for individual’s to overcome such limitations and inefficiencies as proof of one’s superiority to the process and other practitioners.

McConnell offered the following Pony Express ad from 1860:

“Wanted: Young, skinny, wiry fellows not over 18. Must be expert riders willing

to risk death daily. Orphans preferred.”

then he noted the following ad for software developers from the Seattle Times in 1995:

“We realize the skills, intellect, and personality we seek are rare, and our

compensation plan reflects that. In return, we expect TOTAL AND ABSOLUTE

COMMITMENT to project success — overcoming all obstacles to create

applications on time and within budget.”

McConnell then noted one of the world famous programs in technology and practice diffusion: the US Agricultural Extension Service. The Service employs 17,000 people in local offices throughout the USA in support of 3.8 million form workers. He contrasted this to the SEI’s program with 300 people centralized in Pittsburgh to support 1.8 million software developers. That’s over a 40 to 1 advantage for the farm workers! McConnell said this illustrates the slow diffusion of best practices as a national commitment.

McConnell closed by describing the various efforts, largely led by the IEEE-CS and ACM, to provide a professional body of knowledge, and, ultimately, a certification/licensing program for software engineering. [I am, by the way, involved in this program, having been a reviewer for the Process and Quality Assurance areas.] McConnell noted that no other profession affecting the safety and business success of the general population is as unregulated as is the software industry. This is changing, however. ABET, the certification board for university engineering curricula, is beginning to consider software-engineering programs. Also, the state of Texas has already instituted a licensing program for software engineers, as have the Canadian provinces of British Columbia and Ontario.

I spoke to McConnell briefly after the keynote and asked about his knowledge of the UCITA which is being considered, state by state, as a change to the way software is viewed commercially. He was not really aware of this legislation, but it has already passed in Virginia and attempts to put software into the category of purely licensed products, making shrink-wrap licenses, and their disclaimer of fitness for use, legal. This would prevent legal action against software developers for software that fails to perform. Many commercial software producers have been claiming, through lobbyists, that it is too hard to produce error free software and too costly to provide the support needed when it does fail.

Corporate Culture and Focus (Stan Rifkin, Master Systems)

Rifkin was at the SEI for many years and developed the guide used by SEPG’s. He has been at every SEPG Conference since the beginning and has personally helped in the formation of over 60 SEPG’s around the world. He identified three models of market focus that affect corporate (and SPI) culture, which he stated, come from a book called The Discipline of Market Leaders:

· Operational Excellence (characterized by high quality, low cost, innovative process)

· Product Innovation (characterized by a constant stream of new products and patents)

· Customer Intimate (characterized by an interest in ever-increasing “wallet share” of a customer’s business — “schmoozing” with clients, often to make them more satisfied with the fact that this sort of company is not the highest quality, lowest cost, or most innovative in products)

Rifkin suggested that, though they cannot totally abandon any of the basic characteristics of any of these models, companies can experience great difficulty trying to view themselves as essentially more than one of these at a time. He stated that the CMM and much SPI effort are focused on the first model: operational excellence. This can explain resistance to the CMM and SPI in other kinds of companies.

However, elements of what the CMM has to offer are important to companies who adopt one of the other two market models. For example, to be quick to respond to customers and provide end-to-end solutions, effective product architecture and design standards and configuration management are critical to make sure products are easy to change reliably and easy to link together. For an innovative company, lightweight processes, but solid project and risk management are essential.

Tata Consultancy’s SPI Work (S. Ramadorai, CEO)

This had a marketing feel to it, but there was some “data” in this presentation. Tata has a number of organizations at Level 5 on the CMM scale. Ramadorai noted that this has resulted in reductions of ~24% in the effort to carry out a change request, 70% in the time required, and a corresponding increase of 2.5 times in the number of requests that can be handled. He claimed that accurate historical data and forecasting has been what contributes to this the most.

Ramadorai also showed charts which suggest Tata’s average effort devoted to rework has been cut by a factor of 4, e.g., from 80% to 20% in some areas and from 20% to 5% in others. Schedule slippage has been reduced to less than 0.09% with reduction in project management effort as a result.

It was also clear that Tata has a significant new employee training/indoctrination program lasting many weeks at the outset and continuing for the whole first year. Employees are also very active professionally in the IEEE-CS and ACM and Tata maintains active programs with universities in India and the USA.

Parallel Sessions –

Measuring Software Quality (Watts Humphrey, SEI)

As with Humphrey’s talk at the SM/ASM 2000 Conference, I was very pleased with his presentation at this year’s SEPG Conference. His major claim was that “when companies manage software quality, they can typically reduce test defects by 10 or more times” and reduce test time accordingly. He also noted that, because so much attention is focused on finding defects in the software after it has been coded and delivered to test, there is an inordinate focus on hiring (and training people to be) clever coders rather than solid architects and designers.

Humphrey emphasized checking one’s code through reviews and inspections rather than relying on the compiler to catch syntactical errors. This is an aspect of what is known as the “cleanroom” approach to software development. He stated that random typing errors alone can produce about 40 defects per thousand lines of code (KLOC). However, it is the compiler’s job to try to generate code from the input, not assume the input is bad and find defects. If all defects are counted, even an experienced programmer can inject 100 defects per KLOC. Humphrey said that his data from the PSP training program (which focuses on existing practitioners, not students without work experience) shows 10% of syntax errors are not caught by the compiler and make it into testing. This is true because they are correct enough for generating (some sort of) code but produce results not intended by the developer. Then they have to be uncovered by testing and the failure may show up far from the cause.

Similar things happen in maintaining code and trying to fix bugs. Humphrey called this “provocative maintenance” since the effort injects defects while it attempts to fix others. To overcome this, the PSP program asks developers to record data about their own develop practices and behavior. The claim is that it takes little more than about 20 seconds to record a defect and less than 1% of development effort to track and fix them in development. Various historical studies indicate that the costs to find and fix defects increase by an order of magnitude from design to coding and then again from coding to test (and then further from testing if bugs make it into the field, ignoring costs to clients).

Humphrey stated that the goal of testing should not be to find bugs that ought to be caught by more care earlier in the process. Testing should be used to focus on determining that functionality and capability of interest to the client is validated, not just to make it work and get it out the door by a deadline.

Integrating Project and Quality Management (EDS)

This presentation was about conducting the management of quality, though an SQA program, in the same way one would manage a development project: through defined tasks and a project plan with specific milestones. [We have been doing this in Interact with regard to the program to develop their quality system and prepare them for ISO 9001 auditing and CMM assessment.] This presentation provided some examples of the templates used, at least within one EDS organization, to define the SQA plan, identify templates and procedures to be applied, track SQA activities, perform SQA verification/validation (audit) activities, and report results.

Understanding High Maturity Organizations (Charles Webber and Mark Paulk, SEI)

This talk was a repeat, to some extent, of the material Paulk presented at the SM/ASM 2000 Conference; however, Paulk also held a tutorial on this subject and I got a copy of the material from that. This presentation (and the tutorial) focused on behaviors by Level 4 and 5 organizations. using the following scale to discuss them:

· what was “typical” (90%+),

· what happened “mostly” (60–90% of the organizations),

· what “many” did (40–60%), and

· what “some” did (more than one organization anyway!).

One thing Paulk made clear was that “building a product the customer wants to buy” is critical. Attaining high process maturity is a way to assure that managing the process of doing so is predictable, allowing market windows, costs, and functionality to be reliably targeted and met.

The tutorial posed the following questions as important in characterizing organizational behavior with regard to SPI and maturity:

· Which critical processes are (not) being quantitatively managed?

· Are these processes being measured and controlled at the process step level?

· Is day-to-day decision-making based on quantitative analysis (as appropriate)?

· Can the organization demonstrate actual business benefit (SPI trends)?

· How are quality goals prioritized and conflicts resolved?

Experiences Implementing an Electronic Process Guide at Bank of America (Greg Jones, Testing Capabilities Team)

Jones noted at the outset that the Testing Capabilities Team, though an outgrowth of the testing organization, was now more strategically focused on the whole development lifecycle. He discussed a variety of tools with which he had experience as well as noting efforts by Fraunhofer Institute (Germany) and at the SEI to develop tools and approaches for putting process/methodology guides on corporate intranets.

The approach taken emphasized graphical, rather than test, based guidance with “hierarchical drill-down” to provide the level of detail desired by the user. One constraint was that only a basic browser would be required, i.e., no “client” software, since this was designed to be made available to all of BofA: 21 states and Washington, DC; 37 countries; 4 major data centers.

To create the web site, Jones noted they used available diagramming tools wherever possible as long as they could generate HTML outputs or other web-based formats. This included basic PowerPoint diagrams, MSWord to produce HTML versions of procedures, and a more complex tool from the Platinum suite (Process Continuum, which they happened to already have licensed).

A great deal of the effort (2–3 weeks of two people full time) was devoted to taking the test versions of the process guidance they already had and producing web-based versions from which they proceeded to create the final web site.

[I don’t know the potential “politics” of contacting Jones and trying to find out more about what they have done, but it might be useful to explore their effort if it is not considered a problem to seek out a client for “advice” about process documentation.]

Executive’s Little Instruction Book for Implementing SPI (John Maher)

Maher’s intent was to produce a set of short, to the point guidance materials for senior management faced with SPI decisions. One immediate bit of advice was not to “skimp on costs” at the outset and “buy SPI like good boots.” Maher suggested that 5–10% of an organization’s effort loading (if all time is actually recognized) might be needed at the outset to “make progress in a reasonable time.” This contrasts with the 1–3% often recommended over the past decade since this was usually the cost of the formally recognized SEPG once it was established.

Maher’s next bit of advice was that a senior manager is likely going to have to shift focus from operational to strategic concern and “lead and manage as though you were already an improved organization.” He said it was hard, but I think it is an excellent point not often made by others in such presentations. Senior management must act like they expect improved behavior, not hope the organization can get around to it in between other things. But also, such managers should be prepared to move deliberately and continuously, not expecting all needed change to occur within the year.

Maher mentioned culture change, as do many others, and said it was senior management’s job to see that this happens, starting with them. What this may mean, however, is that some of the “best” people (the “heroes”) may leave as they are addicted to “saving projects” and maintaining some level of crisis to prove their value. Maher said to “beware of rewarding firefighters” as they may “become closet arsonists.” (Interestingly, Myers suggested this was really too hard to take on directly, unlike the advice from most others. Myers argued for “alignment” rather than “change” since “culture will win every time.”)

As a senior manager “if you don’t have a deep compelling personal reason to start [SPI], don’t,” Maher advised. SPI is not about “programs.” It’s about how people want to change the way they work on a daily basis. If senior management sees no need to change how people are working, i.e., they feel things are “fine as they area,” then there is no “compelling reason.”

Finally, Maher noted three “walls” likely to be encountered: the senior manager’s team who will not see what’s in it for them; the middle management who will have a similar view; and, oddly enough, customers who will have become used to playing the system and not want anything you do make them change their behavior in any way. Indeed, customer satisfaction may even go down, Maher claims, until it is clear you can “deliver what you say you will” for what you say it will cost when you claim it will be ready. (Others recommend involving clients in the SPI effort by making sure the priority of process change focuses on things they feel will help. This should not be confused with “faster, better, cheaper” demands unassociated with changes that the customer will support.)

Change Realization: Striving Toward Quality Goals (Jenny Flowers and Alicia Esposito, Telcordia Technologies)

Telcordia was formerly known as Bellcore (and, before that, part of Bell Labs). They achieved Level 3 on the CMM scale in the late Fall of 1996 while I was still with them and was an internal SPI consultant for most of their large projects. Last Spring, they achieved a Level 5 rating. Since 1997, they have been speaking at SEPG Conferences about their experiences since they were the largest single organization ever to undergo CMM assessment. (Bigger corporations have had many assessments in their various divisions, but Telcordia’s were the largest single assessments ever recorded.)

This presentation was somewhat “generic” in that it presented an improvement/change approach that was really not used early in Telcordia’s process change efforts. However, it had some useful advice worth passing along.

The first of the key points was the need to “establish a sense of urgency.” This meant identifying the reasons an SPI effort was critically important, e.g., customer dissatisfaction with problems in delivery of products or services or employee dissatisfaction with internal development and process expectations. In Telcordia’s case, this initially involved a management edict (the “benevolent dictatorship” approach to SPI) that people would become involved in the new way of doing things or find other places to work, quite frankly.

The next step was to focus the SPI effort at a senior level, not expect it to succeed as a, largely, bottom-up activity. There are always too many other things, especially middle, managers have to focus upon. The SPI effort must be integrated in these other things, but clearly driven from senior management. The SEPG and SQA efforts have to come from that direction with participation by

senior and middle management in guiding and decision-making if anything is to be accomplished without round-and-round discussions that last for months. (In my own talk I note the tendency for management to allocate people who are “expendable” to such activities, telling them “not to do anything that will hurt” the particular organization they come from. They are, therefore, not respected by their peers, nor empowered to make useful decisions.)

There is a whole series of steps, but starting our correctly, in my experience, is what matters most. These initial ideas get that going properly. The rest is about planning and managing the effort properly and communicating what is going on effectively.

SPI Across the Pacific: How the CMM is Accepted in Japan (So Norimatsu, Nomura Research Institute)

I was quite interested in hearing someone from Japan, even if from an R&D institute, discuss the CMM’s acceptance in Japan. It was interesting, therefore, to hear Norimatsu claim that the US has now surpassed Japan in providing better and cheaper manufactured goods. However, he noted, based on Watts Humphrey’s work that the US and Japan both produce about the same overall low level of software quality — with notable exceptions, of course.

Norimatsu spent a great deal of time talking about the effort to translate CMM materials. Indeed, their availability outside the USA along with appropriate training was, for a long time, a major problem for acceptance of the SEI’s work in this area. Today, most users are still the “early adopters,” i.e., pioneers. Many, Norimatsu stated, are already ISO 9001 registered.

Thus, my overall impression was that the exposure to the CMM was relatively small, but being pursued very deliberately where it did exist, though perhaps due to translation efforts and cultural differences more than anything else.

[A talk I could not attend was another by Stan Rifkin on “Exporting the CMM for Software: The Ugly American” in which, according to his handouts, he suggests problems encountered, especially in France, trying to bring the CMM to Europe. Stan seems to have spent a good bit of time working in Europe and gives the impression many American companies propagate the image of ignorant arrogance in dealing with superficial cultural knowledge abroad. My work on the US TAG, where the ISO 15504 Software Process Assessment standard is being developed, suggests significant competitiveness between nations when it comes to acceptance of models/standards developed in the USA. Discussion with some Australian delegates suggests the USA’s reluctance to participate more actively in international software standards efforts until very recently has been a problem for acceptance of things like the CMM. Were it not for the influence of the Department of Defense, foreign interest might have waned long ago. Now that it is growing, however, some US firms find themselves at a disadvantage, initially, when they do not understand the implications of things like the ISO/IEC standards and models like the CMM. Asian and European companies are acquiring this knowledge as well as being aware of their own regional/national models such as Bootstrap and Trillium.]

Managing Schedules Using Statistical Process Control (SPC) (Ivan Handojo and Bill Pitterman, Telcordia)

This presentation briefly described an approach to monitoring project schedule/milestone adherence through statistical techniques. Very simply, they took the number of total days late on all milestones over the total number of milestone days to date (adjusted by priority) to provide a control chart of whether projects were within or outside accepted historical schedule performance.

Under the covers, I gather there is a lot of data manipulation going on, but the idea is related to the earned value concept which was discussed by one speaker at the SM/ASM 2000 Conference. Both of these approaches, Telcordia’s and the standard earned value one, attempt to use project time accumulated compared to task achievement . The assumption is that all tasks are needed and add some level of value to the end product, either through actual production of required deliverables or through assurance that they meet requirements in some way. On this basis, the milestone time can be used to determine how “well” a project is doing.

Why Projects Need Process Standards (Lewis Gray, Abelia Corporation)

Many people, including from the SEI Process Program, note that Level 1 organizations can produce good software. The issue always is whether such organizations can do it reliably, i.e., again and again. This talk was about why ad hoc process is dangerous even though it is often successful, using the analogy of the dangers of mountain climbing under stressful conditions.

Gray quotes from a book documenting failures (e.g., deaths) in climbing Mount Everest which states “Everyone thinks that they’re thinking very clearly up high, and yet, by virtue of going up into that atmosphere, you greatly increase your likelihood of making an error.” This is due to the effects of “hypoxia, or lack of oxygen to the brain.” Without an effective planning and agreed to practice for climbing mountains, people are far more susceptible to hypoxia, no matter how individually talented they may be.

Performance concerns related to hypoxia resemble those experienced in software projects. People reach a breaking point where their capacity to perform is reduced. They think they are doing okay, denying the need for more care. The breakdown occurs at the worst possible time. Hypoxia is going to occur, the planning and agreed upon practices are there to deal with and compensate for it, not expect to prevent it.

Gray talks about stress and job related failures due to stress. Watts Humphrey has stated that “the intellectual challenge of software is unsurpassed.” Perhaps this parallels the physical challenge imposed by mountain climbing? Under stress and duress, ad hoc processes may likely be the worst thing, yet it is common for organizations to “throw out process” under just such circumstances. This may suggest the processes are not very good to begin with, in which case they need to be changed during periods of low stress, not simply abandoned.

Conclusions –

As usual, speakers emphasized the role of senior management is showing clear and consistent support for software process and quality improvement. Everyone indicated that success in implementing an effective SPI effort was driven by senior management and involved committed middle management. By “committed,” speakers meant personal participation by those managers in the effort. Senior managers had the SPI effort on the agenda of their regular meetings and expected middle management to report on progress. Those tasked with doing the day to day work were drawn from respected members of the staff and their SEPGs reported directly to the senior managers.

It was routinely noted that the measurements and metrics put into place to track progress were a significant aspect of successful SPI efforts. Without this data, showing objective evidence of SPI success would be impossible. Early in SPI efforts, such objective data is vital since subjective response early on will be resistance from those unwilling to embrace change. As Edwards Deming is quoted as having said, “In God we trust. All others bring data.” There are a variety of simple, inexpensive approaches to metrics which companies could try that lead to “Aha!” kinds of discoveries that would interest people in the learning that can come from such measures. A visible support for decision-making based on such data could be an important SPI step.

Finally, companies should consider their level of encouragement for and involvement in professionalism and professional development in software engineering. By this I mean a specific focus on what this means as compared to other professions both technically and ethically. Again as attributed to Deming, quality in product and practice is not simply about “doing one’s best.” It is about consciously striving for continual improvement in what one understands their “best” to be whether it is at the individual or organizational level. Local support for Project Management Institute activities is an example of something that could be replicated with the IEEE Computer Society, Association for Computing Machinery (ACM), or the Software Division of the American Society for Quality (ASQ). Lunchtime colloquia and the invitation of guest speakers from various industry sources would be another evidence of senior management interest in the professional life of their employees. Some companies, like many who presented at this year’s SEPG Conference, have even added professional activities as an aspect of employee performance goals and annual reviews.