Home | Login | Recent Changes | Search | All Pages | Help

ExtremeProgrammingDiscussion

an email from the ExtremeProgramming mailing list:

This is a crucial thing that both Kent and Ron have said in the past few days: requirements generation and validation are outside the scope of XP.

This is important because it puts a nice box around the toolset that we call XP and shows us where to fit it in within our projects. It is purely at the development end of things, dealing with the Customer and Developer. At the other end, where the Customer deals with Users, we need to do something else.

As I have said before, I hope that XP itself does not try to expand to fill the void. XP is a very good toolkit for the Developer/Customer end of things, and I think that it will only get messy if it tries to expand...

I'm adding another book to the XP minifaq: "Adaptive Software Development". JeffMcKenna said that he uses it when consulting as a framework for adapting XP for his client company projects. It's a really good book

I just made a link to AgileSoftware , rather than fit this into an unrelated session description...

New book Questioning Extreme Programming by Pete Mc Breen, author of Software Craftmanship. This book tries to clarify what kinds of projects XP is suited for, and explains XP compared to other processes.

Kent Beck has a forward where he says "Pete claims that the more he looks at XP, the smaller he sees its scope. I see just the opposite. I won't refute his arguments point by point -- this is a forward and I'm supposed to be polite. I will suggest that as you read this, you keep in mind [...] 'the customer' doesn't mean one person, it means a team, as big as or bigger than the development team."

Good reading so far, but not at all "anti-XP". Though it does have a black cover, a contrast to the first XP book having a white cover.

--KeithRay 2002.08.01

In my opinion, XP applies to projects of any size, as long as you can decompose the project into XP-size pieces and each piece has a "customer" - either a person who knows or a document that says. And, if you can't do that, then you're not going to succeed with any method.- JerryWeinberg 2002.08.02

There are combinations of Scrum and XP called "XBreed" and something else...

--KeithRay 2002.08.02

PS: Jeff Mc Kenna has suggested doing an "ExtremeHour" if people are interested. More details on that page.

Keith, I am trying to work out the 500 people situation you offered as a real possibility:

... to coordinate up to 500 people on one large project. Each lowest level team is around 10 people, who have their daily 15 minute scrum meeting reporting progress and roadblocks. One representative from each lowest level attends a "scrum of scrums" meeting with other reps, and so on up a hierarchy of scrum meetings.

So, how does this work? ... I mean in detail -how many levels are there? Is A coordinating person added at a level (and adding to the total number)? I ask this because I once consulted on a project of 200 engineers (not to mention other project participants) where depite the best of intentions, communcations failed at some level. In fact, getting every level of communication to work in such a large project seems to defy the odds.

- BeckyWinant 2002.08.04

"Continuous Care vs Initial Design" at <http://www.objectmentor.com/resources/listArticles?key=topic&topic;=Design>

Various essays at <http://www.objectmentor.com/resources/listArticles?key=topic&topic;=Extreme%20Programming>

KeithRay 2002.08.05

Thanks for the references. Here is what I imagine: teams doing the work, build whatever is required for their piece - which is an iteration doing the smallest amount of work needed to do something useful. In their next iteration, they begin their work on the next smallest useful thing, but they consider how these newer things might break the design used to implement the first iteration and then refactor their design.

I can see how a design evolves from this - in a sense, the vertically through time. What I have a more difficult time seeing is how the horizontal design process occurs across mutliple teams working in closely related domains.

Perhaps the answer is in how the teams and their focus are built. The organization of the teams and what they are assigned to do could lead to a particular design.

Is there more to look at relating to how extreme teams or scrums are put together in the first place?

BobKing 2002.08.06

• How to approach a huge project in Agile manner (Do The Simplest Thing That Could Possibly Work, then evolve)
• How to manage this both horizontally (subsystems / subprojects) and vertically (time dimension).
• When & how to split & spawn.

I'm game ... anybody else?

--BobLee 2002.08.06

In The Tipping Point by Malcolm Gladwell, there's a reference to "the magic number one hundred and fifty" which makes me doubt that anything which occupies 200 or 500 people can properly be called a project. This seems to be the threshold size after which a single, coherent culture breaks up of its own accord into smaller independent tribes. This is in addition to the size/complexity dynamic problem Jerry already mentioned.

In the less fashionable book, Kent also mentions a dynamic which might explain why such huge projects exist - "I manage a 500-person project" gets more points on the golf course that "I manage a 15-person project", irrespective of the relative productivity of the two projects (in units of value, of course, not in LOC or tons of paper).

With Ron Jeffries, who has been known to wonder about "agile elephants", I doubt that a 200-person "project" can really be extreme. As an XPer, my recommendation would be to find 19 worthwhile projects to do, and run each with a 10-person team, the top 10 developers finishing up the original 200-person project.

This being the AYE Wiki and not an XP Wiki, though, I should not be content with pontificating on the subject of size. I have no particular qualification to pontificate on the topic, never having worked in a team larger than 6. That might yet happen, so I'd want to ask how one can be an effective change agent in such large organizations. How does one go about identifying stakeholders, mustering support for change, formulating problem statements etc. when one's observations must respect each of 200+ individuals ?

LaurentBossavit 2002.08.06

I wish I could answer questions about multi-group XP projects, but I haven't done that. This guy has done it: . I'm sure he would respond to a politely-worded email.

See the link above for the full message. Here's a bit:

Your development environment and ours are very similar. We've been using XP practices (I hesitate to call all of our development XP as some of our teams do XP and some of our teams follow some of the practices). We have 100+ developers in 5 locations (used to be 6), separated by 8 time zones (used to be 16), and 3 countries. The one difference is we are a product company with over a dozen products. We've got some good infrastructure in place with multi-site Clearcase, distributed testing tools, etc. I've commented in the reply on things that I've experienced as we've moved towards agile development models and XP in particular.

Charlie

KeithRay 2002.08.06

I think that the Agile part would reduce a 500-person project to a much more manageable N, probably under 150 given the amount of overhead that XP / Agile omits. Martin Fowler's Thoughtworks is also pushing the envelope some for XP.

--BobLee 2002.08.06

This splitting makes a hierarchy, if you like to think of it that way, and the splitting task cannot, I suppose, be done in a terribly agile way. But once it's done, you eventually wind up with lots of agile projects and, hopefully, agile supervision of them to keep them coordinated.

The way this splitting is usually done is not even noticed. Most projects (hopefully) don't build their own computers, create their own operating systems, provide their own power stations, drill for their own fuel for those power stations, and so forth. We assemble projects from other projects that have come before us. Without that, we would be helpless to do anything. - JerryWeinberg 2002.08.06

open space wiki

<http://monet.objectmentor.com/cgi-bin/openspace/wiki.py?OpenSpace>

Agile/XP related questions and answers... sounds like stuff written up in a workshop. I might copy some questions to there, to see if we can get some answers.

--KeithRay 2002.08.11

see VeryLargeAgileProjects

The authors fear XP, especially pair programming, big time. "We haven't tried it, but it seems obvious that..." The bibliography of properly disciplined programming literature is revealing.

--BobLee 2002.08.20

i am certainly willing to publish any kind of misinformation about XP in order to shield the software engineering profession from the misinformation issued by XP advocates.

I haven't really seen any misinformation about XP published by anyone who has actually practiced it.

Sometimes Kent or Ron will say something provocative, such as "Daily builds are for wimps," or people don't believe Martin Fowler when he reports that an on-going XP project hasn't had a bug report in four months.

Or people don't believe Bob Martin when he says that an architecture can evolve during the project and doesn't have to be designed up-front -- there's a guy on usenet right now, with NASA experience, practically calling Bob a liar because he says he done it many times.

KeithRay 2002.08.20

Why don't you change the name to "Moderate Programming"? (MP) That gives it a nice military police sort of feeling, which ought to satisfy many of your more vocal opponents. - JerryWeinberg 2002.08.20

I'm not surprised someone from NASA would be skeptical. Having lived through various methodologies (and XP is just one), I have realized that ALL methodologies can work eXtremely well depending on culture and the type of system. I find there is much good advice one gets from being open to it all, and you need to adjust the methods to fit the organization.

With many NASA subcontractors I have supported their formal modeling and executable modeling practices letting them do a LOT of upfront simulation, saving millions and sometimes billions of dollars.

Relating to your comment about XP and the "misinformation" postings", I responded to an article that Scott Ambler wrote about Executable UML. The problem: it was clear he had no experience, little knowledge and only wrote his opinion. I had expereince and it was clear to me he was spreading "misinformation". We need filters!

Recently I was playing with "types of sytems". I sorted by Frequency of Change and (something that sort of related to) Quanitity of Knowledge. NASA types systems have low change and a relatively clear level of knowledge,

For systems that change frequently, mostly business systems and commercial software, I has seen that XP has tremendous benefits.

We aren't always critical about types of systems when we advocate types of approaches.

- BeckyWinant 2002.08.20

There is a name for doing XP with the dials set at 5 instead of 10 -- "Pretty Adventuresome Programming" -- this reflects the adventures that occur when refactoring is done on un-tested code, integrating less often than daily, and so on..

I'm not exactly a fan of the name "ExtremeProgramming" either.... but I'd expect that no one would think XPers are advocating this method for NASA-level software, either... Though there has been at least one NASA sub-contractor on the XP mailing list...

:-)

KeithRay 2002.08.20

I think its healthy to see NASA sub-contractors on XP lists. Cross fertilization of ideas is usually good.

Pat Medvick once lamented how scientists at the labs rarely intermingle. She said it is the rare chemist who checks into the latest thinking in biology or physics, and so on. Sometimes people in software are as bullheaded.

One of the reasons AYE is such a great experience is that we all come together with the purpose of learning from each other. What a wonderful world it can be!

- BeckyWinant 2000.08.21

I love the "Extreme" part of the name. Take the Unobjectionable Modeling Language. Well, a problem with it is that everybody knows it. Everybody has seen some of it, used some of it. Everybody's a UML expert or well on their way to becoming one.

Whereas people's reaction to XP are a nice first approximation to at least one element of their overall personality. You get at least one useful bit of information in the difference between "No, I don't intend to try or read about it but will tell you everything that's wrong with it" and "Well, I've read Kent Beck's book and here's what I think is wrong with it".

(I've read exactly one UML book - Martin Fowler's.)

LaurentBossavit 2002.08.26

Well, a problem with it is that everybody knows it. Everybody has seen some of it, used some of it. Yes. A problem, but possibly a solution, too.

I believe that a common language is a good thing. You and I would have a rough time communicating in French, Greek, Italian or any other language I do not know well enough to get past ordering dinner. (My failing).

At the same time, that "everybody knows it" also means that everybody gets to abuse it. How do you get a good (qualitative) sense of UML? By talking about good practices, not language! So the whole issue of language is irrelevant.

And that is one of the very best things about XP � good practices. I have colleagues who build models who talk about XM � extreme modeling, which is a colleagial nod to poeple who pay attention to the right things as they do their work.

I have skimmed Martin Fowler's book. It is terse and not very informative for a first time modeler. It makes a few leaps, like listening to a conversation of two college friends talking about a time and place you never were part of.

- BeckyWinant 2002.08.25

Re: Methodologies for Large Projects (I'm a bit tardy joining in.)

I've seen some immense projects - 2,500 developers, plus about 1,500 hardware types - actually work. There's a tunability and decoupling game that can help. The "methodology" actually tunes to the type of thing that you're dealing with. Decoupling provides a level of isolation, so you end up with famlies of teams, of individuals, bouncing whatever changes they make through a series of processes depending on the scope of the change. Or put another way, the people you have to coordinate with are the people you effect.

Specifically:

• Individual development teams used whatever "methodology" made sense within their team. Deliverables were consistent. I think that's a big deal for a system that absolutely, positively has to function for 30 or 40 years, (service life of a submarine) and you know that ahead of time.
• Groups of teams coordinated with each other, pretty much pushing work around among them as they discovered that some things were harder than others. They mostly depended on artifacts - functions and objects - from each other. Internal details across teams were of academic interest, not "shared" responsibilites.
• The larger groups were responsible for providing pretty abstract APIs to "all comers". Changes to the APIs, in content or capability were a _huge_ deal, as they should be.

The team I was on had a kind of unique vantage point. We were working in integration, trying to plug all this stuff together, so we saw the whole thing, at various degrees of details. In our own development process - tools, not the tactical system - we were:

• Pretty rigid about the interfaces: data feeds, output formats and so on. In fact one of the ways we eventually got stuff shipped after a couple years' Brownian motion was "freezing" the interfaces until the systems worked. Then we'd talk about changes to the interfaces. (If they're still changing output formats bi-weekly after two years, there's a different problem there.)
• Rapid iteration of development. We built harnesses - multi-language, multi-tool "make" extensions - necessary to make schema changes on demand, including automating moving data from old to new, and rebuilding all dependent apps. This was 10 years ago at this point - wasn't that common a practice at the time. Intersting contrast to what I know of XP, we weren't iterating so much to deal with slices of functionality, at least not until we got the thing to work. We were iterating to get it to work. Lots of purchased system elements broke all the time - OS, Database, Typesetting SW, Scheduling Stuff, Security Layers, etc.
• A lot of - er - conversation in a bullpen arrangement.

I suspect that pub / sub kinds of interface management, and the supporting technologies (SOAP over XML is the current candidate) enable this kind of tiered processes. It's always useful when you have a thing to hang a process from. So a big project can make two policies:

• Components talk through these messages.
• Changes to these messages are a Big Deal.

That's kind of what happens with using former projects (JerryWeinberg's comment above) anyway. There's a model there, actually. The things we get to re-use from former projects hide behind APIs pretty well. Databases. OS-es. Hardware. And so on. And changing those APIs is a Big Deal however the API is implemented, and the Big Deal defined.

Subtopic, Doing PairProgramming:

[ moved to PairProgramming page ]

Subtopic Comingled Issues with XP

Recently, I've seen a lot of comingling of issues, especially in "web" development. The same folks doing "programming" are doing - often badly - project management, general management (ongoing operations, staff development, roles and responsibilites, etc.), and systems engineering. Sometimes in these projects systems engineering and architecture seems to be assumed, or implicit.

Where do these get done? So, I'm most interested in the scoping discussions above, where some activities are explicitly not part of XP. I really like the explicit delegation, within the organization. "We're doing this programming stuff. Doing that other stuff is somebody else's job."

There was some work, something like 12 years ago or more, on the twin topics of tunable methodologies, and the independent variables (change rate / cost, for example) that determine what methods might work. I need to go find this stuff, as it was way better than anything else I've heard on the topic. Will post if I find that stuff.

- JimBullock, 17-Sept-2020, (Cosmetic edits, 18-Sept-2002)

The XP literature describes FourVariables -- Scope, Resources, Time, Quality -- and recommends leaving Scope free and fixing Time, Resources and Quality. "External" Quality is defined by the customer. Since few customers want features that they have specified to not work correctly, XP fixes quality "high" via automated acceptance tests. XP gurus also say that fixing "internal" quality high via pair programming, refactoring, team code ownership, continuous integration, and automated unit tests makes the whole project go faster -- incremental development is easier if the code is reliable.

My manager has told me that we can't do things the XP way -- Scope, Resources, and Time are fixed. We must have ALL features implemented on time, without additional resources, even though the time allowed is about half of what we estimate we need. However, Quality is not fixed -- as long as we "hand over" to testing on time, it doesn't matter if we have lots of bugs.

To me, a feature that is buggy (doesn't perform according to spec according to the customer) is not much better than a feature that isn't implemented -- it's just a difference in terminology and playing games with how we keep track of progress.

Should I change our tracking of progress to be as vague as other projects in my company, so that we "complete" everything on time, but actually spend the testing period finishing our implementation -- just like the other projects do?

KeithRay 2003.02.17

Depends on whether you respect the principles of your manager and other colleagues who operate this way.

I suspect the primary principle in play at your company is survival. People believe that their employment depends on reenforcing senior management's fantasies. I also suspect that your manager knows that senior management will change their minds when the data is overwhelming about scope and schedule, which happens a few weeks before something is supposed to ship.

This style may fit for your manager and colleagues. Does it fit for you? Are you willing to accept the risk of accurate reporting, which is 1) branding as a heretic, 2) being asked to repent, and 3) being banished from the company.

SteveSmith 2003.02.17

"Should I change our tracking of progress to be as vague as other projects in my company, so that we "complete" everything on time, but actually spend the testing period finishing our implementation [...] ?"

I think you've accurately restated the Zeroth Law of Software Quality - "If it doesn't have to work, it takes no effort at all for my software to be delivered on schedule." -- LaurentBossavit 2003.02.17

The irony is that a year ago, concerned about our customer's complaints about quality, the executives launched a "quality initiative", which eventually "required" that employees write unit tests and do code reviews. However, code reviews are informal (no tracking that they're done at all) and no room was made in the schedule for them, so code reviews are not done. Ditto for unit tests. Actually some managers enforce unit tests and code reviews a little more than mine -- I'm trying to transfer to one of those departments, but I'm "essential" where I am.

KeithRay 2003.02.17

From a testing or support perspective, the buggy feature is worse than one that is not implemented. I suspect that it is from a customer perspective as well. The chances of getting it fixed are not as good as the chances of getting it delivered later.

I'm trying to transfer to one of those departments, but I'm "essential" where I am.

How bad do you want to transfer and how smart is management? Sometimes, pointing out that you are ready to change either companies or departments, their choice, will get you the move you want. But you have to be ready to walk if it doesn't. SherryHeinze

Industrial Extreme Programming is a new "enhanced" XP.

I wrote it up on my blog: "Industrial Extreme Programming" explained on my blog...

<>