OrganizationalInsanity

Steve Smith described being caught between professional preparation for a presentation then having someone make a decision (cutting time and adding another speaker) that seemed counter to the best interests of his presentation and the 80 person audience. (WeAreGoingToHaveToMakeTheBestOfThis)

Last year a new Chief Architect introduced his personal way of diagramming structual architecture. It included more than just structure. There were no written references on this, nor could he clearly articulate guidelines for doing it. Non-standard and enigmatic. All application architects were ordered to use it immediately because it was mandatory for all project reviews. I felt a bit like Alice wondering "Where am I?". And, I was supposed to help the architects construct this. I was as confused as they were. Meanwhile, the Chief Architect eagerly awaited positive results.

When Steve declared his experience a result of Organizational Insanity this is what immediately popped into my mind. What experiences might you plop into this category? - BeckyWinant 07.10.06

I'm just wondering where the Organisation is in this Insanity? It appears to be about single individuals imposing insane decisions on the organisation. Does the organisation allow individuals to exercise power unchecked and isolate those people from effective feeback about their decisions? Is that the Organisational part of the Insanity? - PaulWilson 2006.07.12 (first discussion post on the Wiki - hope I've got the etiquette right)

I think it's nuts that organizations want a predefined process for every project. I think it's reasonable to have organizational guidelines about the results they want. But if every project is unique, how can an organization have a single process that's supposed to fit every project? -- JohannaRothman 2007.07.11 (it is where I am :-)

Johanna, you're lightyears ahead of us! Well, an earth year, anyway. ;-) -- GeorgeDinwiddie 2006.07.15

Johanna, Good Grief I live that nightmare!

The project methodology where I work contains various extensions to the "traditional" path. There are references within the "traditional" path, separate paths outside, and everything is intertwingled. On paper.

I talked to a number of developers and discovered they pretty much work with others on the project to develop in a way that makes sense for the specifics of the project. Sanity underground when insanity reigns overhead.

But, yes, that paper creates the insanity of going through the right "gates", hurdles, whatever to get signoffs. - BeckyWinant 07.11.06

Paul,

You do have the etiquette right. If you're wondering about dates, take a look at StarDate.

You asked Does the organisation allow individuals to exercise power unchecked and isolate those people from effective feeback about their decisions?

The cynical part of me says yes. The realistic part of me adds: Organizational hierarchy encourages people in certain positions to take responsibility for their organization's work. And, if managers or leads or whomever doesn't know how to ask for feedback, that's a huge problem. That's when the insanity multiplies. -- JohannaRothman 2006.07.14

Oooo! Johanna, your feedback comment jiggled a memory. The Chief Architect mentioned above said one day how he knew he was bad on asking for feedback and he hoped to change that. I believe he now asks for feedback, but the side effect is he may take off your head off if he doesn't like what he hears.

You hit the button with that insight! - BeckyWinant 2006.07.20

Becky -

My guess is that after the first time he took off someone's head for telling him what he didn't want to hear, people stopped telling him anything useful. So now he can say he asks for feedback, and blame other people for not telling him what was really going on. Its so interesting how we humans go to great lenghts to fool ourselves.

EstherDerby 2006.07.20

I know this thread is called organizational insanity, and the "organizational" part has already been questioned--and questioned well. But I'd like to question the "insanity" part. The example Becky give seems to be an example of either stupidity or ignorance.

It might be insanity if the so-called chief architect had tried this (what's this?) before and gotten the non-response he's going to get this time. If he then expects something different, then that's insanity.

I can't believe that people in the past have actually submitted to this dumb idea--document everything in an undocumented notation because it's very important that everything be documented. Huh?

Not that it's a new idea. I've seen it many times before: "Do things in exactly the way I'm waving my hands." And every time, people just ignore it, or make a perfunctory effort for a while until the whole thing is quietly dropped. Best to drop it right away. Ignore it, lest you, yourself are insane. - JerryWeinberg 2006.07.23

The chief architect used his diagramming in previous places. Some of those people who used to work with him are now here. Luckily one architect is a very sane guy and he and I spent many hours working through a reasonable way to diagram arhcitecture and getting rid of the confusing parts. Recently the CA came up with new ideas for the diagrams and I am upposed to working with him and a couple others to "make it so".

I do think CA blames. And, maybe not so much insane as a strong ExTJ with delusions of grandeur. :)

I figured back when I was put in the position of dealing with the CA on the "new notation" (and my head spun) that since confusion was the gateway to learning, I was about to learn something! I have been well aware that the lessons are about human interaction and staying away from the fray with my chameleon routine. The diagramming stuff I just try to make easy and useful for the architects. They drive much of that. - BeckyWinant 2006.08.08

I experienced some insanity yesterday. We are planning our FY-07 budget. I suggested that we sum the dollar value of all our high-risk development projects and set a side a management reserve (say 15%) to cover the probable cost over runs in these projects.

The management reserve was quickly and soundly rejected.

This is in the face of the historical record that shows the average cost over run for high-risk development projects to be about 15%.

To increase the insanity, people argued that we did not have cost over runs on our projects. Well, maybe one a year or something miniscule like that. Once again in the face of my research that showed that on the five projects this past year that I investigated all of them had cost over runs.

Denial is wonderful.

DwaynePhillips 9 August 2006

What did you see or hear that led you to believe you had these overruns in the past?
What did they see or hear that led them to believe otherwise?
If you can't agree on what's in front of you, you'll have precious little traction on anything else. So, start there. I do know two useful general definitions of insanity:

The inability to tell the difference between what is going on inside your head and what is going on in the world around you.
Doing the same thing repeatedly, expecting different results.
"They" may be "insane" by the first definition, committed to some idea independent of external evidence. Then again, maybe not. Maybe it's you. In either case, you'll get better traction at finding some common ground if you ask them to explain their conclusions, while yourself being sincerely willing to be convinced, if convincing is to be had. Regardless, don't you engage in insanity by the second definition. Asking the same thing again the same way won't do much good.
-- JimBullock 2006.08.09 (Or maybe not. Do you have a reason to believe otherwise, that you'd like to share?)

Dwayne, is the 15% overrun that you've observed in addition to used-up project contingency? I'm curious because 15% is what I was taught to use as a "standard" contingency to apply explicitly to average-risk projects (whatever those are). Or does your organization routinely disallow built-in project contingency -- like a lot of client organizations I've been in?

-- FionaCharles 9-Aug-2006 (adjusted the wording 10-Aug)

We do not allow built-in project contingency. The thought seems to be that "surely someone has a contingency somewhere." No one, however, can put their finger on this contingency. Hence, we have this insane belief that the contingency exists out there somewhere.

DwaynePhillips 10 August 2006

Explicit contingency can only work when it belongs to somebody who manages it--somebody who has no jobs to spend it on. If it's just a common pool with no gates, it gets sucked up quickly by the worst managers. In other words, I suspect that Dwaynes agency has learned that since they can't or won't manage an explicit continguency, it's better not to have one. - JerryWeinberg 2006.08.13

That's how the PM I like best to work with deals with explicit contingency. She owns it and releases specific amounts to team leads IFF they can satisfactorily account for the need. I've seen this system work very well, on some high-risk projects where it would have been crazy not to have contingency.

I don't know how to deal with contingency otherwise, except by the less-than-optimal practice of padding estimates.- FionaCharles 20-Aug-2006

The TOC folks (Theory of Constraints) say that each estimate is as likely to be under as over. So they schedule at the 50% estimate and put the rest in a buffer. The PM manages the buffers. That's one more technique.

I find that it's too easy for senior managers to try to make the buffers vanish, if the PM is not able to protect the buffers and therefore, the team. -- JohannaRothman 2006.08.21

Johanna

>The TOC folks (Theory of Constraints) say that each estimate is as likely to be under as over.

When all other things are equal, this may be so. I have enough stories indicating all other things are not equal. I'm sure the group can add more.

A few years ago (4?) a kayaking friend shared the CEO of his company decided in January that the entire (international) company would standardize on a given software platform. It took until April/May before the spec could be written such that the preferred vendor could lock in the contract. We discussed this in August as the "consultants" were starting to descend on the organization. Can you believe it took longer than the estimate?

Being from the "hard engineering" side of the family, I thought this was a "soft engineering" problem. Then I got involved in a fast track chemical plant project where the pipe fitters (ever seen what a pipe fitter gets paid?) were cutting out 4" & 6" stainless steel pipelines (ever priced 4" & 6" SS pipe?) today they installed 3 days ago, since the elementary and elevation drawings weren't correct or complete, but there wasn't time to wait. The software cost changes paled in comparison.

I found my copy of "Theory of Constraints" and will flip through it on the flight tomorrow to see what Goldratt says about "all other things being equal".

> I find that it's too easy for senior managers to try to make the buffers vanish

That's why they made it to senior managers.

--DonGray 2006.08.21

It's worse than that. "Middle" how? Likelihood of being high or low? Total amount of error high or low? Number of samples high or low? Depending on how you define each of these terms - and there are standard definitions for each, more honored in the breach than not - you get a different estimate, probability band, and weighting.
It's worse than that. What's the cost of being high or low? If I'm jumping over a creek, I think I'd rather be long than short. If I'm jumping between two cliffs, I think I'd really, really prefer to be long than short.
The less trust and confidence in the organization, the harder I skew to under-promise because the costs of under-delivery are amplified by the history already had. Under-delivery feeds something bad already going on in the social system. (Which is neither the project system nor the production system.) It looks like sub-optimization when you think only locally. It isn't.
One kind of organizational insanity is different kinds of systems in the same organization, working against each other. Unfortunately the word "alignment" has been abused, but that's what's missing.
- JimBullock 2006.08.21 (It's a system of systems, like the inter-web.)
I think I'm going to argue with myself a bit here. The idea that different kinds of systems should necessarily line up doesn't always hold. Maybe I'd like to do one thing in terms of "development team as a learning system" and another thing as "development team as a business function, right now." The insanity comes in two ways:

Setting up systems that balance against each other, then expecting there to be no tension when they do so.
People within one system or another somehow expecting that their local perspective shouldn't have to balance with any other perspective and system.
Sometimes a situation is just hard, with intrinsically conflicting demands. Sometimes the parts of an organization just don't line up. Insanity then is pretending that they do, or should, or most importantly must.
- JimBullock, 2006.08.23

For a rather different take, I claim that much of this difficulty results from a mistaken approach that treats the project as a SINGLE entity. The problem of contingencies is much easier if you don't schedule the entire project but rather plan on what TomGilb calls EVO. He suggests that no phase of a project should be permitted to exceed two to five percent of the project resources.

When evolutionary delivery is the plan, and the goals have already been discovered and measurements for each have been specified, then it is possible to think up a collection of possible next steps at each point in time. Then we estimate the effect of each proposed step and the cost of carrying out that step. Then we can choose the step with the best ROI, the lowest cost, highest payoff step, to do next.

In this approach, it is possible that some projects will be stopped early for one of two reasons. Perhaps it has become clear that the project will not succeed and should be abandoned before wasting more resources. On the other hand, the steps taken so far may have accomplished the whole task or so much of it that resources would be better spent in another way.

Despite Gilb advocating this approach for decades, it is still considered radical and usually impossible. It is radical. It is usually possible and profitable to do things this way, if you are in a position to insist on it. Since there are many reasons to resist such a change, this may require serious committment by members of senior management. W. Edwards Deming and others of his ilk have also had this problem.

If anyone has ANY good advice for how to approach such major reinvention of business practices, I would be eager to hear of it. So far my own efforts have met with very limited success. This calls to mind that there is a vast difference between a clear vision and a short, easy path to realizing that vision. Even so, it seems to me that it is more promising than other approaches mentioned above.

DickKarpinski 22 Aug 2006

Dick, no good advice, alas. A couple of buts�

I can imagine that EVO could work well for inhouse projects, and maybe even better for a comprehensive program of projects.

But those of us on the vendor side would need our customers to undergo an even greater cultural change than you suggest. Often, we�re trying to sell to customers who want big custom development/integration projects for a fixed price. It�s still the mode of thought, even if they can�t find a vendor silly enough to commit. It�s frequently difficult even to convince a customer the vendor needs a requirements discovery phase before committing. Contingency is a major consideration in these circumstances. (Sure our sales people could walk away from those customers, but we all like to eat regularly.)

Forgive the obvious -- logic isn�t always the most compelling force. Justifiably, customers view vendors and their pricing practices with suspicion. I think a lot of customers would dismiss EVO as a more creative way for a vendor to get even more money out of them over the life of an entire project.

Also, even with EVO, estimation and accurate measurement of ROI will still be bets on the future, achieved with varying degrees of information and skill. Yes, it�s easier to estimate the cost and effect of smaller pieces with some degree of accuracy. It�s easier to manage some risks, too. Nevertheless, I think we�ll still have to recognize our fallibility and include some contingency for all those things we forgot to predict. --FionaCharles 22-Aug-2006

Actually a lot of outsourcing deals are exactly a contract to "absolve me of responsibility for all this uncertainty." If things don't come out close enough to anticipated, hey, you can blame the big, bad contractors with relative impunity. Indeed, the customer has paid for this privilege.
OTOH two of my most entertaining - entertaining in a watching a car crash way - times were watching outsourcing / services organizations try to move to a "risk shared" model with their customers. In the end they couldn't stand the uncertainty of abandoning short-interval billing with built-in margins. A few customers took the ill-formed contracts offered - folks so confused they pretty much guaranteed there would be no up side.
The mechanics of estimation and risk management are pretty knowable. EVO is one of the better models - if you can get far enough to use it. More often I have seen the problem be up-stream, and outside the mechanics of estimating, measuring, setting goals, sharing results, etc. etc.
- JimBullock 2006.08.22 (Not that I've seen that happen, ever.)

Sure, Jim, but that's what I meant about our often limited knowledge while estimating. The mechanics won't kill us. The things we don't know (and therefore forget to predict)and the ones we can't control, often do. One small example: the code we or the client buys in from elsewhere that turns out to be of a crappy quality that blows our testing estimates. (Gee -- that's never happened to me!)

And umm -- the mechanics of risk management are indeed knowable (and known), but how often do we see them practiced? Or get to practice them ourselves? --FionaCharles 22-Aug-2006

Sorry. I was unclear.
Worse than the stuff we don't know, a dysfunctional relationship guarantees that we won't effectively use anything we have - what we know, what we don't know, what we know how to do, or not, what we're finding out as the project moves along, whatever. We don't use information we have available because we're all wrapped up in something else, something other than solving the problem at hand.
For example, maybe we have crappy purchased code or the risk of crappy purchased code. Why couldn't that risk get heard and tracked? How long did that fact exist before plans and projections got adjusted to accommodate this new information? What happened when the battle-scarred old veterans brought this risk up early?
When we know better, and still do the silly thing, something else is at work. I've found in most projects that if you can get over the masking problem and talk about what's really going on, almost any estimation approach will do. "Projects" seem particularly prone to this kind of chosen blindness. I don't know why.
- JimBullock, 2006.08.23 (Or am I all wet here? It happens.)

I hope this answer doesn't sound cute as I don't mean it that way. I believe the answer is congruence and incongruence. JerryWeinberg will lead a session on congruence at this year's conference.

I will "assist" Jerry. Some of the most meaningful and helpful (to me) things I have done in the past year is teach a class on congruence.

DwaynePhillips 24 August 2006

That should be a great session.
There are at least two, and perhaps more semi-distinct uses of "congruence" floating around in the human-potential-ish population(s). For me, "congruence" seems kind of like "center" in Aikido in that all the descriptions are a bit misleading, yet not wrong. Even so, when you encounter the real stuff you can tell.
Even the partial descriptions fit kind of like the famous blind men's descriptions of the elephant - not wrong, but incomplete, yet when taken together pointing at something big. A pattern match of, for example: "congruence is very like a snake" taken literally, can lead to looking for snakiness (on a plane or otherwise) to identify congruence. It seems to me more subtle than that, so I expect a session (vs. a shallow description) to be far more compelling.

-- JimBullock 2006.08.24

Dwayne, I agree with you. I read somewhere (sorry, can't remember where) that there are some people who are more interested in maintaining the organization rather than releasing product. That's a completely incongruent stance (what people do vs. what people talk about). -- JohannaRothman 2006.08.24

I can see a situation where the organization is more important to maintain than a releasing a product. I would not want to be in an organization like that CharlesAdams 2006.08.24
. . . and it is incongruent (in this sense of "incongruent") only if "they" say they prefer to release a product, while acting in line with something else, say maintaining the organization. Simply preferring to "maintain the organization" vs. "releasing product" is no more necessarily incongruent than the other preference. Whatever grief there is comes from the disconnect, not the choice.

("Congruent" and "incongruent" here by one, but not another, of the more commonly used definitions. As I said above none of them seems complete to me.)
-- JimBullock 2006.08.24
Jim, thanks for the distinction CharlesAdams 2006.08.25
You are welcome. My self, I'm up to four kind-of-self-consistent, semi-compatible-yet-distinct, incomplete-but-necessary-ish uses of the term "congruence" in practice. Four different descriptions from four different "blind men" walking into various parts of that particular elephant, I suspect. Perhaps five. Maybe four and a half.
-- JimBullock 2006.08.25 (I'm a bit vague, but honest about it. That makes me dumb, not incongruent.)

Updated: Friday, August 25, 2006