WhatMakesTestersGood

ShannonSeverance and I (SteveSmith) met over lunch this week. Shannon told me about the time he spent at ST Labs being a tester. I asked him what makes a tester good. He shared a couple of ideas. The next day I received a message from Shannon that contained the following passage:

..........
I've been thinking about what makes a good tester. For most of the time at ST Labs, the testers held in the highest esteem were the ones who had lots of technical skills. They all had some sort of programming background, either practical or academic, and kept up on APIs, operating system internals, and/or database systems.

Personally, I think diversity in testers is more important. If you have a group of homogenous testers, they will all find the same problems, and miss the same problems. So you need the techie expert who worries about race conditions and conformance in use of the registry, and you need the artsy literary type who will notice that the bitmaps aren't anti-aliased, and catch the misspellings in the menus. Because of the diversity angle, there are testers who I respect a lot, but who are not very skilled technically.

There are common attributes among good testers:

An eye for detail, although the detail that interests them, and that they are looking at will be different.
Interruptability. When testing, there is always a chance you will learn something about the product that requires a closer look, now. Coding seems more stepwise.
A delight in finding problems.
Communication, communication, communication. Both written and verbal.
Some base level of political skills.
The ability to gain satisfaction from a good testing job. Part of that is a delight in finding problems. Sometimes, it seems that a testers work is really indirect. Nothing that the tester does ends up in the final product. So it can look like the tester is dispensable, that the product could be built without their input. I suppose the same can be said for management. Being able to convincingly reason about the value added by testing, and internalize that value is important to maintaining any sort of satisfaction.

..........

What would you add to Shannon's list?

SteveSmith 2002.05.03

I would add skepticism. I think that testers should look at a product or at the product under discussion (in test-first) with skepticism. I like to apply the generic test-of-3 to my testing: - what is the general, normal use of this product? - what's abnormal, but still in the normal range of use? - what's abnormal, that only crazy people would do?

I found a memory leak in a GUI-limited application once, because I was on the phone and kept clicking a button to create a new window, just because I couldn't do anything else and I was wondering what would happen. For this application, that was crazy behavior. However, once we uncovered the memory leak, we found more defects, in the same area. Hah! The developer was amazed, and we decided to do a full peer review of the code in that area. We found more problems.

Now, it's possible that under normal use, no one would ever have found this problem, but it's possible someone else would be stuck on the phone and clicked more create-windows, thereby setting off a chain reaction of defects.

So, skepticism is part of my abnormal, crazy-person testing. -- JohannaRothman 05.05.02

I always found that a good memory helps, allowing you to put together two things that happen at widely different times and be able to use the combination to find and fix the most subtle problems. "Oh, I've seen something like this before!" - JerryWeinberg 2002.05.07

I also look for: - people who really like systems, and delight in understanding how they work: "If I do this, what will it do here?" Techies or not, people who love systems develop instincts about where to look for bugs.

- strong analytical capability, and persistence to work through all the combinations. Also the ability to know which combinations really are different and which ones merely repeats (equivalence partitioning).

- the ability to distinguish where the risks are and focus on those areas

- respect for developers and a non-blaming attitude about bugs - FionaCharles 2002.05.17

This week at a client I saw something else that Fiona's post reminded me of. I look for testers who have a passion for tracking things down to the very end, and are not satisfied with partial explanations of phenomena. - JerryWeinberg 2002.05.17

Curiousity - wondering "What would happen if " and trying it, when you are testing code or tracking it through if you are testing requirements.

A team player attitude - "we have a problem". The ability to show a developer that you are all in this together, not on opposite sides. SherryHeinze 2002.05.20

Some recent experiences have suggested that placating is a common and serious tester's disease. So is blaming. Congruence makes testers good.- JerryWeinberg 2002.07.01

Ability to keep good logs, both of what was done and what it was done on. This can contribute to interruptability and good memory. -DaveLiebreich 2002.09.22

Ability to differetiate the meaningful tests from the multitude of possibilities. Too often, I see testers develop numerous tests (and document them!), when one or two in that area to ferret out risks would have been a much better idea. --JohannaRothman 2002.09.23

Who is responsible for directing testers efforts? How do they know how to choose: differetiate the meaningful tests from the multitude of possibilities? - BeckyWinant 9-23-02

Hi, Becky. If a tester can master the principles behind finding equivalence classes in sets of test data, then she should be able to apply that same principle to sets of test cases. - DaveLiebreich 9-23-02

I have found that all great testers are good storytellers. Then again, most great (fill-in-the-role) are good storytellers, aren't they? - DaveLiebreich 2002-09-23.958

The ability to say "No" congruently (or at least tactfully).

DaveSmith 24 Sep 2002

Updated: Tuesday, September 24, 2002