Information Loss in Software Testing

[article]
Summary:
The higher you climb in the organization, the less information you get: An executive might only see red, yellow, and green for a project. Any time different teams need to communicate complex information, there is bound to be some information loss, and maybe some information control. We just need to understand where and why that happens, and—hopefully—how we can mitigate it.

Awhile back I worked with a team that had a showstopper bug. Management asked why the testers had not found the bug.

The testers did find the bug.

Well, why hadn’t they marked it as a showstopper?

They did mark it as a showstopper. The triage team had downgraded the bug to major, allowing the software to go out.

Why didn’t the team argue more effectively that the bug stay a showstopper?

Wait a minute. What is happening?

Clearly there was a communication issue. Any time different teams need to communicate complex information, there is bound to be some information loss, and maybe some information control. We just need to understand where and why that happens, and—hopefully—how we can mitigate it.

Information Loss

Consider your line software tester. The tester cares about the software they need to cover, the risks involved, and the bugs, all at an impressive level of detail.

Above the tester we have management, who wants some sort of quick, easy summary of these subjects.

It’s not that middle management is lazy. Instead, think of it like a math problem. If a director has 50 employees to manage and gets one hour of context about work being done by each employee per week, then the director is already working overtime—and they haven’t even accomplished anything yet except getting briefed on what their people are working on.

Middle management sits on top of a pyramid, and they need to reduce the information that comes in just to process it. We used to call the process of trying to consume all the information thrown at us “sipping from the firehose.” Cutting down the messages we saw to the vital few was a skill you either learned or, well, you left the company.

Reducing the volume of information means you’ll inevitably have information loss. Visual tools such as mind maps, spreadsheets, content trees, dashboards, and graphical charts can all help mitigate that loss. Perhaps you can deliver 95% of the value with 5% of the words.

However, the higher you climb in the organization, the less information you get: An executive might only see red, yellow, and green for a project. A senior executive might only see red, yellow, and green for a whole series of projects funded as a program.

It’s also easy for information loss to be handled poorly. Counting bugs or lines of code are attempts to make the complexity fit on the top quarter of a page for a decision-maker. We know these aren’t great metrics to track or communicate, but it’s important to realize that this kind of reduction needs to happen in order for people to make decisions.

Instead of laughing at it, we can work to provide better alternatives, such as the low-tech testing dashboard, more descriptive prose, or coverage maps.

Information Control

Sometimes information loss is a necessary evil. Other times, people encourage and orchestrate information loss for their own benefit. This is information control.

Usually people control information for some job security. An example is being the only tester who understands the system. Only you know how to regression test the order entry, or how to build a test environment for product search, or whatever thing it is you “own.” If people need things, they need to go through you.

That subsystem becomes your “fiefdom.” This is a great way to develop a sense of power about your area of knowledge that you control. If you explain it to everyone, they won’t need you because they can just make the changes themselves.

While this kind of information control can appear to be appealing, I would argue it turns you into the person you don’t want to be. Sadly, three things happen if you make this choice.

First, the job will lose its joy. Essentially, you’d be doing the same thing over and over again. You’ll feel bad about yourself, as, on some level, you will be making the simple seem complex. Second, you won’t make friends, as this sort of work will isolate you by definition. Third, when the system is eventually retired, you will have a limited skill set. It will be hard to find a new job.

In The Hobbit and The Lord of the Rings, the character Sméagol jealously guards the ring, causing him to become Gollum. He forgoes other relationships and calls the ring “my precious.” Gollum was not actually evil—he was simply a poor being, bound to the ring’s will, who became corrupted by his desire to hoard the power for himself.

Get your power from sharing information; do not become Gollum.

Better Information Communication

Being chastised by management for not fighting to keep a showstopper bug classified as a showstopper is a real bummer. It did, however, have one upside: It created the opportunity for a conversation about information loss and information control, and how to prevent them.

We can reduce information loss by describing the application in terms of coverage, or being more specific on the impact of known issues. It’s possible to address this at the micro level, a bug at a time, but I would also argue that there is room for a greater discussion around application quality at the macro level.

“Why did this happen?” can be a dark conversation. “How do we prevent it next time?” has a lot more light. Try to keep things on the light side.

Tags: 

About the author

CMCrossroads is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.