Review By: Jon Hagar
07/08/2010This book addresses a long-ignored area of software and systems testing. The industry has produced generic books on testing, web based testing, and other specialized areas, but this is the first book covering the area of embedded software. The book starts with a standard overview of test concepts and methodology, which are designed to assist testers in assembling test approaches and techniques for embedded software. The book's organization continues by covering both the ideas of a master test plan, and then detailed test plans covering lower levels of testing.

Next comes consideration of lifecycle issues, including both the traditional V and newer spiral models. Related to lifecycle issues, the authors consider the activities of both developers and independent test teams. Developer test activities are correlated to the lifecycle stages. Independent test team efforts follow the lifecycle stages explaining what testers are doing at each stage of planning, test preparation, test specification, execution, and completion.

Once the embedded lifecycle and test activities are covered, the book details techniques specifically related to embedded systems. Techniques considered in sections of the book include: risk analysis, testability, inspection, test design, and tester checklists. Each of these is presented with an introduction and then a "cookbook" based process outline. Particular detail is given to test design techniques and the all-important checklist with details specific to the embedded world.

After techniques, there is a section that details infrastructure considerations such as the test environment, which can be a large problem area for embedded systems, tools that again can be unique to embedded software, and automation concepts. The issues of embedded systems are further explored in the areas of input/output, in what the authors term "mixed signal analysis" and digital domain problems, both of which have aspects that are peculiar to the embedded domain.

Finally, there are sections detailing fault-tree analysis and failure-modes-and-effects analysis (FMEAs), both of which are important in testing critical embedded software systems. After dealing with some of the technologies and approaches for testing embedded software, the later sections of the book discuss the important issues of human relations and staffing. These include tester skills, team organizational structures, and finally, the importance of software configuration management in testing embedded systems.

A problem with any test book is what to cover and how. Because of size limitations there is a lot of material that must be skipped or summarized. This book is written as an introduction to the topic of testing such systems since it covers the important topics of planning, organization, and techniques that a tester will face when entering the embedded software world. Additionally, the book discusses in-depth issues that make the embedded tester's job different from that of a Web-based tester. Several sections and topics such as "Mixed Signals" and "Evolutionary Algorithms" treat topics thoroughly and are applied uniquely to the problems faced by embedded software testers.

The book does make a number of claims that are generally accepted in the testing community. But the book does not include data or references to support the claims. There was minimal coverage of topics such as test oracles (a judge or predictor of the outcome of a test); simulation and commercial tools that can be used to support embedded testing; or coverage of regression test issues that can complicate testing embedded software.

Many topic areas were probably limited due to space and size considerations. There were several sections that covered interesting "state of the art" topics. One topic is an interesting treatment of evolutionary algorithms, which are new to many of us and provided good "food for thought" reading. The book's coverage includes enough depth that a tester should be able to get a good start based on the information provided.

Another useful feature of the book is the writing style of some sections, which is such that a test group could pick up the book and "plug it into their environment." These procedures and plans can make the book useful to groups that lack standards and are looking to "jump start" the maturity of their organization. Additionally, the book has a lot of good lists and an information scheme. These features make the book very practical and a good starting point.

The end of the book contains a series of appendices dealing with various topics, including risk classifications, state based testing, automation blueprints, an interesting set of pseudocode covering evolutionary algorithms that can be used in generating test data for embedded software, an example test plan outline, and the section every test book should have—definitions of terms used in the book. In an appendix, the state chart section is presented as a tutorial on how to diagram state charts and apply them to embedded software. The appendix on "blueprint of an automated test suite" provides another "how to" covering producing a planner, reader, translator, test actions, initialization/synchronization, error recovery, and test reporting/checking.

The authors have filled the book with insightful comments and observations. People faced with having to start a project testing embedded systems will find the ideas presented in this book great reference material for their bookshelves. I found myself highlighting these gems, so that I can find them quickly. And, while the book may not cover every area that is being researched or developed in embedded system because of size or space limitations, there is a wealth of information and in-depth ideas.