Step 6: Narrow It Down

NOTE:

This step takes almost a day in my two day course. It's beyond the scope of this website to cover it completely. If you can't take the course, you can learn all about this step in any of these books:

To see which book will best fit your needs, look here.

Divide and Conquer

Mathematics tells us the fastest way to find a single element in an ordered set is binary search. Binary search is the process of repeatedly ruling out half the remaining search area until the element is found. What makes the system you're troubleshooting an ordered set is your knowledge of it, reinforced by manuals and documentation. It's that knowledge that allows you to devise tests to split the search area in half. Below is a simple diagram of a binary search finding the violet component with only six tests:

This technique really shows its power in systems of several hundred thousand components. For instance, binary search could find a single component in a system of 1,048,576 components (a moderate sized automated system) using only 20 tests.

Note:

NOTE: Implicit in all this is that if you keep narrowing it down, whether binary or not, as long as you don't repeatedly double back in areas you've already tested, it is a MATHEMATICAL CERTAINTY you'll eventually solve the problem.

Limitations:

Of course, life isn't this simple. If it were, we'd be getting minimum wage. Several factors combine to make troubleshooting a challenge, even to those employing binary search.

Intermittence:

Intermittence invalidates most tests which could split the search area, resulting in backtracking. It thus renders binary search a useful but insufficient tool for troubleshooting. Intermittence eliminates the mathematical certainty of solution -- indeed many intermittents remain unsolved. There are several techniques to maximize your chance of solving an intermittent.

Ordered Set:

Remember the order comes from your knowledge of the system, and nobody knows everything, including the system documentation. The less complete your knowledge, the more trial and error is necessary. Nevertheless, in real life even a minimum of knowledge allows a reasonable approximation of binary search, so this isn't much of a limitation.

The best way to make your system knowledge into an ordered set is to organize it as a block diagram, which I often term as a "Mental Model", because it can be carried in the head as well as on paper. The following is an example of a Mental Model, which organizes knowledge of the Daemontools software into a Mental Model to achieve an ordered set: