Back when I was in high school, my friend was driving down the highway late at night and leaned toward me, which momentarily sent us careening toward the concrete median. I won’t say my life flashed before my eyes, but the panic must have been evident on my face since he immediately corrected the car’s path. It left me with a lasting dread of driving in the left lane that plagues me to this day. (Yes, I know the rest of the lanes have the excitement of other life-threatening opportunities courtesy of my fellow motorists.)
Not long ago, another friend of mine was driving through a construction zone and was pulled over for a speeding ticket. They hit him with the maximum fine, which was double the norm due to the construction zone. Although I wasn’t present for this occasion, it is a salient example of the sort of thing that can bite you on the road.
These examples were foremost in my mind as I was driving home from the StarEast conference last night. I was driving in the left lane when a median and a construction zone simultaneously appeared before me, sending me retreating to the right lane where I set my cruise control to the speed limit displayed on the signs.
Although I deemed it unlikely that a cop would be waiting for someone to step out of line on that particular stretch of highway on a Saturday night, I certainly had no desire to find out by experimentation. A co-worker of mine recently related her experience with being pulled over on a lonely stretch of highway when she had no time to argue with the officer’s judgment. Honestly, who would want to be dragged back to the middle of nowhere to prove your innocence? (Assuming you can…)
Sometimes, I think our test cases can adhere to the same algorithm I use in driving: That bit us once and the consequences were terrible! We must test it every time!
James Bach mentioned in his Creative Thinking talk at StarEast that we have diminishing returns for this sort of testing. When we know we will always test this area, we are more likely to implement correctly.
I think we as testers also can easily become complacent with areas we have tested over and over again. This has definitely happened to me!
“Because the first bite always tastes best!” – Ramona Quimby
When we get a new feature, we think of all the interesting and malicious and foolish ways to use the system. We get to take the first bite out of it and savor the sweetness of the quick payoff.
It’s so much more difficult to stick with testing regression cases by hand for each release that goes out the door. We continue in our probably unnecessary task because we cannot bear the thought of missing the same consequence a second time. We end up with additional cases that cover certain known error conditions that just lengthen our testing cycle.
Similarly, by limiting my speed and proceeding with caution to avoid the ticket, I lengthened my drive home so that my arrival time was after midnight. Although I guaranteed that the hazards I was vigilantly monitoring did not occur, my time might have been better spent accomplishing the goal.
Likewise, I know the terrifying experience of veering toward a median, so I am much more likely to avoid that danger and the driving habits that would produce it, guarding against a known case.
Do you maintain cases that slow you down more than benefit your release cycle? Do you continue to execute them manually?
Or do you automate these tests to free up your manual execution time for new approaches?