15 years ago, mid-October
Developers and Testers Test
Posted by pbirnie under technology
This is a quote by David M.Moore, Director of Development, Microsoft from the book Writing Solid Code:
"Before we'd had a testing group, the business applications developer had relied on the OEM customer to test the product to find bugs . This arrangement worked out well until we started to ship the retail product directly to the end users, before we'd shipped it to any OEM customers. For an interval, before the testing group was going strong, the developers had to test the retail products themselves. The developers who lived through that experience learned that they had to be very careful not to introduce bugs as they wrote and debugged code. They found out the hard way how costly it was to release a product that had bugs in it."
"But as the testing group got bigger, the development groups got more and more dependant on the testing group to find bugs. The development groups soon adopted the attitude that the testing group was responsible for finding all bugs. This led to such serious problems - slipped schedules, buggy features, incomplete features, even cancelled products - that something had to be done. Many developers felt no shame if bugs were found in their code after the product had been shipped. They'd ask indignantly, "Why didn't Testing find that bug before we shipped?" Testing should have responded, "Why did you put that bug in the product in the first place?"
"Eventually the developers began to realise that Testing can never find all of the bugs in a piece of software. The bugs might be in the design, or in the specifications, or in the analysis of the customers needs. And testers can't do complete code coverage or path coverage in their tests. Bugs might be hidden in obscure and rarely tested code. Bugs can be temporarily masked by the operations of other parts of the program - or by the testing environment. These are the kinds of bugs that testers have a hard time finding. Because of these factors, a testing group will usually find only find 60 percent of the bugs in a product."
"The developers can bring more knowledge and tools to reviewing and testing the code. When the developers set their minds to it, they can find over 90 percent of the bugs in the code. If the developers give the responsibility for finding the bugs to the testers, the users of the product will find 40 percent of the bugs. If Development and Testing both work to find bugs, the users will end up finding less than 4 percent of the bugs. And that 4 percent could be found by the users during the beta test of the product. "
