Wednesday, September 9, 2009

Though I've never been heavily involved in testing, automated or otherwise, I've had some thoughts about it. There is a lot of effort put into manual testing where I work, and, late in a release cycle, if there is still a change that had to be put in, then incremental testing would done after that change was put in, as a full test cycle would be pretty expensive. In these cases, the tests that needed to be done would be specified informally, based on what code had been changed.

If a code coverage tool were integrated into the testing process, and if the version control system were also integrated, and if each test case that was run had the code that it covered recorded, then it would be possible to generate a list of test cases that would cover the changed code and a list of test cases that would only cover unchanged code.

There are obstacles for such a scheme, though. It would be expensive with a low return in a rapidly evolving code base. If there are multiple testers running on a single system, getting the coverage recordings would be difficult. If the code is stateful, the unchanged code paths can be affected by the changed code. Changes in configuration files or other external changes can affect the code paths taken. These and other reasons are probably why I've never heard of anything like this before.

No comments:

Post a Comment