Software Testing Metrics
Software Testing Metrics provide visibility into both the quality of the test plan as well as the maturity of the product. They enable quantitative insight into the effectiveness of the software testing process and provide feedback as to how to improve the testing process. There is no general consensus on the metrics that should be used in software testing. However, there are several metrics that are in common use that can provide valuable insights.
Test Coverage is one of the most commonly used Software Testing Metrics. Quantitatively, Test Coverage is often defined as the total number of test cases/total number of requirements. The Test Coverage metric gives you an idea of the completeness of your test plan. As new features are added, this metric will momentarily decrease until your test plan starts to incorporate test cases that cover the newer features. Keep in mind that the Test Coverage metric can be defined in other ways. We can also define Test Coverage as the total number of test cases/total number of possible identified paths. This metric should be defined based on organizational and team needs.
Another commonly utilized metric is the Quality Ratio. This typically refers to the number of successfully executed test cases/total number of test cases. This metric provides feedback on the current release quality and the presence or absence of defects. The Quality Ratio can also be categorized by functional areas. This will provide greater insight into where the defects were actually found. Another metric that is related to the Quality Ratio but calculated a little differently is the Defect Density. The Defect Density is usually calculated as the number of defects/number of lines of code. The Quality Ratio metric is a more software tester centric metric whereas the Defect Density is a more developer centric metric.
With respect to automated testing, a frequently used metric is the Automation Index. This is calculated as the number of automatable test cases/total number of test cases. This metric describes the extent to which software testing can be automated. This leads to another automated testing metric sometimes called the Automation Coverage. This is computed as the number of automated test cases/total number of automatable test cases. Just as the Test Coverage metric described you an idea of the completeness of your test plan, the Automation Coverage metric can give you an idea of the completeness of your test automation plan.
Software Testing Metrics such as the ones above should not be used to compare different types of projects as they are less meaningful when used to compare different projects. However, they are extremely valuable as measures of progress within the same project.