This is a series of posts which talk about decision time when a major product such as Microsoft Office is near release, and the product is not yet stable (which means that the team that should sign off the release is uncomfortable with the current quality levels of the application). The release being on schedule is fairly important for the company because of the strategic, and revenue reasons, and delaying it will have an impact. The previous post on this topic (getting information up the management chain) talked about how there should be a process whereby the team does frequent status reviews, keeping upto date with their issues and risks, including the high end ones; and the ones that are more risky are bumped upto the chain of senior management with mitigation strategies and concerns. In addition, periodic status reports that can be summarised to showcase the important issues let senior managers get a good grip on issues that are not going away and which can cause huge amounts of problems.
One of the biggest problems that can be seen is about the rate of bug closure. During the development cycle, there is a certain pattern in terms of defect find rates and defect closure rates, with the expectation that as the team nears the end of the cycle, the rate of defect closure is better than the rate of defect creation. When this does not happen, things get tricky for the development team, since if the number of open defects do not come down, it makes it very difficult to trend down to the point where it is safe in terms of quality to start seeing the application as ready for release. At such times, a pressure to release the product sees the team getting ready to release the product even when they feel that it is not fully ready. In addition, if the number of new defects being found is not declining, it means that the application is still not ready in terms of quality to be released, and no amount of pressure will make the product ready for release. What pressure will do is to make it difficult for team members to admit that the product is not suitable for release (you will need somebody very determined and honest to make the point in front of the management team to argue about the product not being fully ready as yet).
All of these data in terms of the number of defects being filed, the severity of defects being files, the defect closure rate, any abnormal defect deferral rate can all be determined through defect metrics tracking, and it is very important that the team maintains such metrics, since that provides a lot of data that helps in determining whether the quality level of the product is good, or whether it is getting better, or whether there is still some time to go before the product could be good enough to be released.
Read this book for some more advice in this area: Judgement Calls: Making Good Decisions in Difficult Situations –
More about this in the next post (serious communication problems) ..