Learn Software Development » Defect http://learnsoftwareprocesses.com All about the processes involved in software development Sun, 20 May 2012 19:17:40 +0000 en hourly 1 What happens when your product is not stable close to release date ? – Part 5 http://learnsoftwareprocesses.com/2012/05/10/what-happens-when-your-product-is-not-stable-close-to-release-date-part-5/ http://learnsoftwareprocesses.com/2012/05/10/what-happens-when-your-product-is-not-stable-close-to-release-date-part-5/#comments Thu, 10 May 2012 20:47:07 +0000 ashish http://learnsoftwareprocesses.com/?p=1092 This is a series of posts which talk about decision time when a major product such as Microsoft Office is near release, and the product is not yet stable (which means that the team that should sign off the release is uncomfortable with the current quality levels of the application). The release being on schedule [...]]]> This is a series of posts which talk about decision time when a major product such as Microsoft Office is near release, and the product is not yet stable (which means that the team that should sign off the release is uncomfortable with the current quality levels of the application). The release being on schedule is fairly important for the company because of the strategic, and revenue reasons, and delaying it will have an impact. The previous post on this topic (getting information up the management chain) talked about how there should be a process whereby the team does frequent status reviews, keeping upto date with their issues and risks, including the high end ones; and the ones that are more risky are bumped upto the chain of senior management with mitigation strategies and concerns. In addition, periodic status reports that can be summarised to showcase the important issues let senior managers get a good grip on issues that are not going away and which can cause huge amounts of problems.
One of the biggest problems that can be seen is about the rate of bug closure. During the development cycle, there is a certain pattern in terms of defect find rates and defect closure rates, with the expectation that as the team nears the end of the cycle, the rate of defect closure is better than the rate of defect creation. When this does not happen, things get tricky for the development team, since if the number of open defects do not come down, it makes it very difficult to trend down to the point where it is safe in terms of quality to start seeing the application as ready for release. At such times, a pressure to release the product sees the team getting ready to release the product even when they feel that it is not fully ready. In addition, if the number of new defects being found is not declining, it means that the application is still not ready in terms of quality to be released, and no amount of pressure will make the product ready for release. What pressure will do is to make it difficult for team members to admit that the product is not suitable for release (you will need somebody very determined and honest to make the point in front of the management team to argue about the product not being fully ready as yet).
All of these data in terms of the number of defects being filed, the severity of defects being files, the defect closure rate, any abnormal defect deferral rate can all be determined through defect metrics tracking, and it is very important that the team maintains such metrics, since that provides a lot of data that helps in determining whether the quality level of the product is good, or whether it is getting better, or whether there is still some time to go before the product could be good enough to be released.

Read this book for some more advice in this area: Judgement Calls: Making Good Decisions in Difficult Situations –

More about this in the next post (serious communication problems) ..

]]> http://learnsoftwareprocesses.com/2012/05/10/what-happens-when-your-product-is-not-stable-close-to-release-date-part-5/feed/ 0 Different types of software errors and handling them properly http://learnsoftwareprocesses.com/2012/02/14/different-types-of-software-errors-and-handling-them-properly/ http://learnsoftwareprocesses.com/2012/02/14/different-types-of-software-errors-and-handling-them-properly/#comments Tue, 14 Feb 2012 18:58:14 +0000 ashish http://learnsoftwareprocesses.com/?p=981 Errors are a major headache to software programmers, developers as well as testers. They cause the whole software system or application to falter, produce unexpected results and cause the software to behave abnormally. Some errors cause more harm while some cause less, some of them are easy to discover whereas some are hideous and cause [...]]]> Errors are a major headache to software programmers, developers as well as testers. They cause the whole software system or application to falter, produce unexpected results and cause the software to behave abnormally. Some errors cause more harm while some cause less, some of them are easy to discover whereas some are hideous and cause huge frustration to developers and testers. Some errors are as active and disruptive like a volcano and others are dormant. Every software system or application suffers from many errors, and it is now seen as impossible to remove all the errors from a software system.
Therefore error handling becomes an important factor in deciding the success of a program. Error handling is the way a program is built to handle the errors that disturb its functioning.
- The error handling procedure should be very strong and smart.
- Error handling requires a lot decision making.
- The error handling process, like other processes, is also a victim of defects and can sometimes not work properly.
The main steps involved in an Error handling process are detection, anticipation and resolution of the errors that occur during the execution of the software program or application. Some applications even employ programs called “error handlers” developed specifically for handling the errors. A software system or application is said to have good error handling capabilities if it is able to recover from errors without causing the whole program to terminate, or if it is not able to handle that error, properly terminate the program without causing any data loss. Such forceful termination is nothing but an error handling mechanism.
The basic factors causing the run time errors are invalid input data and adverse function parameters.
Lack of memory is another defect causing factor.
A Software application comprises of various small programs. These programs may conflict with each other during the run time. Similarly web applications also experience errors due to electrical noise and malware or other undue pressure on the server.
A software system or application can overcome these errors by its error handling process. Thus we can define the error handling defects as the defects that reduce the efficiency of the error handling process.
On the initiation of the error handling process, the discrepancy between the expected behavior and actual behavior is identified. Whenever there is some discrepancy in the behavior of the program, a defect is created. The test script that was being executed at the time of encounter of the defect is tested. This process is called defect creation. After this, the discovered defect is verified i.e., whether or not the defect is valid. A severity level is assigned to the defect. This severity level indicates the impact and visibility of the defect on the program. The defect can cause the core functionality to go out of order or stop working. It can affect the operational environment. Such defects prevent the user from accessing the features and functionalities of the software system or application.
Incorrect navigation links are also a defect.
According to the level of severity of the encountered errors and the problems they can cause, they are assigned priorities. This process is defect prioritization. Several priority codes have been defined. For example, there are some defects that do not even allow the testing to take place. Defects causing such errors are given the highest priority. The defect is once again confirmed and this process is called defect confirmation. After the defect confirmation the defect is analyzed, the affected code is redesigned, developed and tested again for any shortcomings. This process following the defect confirmation is called defect resolution. The defects after being resolved are once again reviewed by the developer and certain test scripts are run to confirm that the defect has been resolved. After the verification the defect is closed.

Lessons Learned in Software Testing Software Testing: An ISTQB-ISEB Foundation Guide Software Testing (2nd Edition)
]]> http://learnsoftwareprocesses.com/2012/02/14/different-types-of-software-errors-and-handling-them-properly/feed/ 0 Bug ageing – why it is important to follow in a software project – Part 3 http://learnsoftwareprocesses.com/2012/01/08/bug-ageing-why-it-is-important-to-follow-in-a-software-project-part-3/ http://learnsoftwareprocesses.com/2012/01/08/bug-ageing-why-it-is-important-to-follow-in-a-software-project-part-3/#comments Sun, 08 Jan 2012 19:20:32 +0000 ashish http://learnsoftwareprocesses.com/?p=950 In the previous 2 instances of this post (Bug ageing and software processes), I have been talking about the need to measure how much time it takes for the bugs generated by the QE team to get fixed, and to see whether taking more time for fixing such bugs is seen as a problem. In [...]]]> In the previous 2 instances of this post (Bug ageing and software processes), I have been talking about the need to measure how much time it takes for the bugs generated by the QE team to get fixed, and to see whether taking more time for fixing such bugs is seen as a problem. In almost every case, you need to ensure that these bugs get fixed within a definite period of time and not keep on lingering on.
One of the important questions is about how soon bugs should be fixed and not kept hanging. It is not easy for a group to define the time period within which bugs should be fixed, and not kept pending. I have tried that, asking a group of development and quality mangers to agree to a time period in which bugs should be fixed (or a bug that is not fixed after say 120 days should be escalated to a bug review team); and there was no agreement even after several rounds of discussion. In the end, there was a proposal that just took an average of the different proposals and we agreed to monitor statistics related to bugs.
What came out very critically from this entire discussion was several points that I will layout in this ongoing discussion:
- It is important to differentiate between bugs depending on their severity. So, a bug that is of higher severity or priority (say an application crash or some problem that related to data loss / corruption) needed to be fixed much earlier than a bug that had a small problem which a user may or may not ever face. Set different standards for these different types of bugs; and trust me, the team will totally agree to the logic (if you don’t do it this way, they will get totally confused about how different bugs whose critically are so different are treated the same).
- Do an analysis of bugs where the age of the bug is higher, and whether those bugs were actually fixed or had to be withdrawn. Our statistic showed us that when a bug is not fixed within a short number of days after being created, then the chances of the bug never being fixed are also higher. This also goes back to the previous post where I outlined some problems in terms of reproducing bugs when the bug has been open for a longer period of time. Do this analysis on your own project, and you might be able to see this pattern (and should worry you, since that means that there are critical bugs that you could have got fixed, but were never fixed).

Contd.. in the next post …

]]> http://learnsoftwareprocesses.com/2012/01/08/bug-ageing-why-it-is-important-to-follow-in-a-software-project-part-3/feed/ 0 Bug ageing – why it is important to follow in a software project – Part 2 http://learnsoftwareprocesses.com/2012/01/04/bug-ageing-why-it-is-important-to-follow-in-a-software-project-part-2/ http://learnsoftwareprocesses.com/2012/01/04/bug-ageing-why-it-is-important-to-follow-in-a-software-project-part-2/#comments Wed, 04 Jan 2012 14:56:58 +0000 ashish http://learnsoftwareprocesses.com/?p=948 In the previous post (Bug ageing and software projects), I talked about the concept of bug ageing and why it is important. In this post, I will continue on that topic and write more about this area. One of the biggest problems with bug ageing is that the more time there is between when a [...]]]> In the previous post (Bug ageing and software projects), I talked about the concept of bug ageing and why it is important. In this post, I will continue on that topic and write more about this area.
One of the biggest problems with bug ageing is that the more time there is between when a bug gets filed and when it finally gets seen, the sequence of steps gets lost. I have seen people insisting that if you document all the steps that you take while filing a bug, you will be able to reproduce the bug. Hogwash. This may not work in every scenario. For example, when somebody is writing the steps for a complicated crasher bug that happens once in a while, there are a number of factors at work that could have caused the crash in the application. It is only the perfect person who is able to reproduce all the twists and turns that they took which caused the crash. Suppose you have an application that allows you to add text, images and sounds while creating a greeting card, a crash that I have seen came during a specific situation when a sound was added, then a video, then text, then the sound was changed, and the text was modified. Now, when you increase the number of steps, it is going to be harder to reproduce such a crash, especially when you are called upon to do so many weeks later. As a result, we set the condition that all crashes need to be looked at within 48 hours of the crash being reported, so that we don’t lose it. The tendency is simple to let the crash go out, since it came in a complicated scenario; but guess what, with so many people coming online to express their opinions, things get very complicated. For a crash, which seemed a low scenario, we had a user lose around 1 hour of effort, and the amount of feedback in user forums was incredible to see; and we use that scenario as an education to people about how not to let such significant items such as crashes go by.
The other major problem with taking time to resolve a bug is due to environmental factors. Suppose a crash is reported, and part of the problem is the interaction of some components on the machine of the tester with some components inside the application. Over a period of time, with system updates, other applications getting modified, or other environmental conditions, this crash or bug is not longer visible. The developer does not have a reproducible scenario, but has to fix and resolve issues. In this case, most of the time, eventually the bug is let go as not reproducible, and we end up with a possible user crash scenario that we could have fixed, but were not able to find because the base environmental factors changed in the meantime. All of these end up with some costs that we have to pay later.

]]> http://learnsoftwareprocesses.com/2012/01/04/bug-ageing-why-it-is-important-to-follow-in-a-software-project-part-2/feed/ 0 Bug ageing – why it is important to follow in a software project http://learnsoftwareprocesses.com/2011/12/19/bug-ageing-why-it-is-important-to-follow-in-a-software-project/ http://learnsoftwareprocesses.com/2011/12/19/bug-ageing-why-it-is-important-to-follow-in-a-software-project/#comments Mon, 19 Dec 2011 11:58:24 +0000 ashish http://learnsoftwareprocesses.com/?p=944 First of all, what is bug ageing ? Well, in a very broad sense, bug ageing is the amount of time that a bug is with a certain person. A bug can follow many different paths during its history – the bug can be with a developer, pending for the bug to be fixed. The [...]]]> First of all, what is bug ageing ? Well, in a very broad sense, bug ageing is the amount of time that a bug is with a certain person. A bug can follow many different paths during its history – the bug can be with a developer, pending for the bug to be fixed. The bug can be with the tester, pending some information to be provided to the developer or when the bug is marked fixed, or the bug can be with management, where they weigh the cost vs. benefits of the bug and decide on a future course of action. In a typical scenario, all these bug actions happen within a short period of time and the bug is resolved (fixed, deferred, or withdrawn) with a short period of time.
However, there are numerous cases where the bug is not fixed quickly. There can be multiple reasons for the same:
- The bug is with the developer, but the developer has a large number of bugs on their plate and hence is not able to give this specific bug the priority that it deserves
- The developer has been told to focus on a specific product area and hence is not able to focus on bugs from other areas, and hence these bugs remain unattended for some time
- The bug is marked in an area which is going to be handled at a future part of the schedule and hence the bug is marked to be fixed when attention is provided to the area further on in the schedule
- The developer wants to get the number of bugs on their plate down, and hence focuses on the bugs that are easier to fix, and tougher bugs have to wait for more time before they are given attention
- It is not easy to recreate the conditions in which the QE found the bug, and hence the developer, showing signs of inertia, is waiting and will push the bug fix when he / she had more time

Similarly, the QE could be having a number of reasons for delaying the bug fix. Some of these are:
- The QE is very busy in testing a specific area of the product, and is not able to spend time on the specific bug
- The bug may have been filed by somebody else, but is assigned to a specific QE because that is their area. As a result, the QE is not very enthused about the bug (no ownership feeling over the bug)
- The QE has a large number of bugs, and the scenario to resolve this bug means using a much older build (and installing the latest build and removing a later build takes time (with larger applications, this could mean as much as 1-2 hours just to install a build)); hence there is inertia in going ahead in recreating the bug

In all such cases, when bugs wait a large amount of time to get resolved, there is a pending time bug. One of the biggest problems in older bugs is that, when a bug would have been filed many months ago, the dev or QE lose track of the scenario in which the bug was filed (even though the steps would have been written down) and find it harder to replicate the bug as time increases.
I will write more about this case in the next post ..

]]> http://learnsoftwareprocesses.com/2011/12/19/bug-ageing-why-it-is-important-to-follow-in-a-software-project/feed/ 0