Learn Software Development » Quality

What happens when your product is not stable close to release date ? – Part 1

ashish — Mon, 07 May 2012 20:19:36 +0000

This is probably one of the most controversial topics in software development. You have a software on which the company has dedicated an insane amount of effort on, the software contributes a significant percentage (upwards of 25%) of the revenue of the company and it is a well know software. This could be MS-Office, could be Photoshop, could be the latest release of Apple OS, or could be other similar software that are very well known. Now, these software have a known release date, and given the power and money riding on these software, it is extremely important that the software makes its release date. The problems if the date gets delayed are many: The company suffers at the hands of the market and analysts, its reputation takes a dip, revenue already slotted for the current quarter has to be scaled back, and somebody in the chain of management has to take a hit.
However, what happens when you reach a scenario where the software is not yet fully ready ? There are still bugs in the software, not everything is fully stable, and quality teams are hesitant to release the software. In some cases, the apprehension of the teams are unfounded, and the software is good enough to release and earn accolades in the market. However, the difficulty comes when the software is not yet ready, when the apprehensions of the quality teams is correct. In such a case, if the software were to be released, things could get problematic for the company real quick. A release that has suspect quality gets pilloried real fast; with more social media and more active consumers, items such as reduced ratings on Amazon and a lot of comments of poor quality on Facebook / twitter can cause huge damage to the reputation of the company and will most certainly lead to forced removal of people in the company.
So the challenge is to determine which is the current state. This is probably one of the most important decisions that the senior management team of the group has to decide, and it is a very hard decision. The chance of making a mistake in this area is incredible, and if there are obvious mistakes, then it is fairly career limiting for the people who did make the mistake. And you certainly don’t want to be the senior management person who released a product that was of bad quality, and at the same time you don’t want to be known as the person who was unable to take a realistic call of whether the product quality was good or bad. The previous sentence was just a confirmation about the difficult position that management is in when trying to evaluate the current position.

Read this book for some more advice in this area: Judgement Calls: Making Good Decisions in Difficult Situations –

In the next part (Part 2), I will talk more about this decision, the factors that guide it, and what all to do ..

What is the difference between static and dynamic testing?

ashish — Mon, 20 Feb 2012 18:28:47 +0000

Static testing and dynamic testing are one of the frequently discussed testing methodologies. Dynamic testing and static testing form two major and extremely important categories of software testing methodologies. Most of us are clear with the meaning of the terms “static” and “dynamic”. Keeping in mid the meaning of these terms, it is very easy to define the terms “static testing” and “dynamic testing”. Static testing and dynamic testing are also known as “static analysis” and “dynamic analysis” respectively. We shall describe these two terms simultaneously in the form of a list of differences between them. So while understanding these two methodologies you might as well as be able to compare them also.
1. Static means stationary, not moving or not in use. So from the term itself we deduce the definition of the static testing i.e., static testing is a testing methodology which practically does not involves the software system or application during the testing process. The term “dynamic” means moving or in use. So for dynamic testing or analysis also we deduce that it is a type of testing which practically involves the software system or application in the testing process.
2. Static testing is all about the static behavior of a software system or application whereas the dynamic testing is concerned about how the software system or application performs dynamically i.e., its dynamic behavior. It usually involves testing of the reflexes of the system i.e., the way the software system responds or reacts towards the variables that are dynamic in nature.
3. Static testing makes use of static variables i.e., the variables which are not suppose to change with time and thus are constant. The dynamic testing makes use of the dynamic variables which are not constant but change with the time along with the execution of the program. It is not necessary that the values should compulsorily change during the execution. The point here is that the dynamic variables have the tendency to get modified whereas the static variables do not possess this tendency or property.
4. If one wants to perform a dynamic test for any program, he/ she need to compile the software, run it and work with it. Dynamic testing involves extensive work of inputting data values to the variables as well as the checking of the output and whether or not it is up to the expectations of the programmer. Like in every other testing methodology here also the actual program output is checked against the desired program output. At the end the Input and output data is checked for validation of the software. Many methodologies have been developed for dynamic testing. Few have been listed below:
Unit tests
System tests
Integration tests and
Acceptance test
Dynamic testing involves a detailed workout. On the other hand, static testing is way too lenient when it comes to the testing efforts. It is not so extensive and is typically involved with the determination of the sanity of the code, document or algorithm. It is pretty much involved with the syntax checking of the program code and manual review of the documentation or the source code to find errors and bugs. This is the basic aim of static testing.
5. Dynamic testing forms an extremely important part of the validation process in contrast to the static testing which forms a major part of the verification process.
6. Functional test techniques or black box testing techniques as they are called, are what are employed by the Dynamic testing process. White box testing techniques or structural test techniques constitute the static testing.

Introduction to Software Testing	Testing Computer Software, 2nd Edition	Software Testing Foundations

Different types of software errors and handling them properly

ashish — Tue, 14 Feb 2012 18:58:14 +0000

Errors are a major headache to software programmers, developers as well as testers. They cause the whole software system or application to falter, produce unexpected results and cause the software to behave abnormally. Some errors cause more harm while some cause less, some of them are easy to discover whereas some are hideous and cause huge frustration to developers and testers. Some errors are as active and disruptive like a volcano and others are dormant. Every software system or application suffers from many errors, and it is now seen as impossible to remove all the errors from a software system.
Therefore error handling becomes an important factor in deciding the success of a program. Error handling is the way a program is built to handle the errors that disturb its functioning.
- The error handling procedure should be very strong and smart.
- Error handling requires a lot decision making.
- The error handling process, like other processes, is also a victim of defects and can sometimes not work properly.
The main steps involved in an Error handling process are detection, anticipation and resolution of the errors that occur during the execution of the software program or application. Some applications even employ programs called “error handlers” developed specifically for handling the errors. A software system or application is said to have good error handling capabilities if it is able to recover from errors without causing the whole program to terminate, or if it is not able to handle that error, properly terminate the program without causing any data loss. Such forceful termination is nothing but an error handling mechanism.
The basic factors causing the run time errors are invalid input data and adverse function parameters.
Lack of memory is another defect causing factor.
A Software application comprises of various small programs. These programs may conflict with each other during the run time. Similarly web applications also experience errors due to electrical noise and malware or other undue pressure on the server.
A software system or application can overcome these errors by its error handling process. Thus we can define the error handling defects as the defects that reduce the efficiency of the error handling process.
On the initiation of the error handling process, the discrepancy between the expected behavior and actual behavior is identified. Whenever there is some discrepancy in the behavior of the program, a defect is created. The test script that was being executed at the time of encounter of the defect is tested. This process is called defect creation. After this, the discovered defect is verified i.e., whether or not the defect is valid. A severity level is assigned to the defect. This severity level indicates the impact and visibility of the defect on the program. The defect can cause the core functionality to go out of order or stop working. It can affect the operational environment. Such defects prevent the user from accessing the features and functionalities of the software system or application.
Incorrect navigation links are also a defect.
According to the level of severity of the encountered errors and the problems they can cause, they are assigned priorities. This process is defect prioritization. Several priority codes have been defined. For example, there are some defects that do not even allow the testing to take place. Defects causing such errors are given the highest priority. The defect is once again confirmed and this process is called defect confirmation. After the defect confirmation the defect is analyzed, the affected code is redesigned, developed and tested again for any shortcomings. This process following the defect confirmation is called defect resolution. The defects after being resolved are once again reviewed by the developer and certain test scripts are run to confirm that the defect has been resolved. After the verification the defect is closed.

Lessons Learned in Software Testing	Software Testing: An ISTQB-ISEB Foundation Guide	Software Testing (2nd Edition)

Delivering to multiple clients – need to deliver through the Sprint rather than one deliverable at the end

ashish — Sun, 03 Oct 2010 18:43:32 +0000

We all learn things as we see stuff happening, and one of the items that I learnt was about a team that had changed to using the Scrum model of development, and was learning how to deal with multiple clients. Consider the case of large software companies such as Microsoft, Apple, and many others, where there are many different groups working on different products. However, the products that they are working on have many common needs, such as code for showing UI, code for handling calls to various media items such as disk drives, CD devices, cameras, etc. It does not make sense for separate teams to write their own code for such handling, and hence it is far more feasible for there to be a single team that works on developing these technologies, and the output of these common teams is used by the various product teams.
One of the biggest problems faced by such common teams (or we can call them technology teams) is their need to match their output schedule to the schedules of the various product teams, and this is something that remains in flux. Different products have their own schedules (with these typically not being very flexible, since there could be the need to meet a specific timeline such as having a new version of the product ready for Christmas, or to be available to meet the perception of financial analysts, or to thwart the release of a competitive product, and so on). So, the common situation is that when the technology team starts interactions with the various product teams (and we can call them the client teams), they are presented with schedules that do not seem to easily integrate with each other. One team may want a specific release near the end of a month, while for another group, there is a ‘drop dead’ need to have specific features in by the middle of the month. Reconciling them is easy if one of the teams is much stronger in terms of influence, since then the schedule will be driven by the more powerful team. However, when this is not the case, then the release timelines can get difficult.
There was one such team that was run by a colleague of mine. His team used to work on the traditional waterfall, and that was a disaster zone since they had to forever be ready to release for some team or the other. After much consultation, the team decided that a Scrum model would work better, especially since they could time the length of their Sprints, and have software available in a fairly good state at the end of every Sprint. So, they worked out a Sprint schedule whereby they worked out a Product Owner, and all requests needed to go through the Product Owner, and the time period of every Sprint was 4 weeks.
However, by the start of the second Sprint itself, it became clear that 4 weeks was just not making it. They had expectations from client teams that would just not work based on a release every 4 weeks (for example, there was a team that was very near release, and needed quick support for bug fixes – so they needed a turnaround time of a week for getting a new release that fixed bugs; the monthly cycle would just not work for them).
The technology team did an evaluation, and really did not want to move away from the 4 week sprint cycle (they felt that they were not mature enough in Scrum to try and handle a Sprint cycle of 1-2 weeks, and wanted to ensure that their team got a month to work and release features). Eventually, they came out with a solution that seemed complicated, but that seemed to work. They would still work on a 4 week Sprint cycle, but added some additional tasks to the Sprint backlog for coordinating a weekly release. This required some dedicated coordination from the ScrumMaster and some team members and went through some additional challenges, but after around a month, they were able to do this in a way that met the needs of the client teams. The team would ensure that when tasks were added for the Sprint backlog, there was an effort to time the expected output of the tasks, and then testing was added so that at the end of every week, a release was possible with a high level of quality. One could call this as saying that they had moved to a Sprint cycle of 1 week, but this was not true, specifically when it came to planning for the Sprint (that was done one a monthly basis).

Engineering Practices Not Taken Care of by Scrum – Some details

ashish — Tue, 31 Aug 2010 19:07:46 +0000

Scrum never talks about engineering practices which is a good thing but this sometimes lands teams into trouble. The teams not only adopt the Scrum practices but also the principles which makes their progress sluggish and results in the code base being in a mess. This is a situation which is present in most of the cases where a failure has happened and the teams state that they have been implementing Scrum. Given that it is far easier to blame a process rather than do the necessary hard introspection, teams blame Scrum about not being able to provide engineering practices but they miss the entire point; Scrum is not about engineering practices, it’s about management. Engineering practices are whole and sole responsibility of the teams. Scrum only creates organized teams. They organize themselves, they organize their tools and they organize their engineering practices, and Scrum then opens up the entire process in terms of more information that teams have to learn to handle, and use to improve the way that they do things.
Consider some examples: The definition of Done (DoD) is defined by the development team. The criteria for done can be different for different teams; some teams might consider a piece done once the code has been reviewed others may consider is done only then functional tests are carried out and still others may call it done when the documentation is also done. It becomes necessary to keep an optimal level of done so that technical debt does not come into picture too much. And what is ‘technical debt’ ? Well, the concept of technical debt states that while adding a piece of functionality, two different approaches can be used. The first one is doing a quick job and the other one is a clean design. The quick design would mostly be messy, and one of the bigger problems is that making further changes become harder in future. So doing things in a quick and dirty way, results in technical debt which in turn requires extra effort to be put when future development is done. Doing a quick job is useful when you need to meet a deadline and can afford the extra effort later. However, you need to be sure that this is the approach you want to take, and not be surprised at the extra effort needed to be put in later.
The main difference between classical approach (Waterfall, etc) and scrum approach of development is that in scrum developers have more freedom to define level of done and amount of work to be done. They decide on the User stories to work in each of the sprint (through the estimation, although the priority comes from the Product Owner). Developers who do not use professional practices fail; but they fail independently of the processes used; blaming the processes of Scrum is convenient, but some inspection by external parties can easily reveal problems in the way that they do things.
There is sometimes a talk about too much quality being pumped into the product. Sometimes it may be a cause of concern for the owner if the team takes too much time for quality. The concept of iron triangle comes into picture here. This concepts states that Scope (features, functionalities etc.) , Resources (cost) and Schedule (time) can be considered to be three vertices of a triangle and Quality is the area enclosed by this triangle. Different groups have different priorities: users want more scope, senior management wants cut in time, financial team wants cut in budget and development team wants more quality. The end aim is to strike a balance into all the four entities. Scrum sets resources and time and lets developers decide the scope. Speed is scope/time and it’s a measure of output. If the product owner wants a higher velocity then he must work towards removing impediments; but dropping of engineering practices is not a solution. Compromise in quality is a slippery slope best avoided, and can result in so many problems later that any benefits from reducing quality are to be avoided. Consider advising a stuntman to drop safety equipment; he will never and so should be the case with developers (and even more so with the people responsible for the quality of the features and the product).