Learn Software Development » QA

Delivering to multiple clients – need to deliver through the Sprint rather than one deliverable at the end

ashish — Sun, 03 Oct 2010 18:43:32 +0000

We all learn things as we see stuff happening, and one of the items that I learnt was about a team that had changed to using the Scrum model of development, and was learning how to deal with multiple clients. Consider the case of large software companies such as Microsoft, Apple, and many others, where there are many different groups working on different products. However, the products that they are working on have many common needs, such as code for showing UI, code for handling calls to various media items such as disk drives, CD devices, cameras, etc. It does not make sense for separate teams to write their own code for such handling, and hence it is far more feasible for there to be a single team that works on developing these technologies, and the output of these common teams is used by the various product teams.
One of the biggest problems faced by such common teams (or we can call them technology teams) is their need to match their output schedule to the schedules of the various product teams, and this is something that remains in flux. Different products have their own schedules (with these typically not being very flexible, since there could be the need to meet a specific timeline such as having a new version of the product ready for Christmas, or to be available to meet the perception of financial analysts, or to thwart the release of a competitive product, and so on). So, the common situation is that when the technology team starts interactions with the various product teams (and we can call them the client teams), they are presented with schedules that do not seem to easily integrate with each other. One team may want a specific release near the end of a month, while for another group, there is a ‘drop dead’ need to have specific features in by the middle of the month. Reconciling them is easy if one of the teams is much stronger in terms of influence, since then the schedule will be driven by the more powerful team. However, when this is not the case, then the release timelines can get difficult.
There was one such team that was run by a colleague of mine. His team used to work on the traditional waterfall, and that was a disaster zone since they had to forever be ready to release for some team or the other. After much consultation, the team decided that a Scrum model would work better, especially since they could time the length of their Sprints, and have software available in a fairly good state at the end of every Sprint. So, they worked out a Sprint schedule whereby they worked out a Product Owner, and all requests needed to go through the Product Owner, and the time period of every Sprint was 4 weeks.
However, by the start of the second Sprint itself, it became clear that 4 weeks was just not making it. They had expectations from client teams that would just not work based on a release every 4 weeks (for example, there was a team that was very near release, and needed quick support for bug fixes – so they needed a turnaround time of a week for getting a new release that fixed bugs; the monthly cycle would just not work for them).
The technology team did an evaluation, and really did not want to move away from the 4 week sprint cycle (they felt that they were not mature enough in Scrum to try and handle a Sprint cycle of 1-2 weeks, and wanted to ensure that their team got a month to work and release features). Eventually, they came out with a solution that seemed complicated, but that seemed to work. They would still work on a 4 week Sprint cycle, but added some additional tasks to the Sprint backlog for coordinating a weekly release. This required some dedicated coordination from the ScrumMaster and some team members and went through some additional challenges, but after around a month, they were able to do this in a way that met the needs of the client teams. The team would ensure that when tasks were added for the Sprint backlog, there was an effort to time the expected output of the tasks, and then testing was added so that at the end of every week, a release was possible with a high level of quality. One could call this as saying that they had moved to a Sprint cycle of 1 week, but this was not true, specifically when it came to planning for the Sprint (that was done one a monthly basis).

What is code coverage ?

ashish — Sat, 21 Mar 2009 16:12:26 +0000

People know what software testing is, and most people in the profession can differentiate between white box and black box testing. However, when you get into more details, and look to identify how testing can provide even greater value, the benefit of measures such as code coverage become apparent. At the same time, there would be a large number of software professionals who are not even aware of what code coverage is, and what are its key benefits. So the idea of this article is to try and articulate some of the benefits.
So what is Code Coverage ?
Well, Code coverage is a type of measure used in software testing which tries to answer the questions about the degree to which the entire source code of an application has been tested. Traditional black box testing, with its focus on functional testing cannot even come close to trying to answer this question, although White Box testing does come closer to trying to answer this question. In fact, if you examine code coverage practices in detail, it is possible to say that code coverage is a form of testing that inspects the code directly and is therefore a form of white box testing.
Code coverage techniques were amongst the first techniques invented for answering the question of systematic software testing. For those proponents of the text plan / case based method of testing, code coverage testing works on the principle that it is entirely possible that sections of a software application remain untouched by test data, and as a result, then it is not possible to say, with any degree of certainty, that these sections do not contain residual errors.
How do you go ahead with actually trying to do code coverage ?
Code Coverage requires support from engineering to proceed. Why is this so ? Getting code coverage in practise requires a different procedure from the normal software build – The target application / software is configured to be built with special options / libraries and/or run under a special environment such that every function that is exercised (executed) in the program(s) is mapped back to the function points in the source code. Doing this systematic process (although requires effort and the value does not seem immediately clear to individual developed and QE) allows developers and quality assurance personnel to look for parts of a system that are rarely or never accessed under normal conditions (error handling and the like) and helps reassure test engineers that the most important conditions (function points) have been tested. Once this exercise has been done, the output is further analysed to see what areas of code have not been exercised and the tests are updated to include these areas as necessary. Once this exercise has been completed (and it may need to be done on a regular basis as the code is in the process of being developed), it gives a much higher level of confidence about the overall quality of the code.

What’s the role of documentation in QA?

ashish — Thu, 12 Feb 2009 03:48:10 +0000

Sometimes, for people who are new to the software line (and specifically new to the testing business), there are questions about how important documentation would be to regular Quality work. After all, the major work is about testing some software and getting the bugs resolved, how important could testing be ? Well, the role of documentation in enabling the success of QA activities is critical. Note that documentation can be in electronic form as well, no necessity that it should be only paper, and in fact, recent trends are more towards having electronic forms that can be placed in source control areas in their specific directory locations.
QA practices should be documented such that they are repeatable, and are not dependent on individual people. Software artifacts and processes such as requirement and design specifications, design documents such as architecture, HLD, LLD, business rules, inspection reports, configuration control design and operational documents, code changes, software test plans, test cases, bug reports and decision making on important bugs, user manuals, etc. should all be documented such that these can be referred later (and prove very useful in case the project personnel are changed or should be transitioned to other teams).
As a part of documentation, there needs to be a system for easily finding and obtaining documents and determining what documentation will have a particular piece of information. Change management for documentation should be used in all cases, else you will find later that it is hard to figure out why something changed and what were the reasons behind it.
One of the most common reasons for failures or overruns / delays in a complex software project is to have poorly documented requirements specifications. Requirement specifications are the details describing an application’s externally-perceived functionality and properties. The condition for requirements should be clear, complete, reasonably detailed, cohesive, attainable, and testable. A non-testable requirement would be, for example, ‘user-friendly’ (too subjective). A testable requirement would be something like ‘user needs to enter their date of birth while creating their profile’. Determining and organizing requirements details in a useful and efficient way can be a difficult effort; different methods are available depending on the particular project.
Care should be taken to involve ALL of a project’s significant ‘customers’ and important stakeholders in the requirements process. ‘Customers’ could be in-house personnel or out, and could include end-users, customer acceptance testers, customer contract officers, customer management, future software maintenance engineers, salespeople, etc. Anyone who could later derail the project if their expectations aren’t met should be included if possible. This also helps in ensuring that changes later are minimized (can never be eliminated).
Organizations vary considerably in their handling of requirements specifications. Ideally, the requirements are spelled out in a document with statements such as ‘The product shall…..’. ‘Design’ specifications should not be confused with ‘requirements’; design specifications should be traceable back to the requirements.
In some organizations requirements may end up in high level project plans, functional specification documents, in design documents, or in other documents at various levels of detail. No matter what they are called, some type of documentation with detailed requirements will be needed by testers in order to properly plan and execute tests. Without such documentation, there will be no clear-cut way to determine if a software application is performing correctly.
Test plans need to be documented properly with good change control, since a test plan forms the basis of determining the areas of testing, scope, responsibilities, etc. A test plan forms the first level of generation of confidence in the test strategy for the project.

Properties of a test / QA / QE Manager

ashish — Wed, 28 Jan 2009 18:49:39 +0000

Testing is a vital and critical part of the overall software development process, and is very important that the overall testing environment have the right mix of aggression and thoroughness. A large amount of this attitude comes from the person who leads the testing team. So, what makes a good QA or Test manager?
There are many attributes that a good test, or QA manager should have. Here are some of them:
• The test manager should be very familiar with the software development process. This is the only way that the rest of the testing team can develop the feel for when they should be doing what activity.
• The test manager has be able to ensure that the overall enthusiasm of the team remains high, and promote a positive atmosphere, despite what is a somewhat ‘negative’ process (e.g., looking for or preventing problems). People should be made to feel that they have an important role in ensuring that customers get a software that works well.
• The test manager should be able to promote teamwork to increase productivity. Teamwork between the members of the testing team is critical, given that each of them may handle a separate area, and may have several elements of intersection. In addition, each person can have a different field of specialization, and together they can cover a large area.
• The test manager should be able to promote cooperation between software, test, and QA engineers. This is not so easy sometimes, but is very critical. It is a close interaction between dev and QE that results in a deeper understanding of where software can go wrong.
• The test manager have the diplomatic skills needed to promote improvements in QA processes. Sometimes software and hardware can be expensive, and management may not really understand or appreciate the need for such, and it is in such cases that the test manager can better explain.
• The test manager must have the ability to withstand pressures and say ‘no’ to other managers when quality is insufficient or QA processes are not being adhered to. It is the test manager who is responsible for quality.
• The test manager must have people judgement skills for hiring and keeping skilled personnel
• The test manager must be able to communicate with technical and non-technical people, engineers, managers, and customers.