Learn Software Development » QE

Delivering to multiple clients – need to deliver through the Sprint rather than one deliverable at the end

ashish — Sun, 03 Oct 2010 18:43:32 +0000

We all learn things as we see stuff happening, and one of the items that I learnt was about a team that had changed to using the Scrum model of development, and was learning how to deal with multiple clients. Consider the case of large software companies such as Microsoft, Apple, and many others, where there are many different groups working on different products. However, the products that they are working on have many common needs, such as code for showing UI, code for handling calls to various media items such as disk drives, CD devices, cameras, etc. It does not make sense for separate teams to write their own code for such handling, and hence it is far more feasible for there to be a single team that works on developing these technologies, and the output of these common teams is used by the various product teams.
One of the biggest problems faced by such common teams (or we can call them technology teams) is their need to match their output schedule to the schedules of the various product teams, and this is something that remains in flux. Different products have their own schedules (with these typically not being very flexible, since there could be the need to meet a specific timeline such as having a new version of the product ready for Christmas, or to be available to meet the perception of financial analysts, or to thwart the release of a competitive product, and so on). So, the common situation is that when the technology team starts interactions with the various product teams (and we can call them the client teams), they are presented with schedules that do not seem to easily integrate with each other. One team may want a specific release near the end of a month, while for another group, there is a ‘drop dead’ need to have specific features in by the middle of the month. Reconciling them is easy if one of the teams is much stronger in terms of influence, since then the schedule will be driven by the more powerful team. However, when this is not the case, then the release timelines can get difficult.
There was one such team that was run by a colleague of mine. His team used to work on the traditional waterfall, and that was a disaster zone since they had to forever be ready to release for some team or the other. After much consultation, the team decided that a Scrum model would work better, especially since they could time the length of their Sprints, and have software available in a fairly good state at the end of every Sprint. So, they worked out a Sprint schedule whereby they worked out a Product Owner, and all requests needed to go through the Product Owner, and the time period of every Sprint was 4 weeks.
However, by the start of the second Sprint itself, it became clear that 4 weeks was just not making it. They had expectations from client teams that would just not work based on a release every 4 weeks (for example, there was a team that was very near release, and needed quick support for bug fixes – so they needed a turnaround time of a week for getting a new release that fixed bugs; the monthly cycle would just not work for them).
The technology team did an evaluation, and really did not want to move away from the 4 week sprint cycle (they felt that they were not mature enough in Scrum to try and handle a Sprint cycle of 1-2 weeks, and wanted to ensure that their team got a month to work and release features). Eventually, they came out with a solution that seemed complicated, but that seemed to work. They would still work on a 4 week Sprint cycle, but added some additional tasks to the Sprint backlog for coordinating a weekly release. This required some dedicated coordination from the ScrumMaster and some team members and went through some additional challenges, but after around a month, they were able to do this in a way that met the needs of the client teams. The team would ensure that when tasks were added for the Sprint backlog, there was an effort to time the expected output of the tasks, and then testing was added so that at the end of every week, a release was possible with a high level of quality. One could call this as saying that they had moved to a Sprint cycle of 1 week, but this was not true, specifically when it came to planning for the Sprint (that was done one a monthly basis).

What is code coverage ?

ashish — Sat, 21 Mar 2009 16:12:26 +0000

People know what software testing is, and most people in the profession can differentiate between white box and black box testing. However, when you get into more details, and look to identify how testing can provide even greater value, the benefit of measures such as code coverage become apparent. At the same time, there would be a large number of software professionals who are not even aware of what code coverage is, and what are its key benefits. So the idea of this article is to try and articulate some of the benefits.
So what is Code Coverage ?
Well, Code coverage is a type of measure used in software testing which tries to answer the questions about the degree to which the entire source code of an application has been tested. Traditional black box testing, with its focus on functional testing cannot even come close to trying to answer this question, although White Box testing does come closer to trying to answer this question. In fact, if you examine code coverage practices in detail, it is possible to say that code coverage is a form of testing that inspects the code directly and is therefore a form of white box testing.
Code coverage techniques were amongst the first techniques invented for answering the question of systematic software testing. For those proponents of the text plan / case based method of testing, code coverage testing works on the principle that it is entirely possible that sections of a software application remain untouched by test data, and as a result, then it is not possible to say, with any degree of certainty, that these sections do not contain residual errors.
How do you go ahead with actually trying to do code coverage ?
Code Coverage requires support from engineering to proceed. Why is this so ? Getting code coverage in practise requires a different procedure from the normal software build – The target application / software is configured to be built with special options / libraries and/or run under a special environment such that every function that is exercised (executed) in the program(s) is mapped back to the function points in the source code. Doing this systematic process (although requires effort and the value does not seem immediately clear to individual developed and QE) allows developers and quality assurance personnel to look for parts of a system that are rarely or never accessed under normal conditions (error handling and the like) and helps reassure test engineers that the most important conditions (function points) have been tested. Once this exercise has been done, the output is further analysed to see what areas of code have not been exercised and the tests are updated to include these areas as necessary. Once this exercise has been completed (and it may need to be done on a regular basis as the code is in the process of being developed), it gives a much higher level of confidence about the overall quality of the code.

Questions regarding software testing and functionality

ashish — Mon, 16 Mar 2009 04:22:50 +0000

A couple of questions related to software testing, especially with regard to functionality ..

What if the application has functionality that wasn’t in the requirements?
One might wonder how it is possible to have a situation where the application has functionality that was not in the requirements; but it is possible. In a badly controlled software project, it is possible that functionality may be included based on direct request from clients; another case is when the project has been partly done by another company or is a migration project. In such cases, the software may have many undocumented functionality. The implications of such functionality are related to the extra effort required for testing, for documentation, for bug fixing, for internationalization.
Given the impact, it is necessary to do the serious effort needed to determine if an application has significant unexpected or hidden functionality, and the very fact that it may be necessary to do this analysis indicates problems in the software development process. What do you do if such functionality is found? If the functionality isn’t necessary to the purpose of the application, a decision should be taken to determine whether it should be removed, as it may have unknown impacts or dependencies that were not taken into account by the designer or the customer.
If not removed, analysis needs to be done to determine added testing needs or regression testing needs (and such extra effort may be non-trivial, or may add to the overall risks). Management should be made aware of any significant added risks as a result of the unexpected functionality. If the functionality only effects areas such as minor improvements in the user interface, for example, it may not be a significant risk. However, in no case should such functionality be taken casually.

What if the software is so buggy it can’t really be tested at all?
One would not like to be in such a situation, but can happen easily enough. Suppose the software cycle has been under a lot of stress in the design and development phase, then code can be real buggy. Further, if the processes related to review and code standards are not implemented, then the software could be real buggy. What does QE do in this case, given that they still need to do their job and complete testing.
The best bet in this situation is for the testers to go through the process of reporting whatever bugs or blocking-type problems initially show up, with the focus being on critical bugs. Since this type of problem can severely affect schedules, and indicates deeper problems in the software development process (such as insufficient unit testing or insufficient integration testing, poor design, improper build or release procedures, etc.) managers should be notified, and provided with some documentation as evidence of the problem. If required, development could be tasked with only bug fixing and no new feature work until a situation is reached where the code is much more bug free; and schedules can also be extended.

What’s the role of documentation in QA?

ashish — Thu, 12 Feb 2009 03:48:10 +0000

Sometimes, for people who are new to the software line (and specifically new to the testing business), there are questions about how important documentation would be to regular Quality work. After all, the major work is about testing some software and getting the bugs resolved, how important could testing be ? Well, the role of documentation in enabling the success of QA activities is critical. Note that documentation can be in electronic form as well, no necessity that it should be only paper, and in fact, recent trends are more towards having electronic forms that can be placed in source control areas in their specific directory locations.
QA practices should be documented such that they are repeatable, and are not dependent on individual people. Software artifacts and processes such as requirement and design specifications, design documents such as architecture, HLD, LLD, business rules, inspection reports, configuration control design and operational documents, code changes, software test plans, test cases, bug reports and decision making on important bugs, user manuals, etc. should all be documented such that these can be referred later (and prove very useful in case the project personnel are changed or should be transitioned to other teams).
As a part of documentation, there needs to be a system for easily finding and obtaining documents and determining what documentation will have a particular piece of information. Change management for documentation should be used in all cases, else you will find later that it is hard to figure out why something changed and what were the reasons behind it.
One of the most common reasons for failures or overruns / delays in a complex software project is to have poorly documented requirements specifications. Requirement specifications are the details describing an application’s externally-perceived functionality and properties. The condition for requirements should be clear, complete, reasonably detailed, cohesive, attainable, and testable. A non-testable requirement would be, for example, ‘user-friendly’ (too subjective). A testable requirement would be something like ‘user needs to enter their date of birth while creating their profile’. Determining and organizing requirements details in a useful and efficient way can be a difficult effort; different methods are available depending on the particular project.
Care should be taken to involve ALL of a project’s significant ‘customers’ and important stakeholders in the requirements process. ‘Customers’ could be in-house personnel or out, and could include end-users, customer acceptance testers, customer contract officers, customer management, future software maintenance engineers, salespeople, etc. Anyone who could later derail the project if their expectations aren’t met should be included if possible. This also helps in ensuring that changes later are minimized (can never be eliminated).
Organizations vary considerably in their handling of requirements specifications. Ideally, the requirements are spelled out in a document with statements such as ‘The product shall…..’. ‘Design’ specifications should not be confused with ‘requirements’; design specifications should be traceable back to the requirements.
In some organizations requirements may end up in high level project plans, functional specification documents, in design documents, or in other documents at various levels of detail. No matter what they are called, some type of documentation with detailed requirements will be needed by testers in order to properly plan and execute tests. Without such documentation, there will be no clear-cut way to determine if a software application is performing correctly.
Test plans need to be documented properly with good change control, since a test plan forms the basis of determining the areas of testing, scope, responsibilities, etc. A test plan forms the first level of generation of confidence in the test strategy for the project.

Properties of a test / QA / QE Manager

ashish — Wed, 28 Jan 2009 18:49:39 +0000

Testing is a vital and critical part of the overall software development process, and is very important that the overall testing environment have the right mix of aggression and thoroughness. A large amount of this attitude comes from the person who leads the testing team. So, what makes a good QA or Test manager?
There are many attributes that a good test, or QA manager should have. Here are some of them:
• The test manager should be very familiar with the software development process. This is the only way that the rest of the testing team can develop the feel for when they should be doing what activity.
• The test manager has be able to ensure that the overall enthusiasm of the team remains high, and promote a positive atmosphere, despite what is a somewhat ‘negative’ process (e.g., looking for or preventing problems). People should be made to feel that they have an important role in ensuring that customers get a software that works well.
• The test manager should be able to promote teamwork to increase productivity. Teamwork between the members of the testing team is critical, given that each of them may handle a separate area, and may have several elements of intersection. In addition, each person can have a different field of specialization, and together they can cover a large area.
• The test manager should be able to promote cooperation between software, test, and QA engineers. This is not so easy sometimes, but is very critical. It is a close interaction between dev and QE that results in a deeper understanding of where software can go wrong.
• The test manager have the diplomatic skills needed to promote improvements in QA processes. Sometimes software and hardware can be expensive, and management may not really understand or appreciate the need for such, and it is in such cases that the test manager can better explain.
• The test manager must have the ability to withstand pressures and say ‘no’ to other managers when quality is insufficient or QA processes are not being adhered to. It is the test manager who is responsible for quality.
• The test manager must have people judgement skills for hiring and keeping skilled personnel
• The test manager must be able to communicate with technical and non-technical people, engineers, managers, and customers.