Learn Software Development » Testing

Questions regarding software testing and functionality

ashish — Mon, 16 Mar 2009 04:22:50 +0000

A couple of questions related to software testing, especially with regard to functionality ..

What if the application has functionality that wasn’t in the requirements?
One might wonder how it is possible to have a situation where the application has functionality that was not in the requirements; but it is possible. In a badly controlled software project, it is possible that functionality may be included based on direct request from clients; another case is when the project has been partly done by another company or is a migration project. In such cases, the software may have many undocumented functionality. The implications of such functionality are related to the extra effort required for testing, for documentation, for bug fixing, for internationalization.
Given the impact, it is necessary to do the serious effort needed to determine if an application has significant unexpected or hidden functionality, and the very fact that it may be necessary to do this analysis indicates problems in the software development process. What do you do if such functionality is found? If the functionality isn’t necessary to the purpose of the application, a decision should be taken to determine whether it should be removed, as it may have unknown impacts or dependencies that were not taken into account by the designer or the customer.
If not removed, analysis needs to be done to determine added testing needs or regression testing needs (and such extra effort may be non-trivial, or may add to the overall risks). Management should be made aware of any significant added risks as a result of the unexpected functionality. If the functionality only effects areas such as minor improvements in the user interface, for example, it may not be a significant risk. However, in no case should such functionality be taken casually.

What if the software is so buggy it can’t really be tested at all?
One would not like to be in such a situation, but can happen easily enough. Suppose the software cycle has been under a lot of stress in the design and development phase, then code can be real buggy. Further, if the processes related to review and code standards are not implemented, then the software could be real buggy. What does QE do in this case, given that they still need to do their job and complete testing.
The best bet in this situation is for the testers to go through the process of reporting whatever bugs or blocking-type problems initially show up, with the focus being on critical bugs. Since this type of problem can severely affect schedules, and indicates deeper problems in the software development process (such as insufficient unit testing or insufficient integration testing, poor design, improper build or release procedures, etc.) managers should be notified, and provided with some documentation as evidence of the problem. If required, development could be tasked with only bug fixing and no new feature work until a situation is reached where the code is much more bug free; and schedules can also be extended.

Testing of Web Applications

ashish — Sun, 01 Mar 2009 17:35:38 +0000

The quality of a web application can be pretty evident right from the onset (from the beginning of testing). Some of the key things to check for, and that are visible right in the beginning are:
- Slow response time,
- Problems with the accuracy of information,
- Bad design / workflow problems or not having ease of use will compel the user
to click to a competitor’s site
Problems such as these that are easily visible directly translate into loss of users, declining or stagnant sales and a very poor image of the company.

These are outcomes that companies seek to avoid at any costs, and that is why there is the need for a strong QE effort. As a part of this, the following techniques can be used to do a more thorough checking:
1. Good Functionality testing – This sort of testing makes sure that the features that are visible to a user and affect interactions are working properly and as desired.
Some of these objects in a WebApp include: User enterable forms, Searches and their results, Pop-up windows (most users hate them because of their heritage), shopping carts.

2. Usability testing – Many users have a low tolerance for anything that is difficult to use, making having a usability testing program a critical part of the testing of any WebApp. You need to ensure that a user’s first impression is very important; what makes this more complex is that now-a-days applications have become complicated and cluttered with an increasing number of features.

The main steps involved in usability testing are:
– Identify the purpose of WebApp.
– Identify the intended users.
– Define the tests, review them for thoroughness and conduct usability testing.
– Collect information through various mechanisms.
– Carry out an analysis of the acquired information.
– Make the necessary changes based on the acquired information.

3. Navigation testing – Navigation testing makes sure that all navigation syntax and semantics are exercised to uncover any navigation errors. It should not happen that a user clicks on a navigation aid and then either reaches a dead end or goes off into a wrong direction.

4. Forms testing – WebApps that use forms requires tests to ensure that each field is working properly (including the validations such as not allowing users to enter more than a certain amount of text, fields not being left blank, etc) and that the form posts all the data as intended by the designer.

5. Content testing – Content is evaluated at both a syntactic and semantic level.At the syntactic level, spelling, punctuation, and grammar are assessed. At semantic level, correctness, consistency, and lack of ambiguity are all assessed. It creates a very bad impression if the user finds spelling mistakes (user assumes that there must be something wrong if the company put wrong spellings or wrong grammar and did not find it till now, or the company does not care that there are problems on the site)

6. Compatibility testing – This testing is done by executing the webApp under every browser/ platform combination to ensure that the web applications are working properly under different environments. This sort of testing is easier said than done, since the number of browsers and operating systems in the market are huge.

7. Performance testing – This testing evaluates the system performance under normal and heavy usage. An application that takes long to respond may frustrate the user which could result to move to a competitor’s site. This testing ensures that the website server responds to browser requests within defined parameters. The system should work perfectly and speedily under normal expected usage, and if possible, should be handle some amount of extra load.

8. Load testing – The purpose of this testing is to present real world experiences, typically by generating many users simultaneously accessing the web application. Typically, companies use automated test tools to increase the ability to conduct
a valid load test as it emulates thousand of users by sending requests simultaneously to the application. Critical, as failure under heavy loads does not convey a good impression, and may make the system susceptible to attacks by recreating heavy loads.

9. Security testing – Security is one of the primary concerns when communicating and conducting businesses over internet. One break-in can spoil the reputation of a company and lead to loss of business, stealing of user data, and other such cases with horrible consequences. Finding the vulnerabilities in an application that could grant an unauthorized user to access the system is important. Equally important is being able to track all access to the system, and do a frequent scan of these accesses.

WebApp testing – a short summary

ashish — Sun, 01 Mar 2009 05:39:00 +0000

A WebApp or Web Application is a type of application / software that is accessed via a web browser (such as Internet Explorer, Firefox, Opera, Safari) over a network such as the Internet or an intranet. It is typically coded in a browser-supported language (such as HTML, JavaScript, Java, etc.) with the browser environment making the application executable.
With such applications becoming more widespread, testing of such applications is much bigger than it was earlier. Here is a short summary about WebApp testing, and future posts will explore this area in more detail.
WebApp testing is a collection of related activities that has the goal of uncovering the errors in Web Applications related to content, function, usability, navigability, performance, capacity, and security. To accomplish this a testing
strategy that encompasses both reviews and executable testing is applied throughout the Web engineering process.
Generally testing of Web Applications is done by the same set of people who would be involved if the application was a normal client-server application, so webApp testing is done by web engineers and managers, customers, end-users and other stakeholders. This is generic advice for testing, and very relevant for Web Applications. Testing should not wait until the project is finished. It should start before you start thinking of writing a single line of code.
Testing is the process of finding errors and correcting them. The same philosophy goes with Web Applications also. In fact, WebApp testing becomes a more challenging
task for the web engineers as these applications reside on a different network and accessible by varied environments encompassing different operating systems, browsers, platforms. With such varied environments, the possibility of finding errors increases.
Web based applications present new challenges, with some of these challenges are:
1. Short release cycles
2. Constantly changing technology
3. Possible huge number of users during initial website launch
4. Inability to control the user’s running environment
5. 24-hour availability of the web site.

Testing of World Wide Web sites ..

ashish — Tue, 17 Feb 2009 19:32:18 +0000

There are an increasing number of transactions happening on the internet. Whether this be shopping sites, news sites, social networking sites, email sites, media sites, etc., people have come to depend on them to an increasing degree. To ensure that sites are dependable, testing needs to happen. However, testing of internet sites cannot follow the exact same process as that of client-server or other such systems. So the important question is – How can World Wide Web sites be tested?

For those familiar with client-server applications, testing of web sites is somewhat of an advantage, since in a simplistic way, web sites are essentially client/server applications – with web servers and ‘browser’ clients. However, this has a number of complications built in, such as having to consider various factors such as the interactions between html pages, complexity of TCP/IP communications and a wide variety of Internet connections, having to deal with client-side firewalls, applications that run in web pages (such as applets, javascript, plug-in applications), and applications that run on the server side (such as cgi scripts, database interfaces, logging applications, dynamic page generators, asp, etc.). To increase the fun and complexity, given the number of years that the internet has been in existence, there are now a wide variety of servers and browsers (with differenet users having their own preferences for the browser), and with browsers having many versions in co-existence (and as a result, small but sometimes significant differences between them), variations in connection speeds, rapidly changing technologies, and multiple standards and protocols. Starting from a simple client-server architecture, the end result is that testing for web sites can become a major ongoing effort. And these are not all, with other considerations including:
• Estimating the the expected loads on the server (e.g., number of hits per unit time?), and what kind of performance is required under such loads (such as web server response time, database query response times). For this purpose, there is a need to determine what are the kinds of tools will be needed for performance testing (such as web load testing tools, other tools already in house that can be adapted, web robot downloading tools, etc.)?
• Determining the target audience, and also trying to determine the kind of browsers they will be using. What kind of connection speeds will they by using? Are they intra- organization (thus with likely high connection speeds and similar browsers) or Internet-wide (thus with a wide variety of connection speeds and browser types)?
• Performance on the client side is becoming much more important, and some of the ways in which this performance is perceived was: how fast should pages appear, how fast should animations, applets, etc. load and run. How well does the site compare with other sites in terms of performance ?
• Will down time for server and content maintenance/upgrades be allowed? how much? Downtime on the internet world is supposed to be minimal to non-existent.
• What kinds of security (firewalls, encryptions, passwords, etc.) will be required and what is it expected to do? How can it be tested? Testing of security for internet sites is a major effort and needs to be handled professionally and systematically.
• How reliable are the site’s Internet connections required to be? And how does that affect backup system or redundant connection requirements and testing?
• What processes will be required to manage updates to the web site’s content, and what are the requirements for maintaining, tracking, and controlling page content, graphics, links, etc.? Content management for a site needs to work correctly and properly, and transparently to the user.
• Which HTML specification will be adhered to? How strictly? What variations will be allowed for targeted browsers?
• Will there be any standards or requirements for page appearance and/or graphics throughout a site or parts of a site??
• How will internal and external links be validated and updated? how often?
• Can testing be done on the production system, or will a separate test system be required? How are browser caching, variations in browser option settings, dial-up connection variabilities, and real-world internet ‘traffic congestion’ problems to be accounted for in testing?
• How extensive or customized are the server logging and reporting requirements; are they considered an integral part of the system and do they require testing? Server logging and analysing the logs need to be designed properly, otherwise a mistake in data can lead to severe problems.
• How are cgi programs, applets, javascripts, ActiveX components, etc. to be maintained, tracked, controlled, and tested?

What’s the role of documentation in QA?

ashish — Thu, 12 Feb 2009 03:48:10 +0000

Sometimes, for people who are new to the software line (and specifically new to the testing business), there are questions about how important documentation would be to regular Quality work. After all, the major work is about testing some software and getting the bugs resolved, how important could testing be ? Well, the role of documentation in enabling the success of QA activities is critical. Note that documentation can be in electronic form as well, no necessity that it should be only paper, and in fact, recent trends are more towards having electronic forms that can be placed in source control areas in their specific directory locations.
QA practices should be documented such that they are repeatable, and are not dependent on individual people. Software artifacts and processes such as requirement and design specifications, design documents such as architecture, HLD, LLD, business rules, inspection reports, configuration control design and operational documents, code changes, software test plans, test cases, bug reports and decision making on important bugs, user manuals, etc. should all be documented such that these can be referred later (and prove very useful in case the project personnel are changed or should be transitioned to other teams).
As a part of documentation, there needs to be a system for easily finding and obtaining documents and determining what documentation will have a particular piece of information. Change management for documentation should be used in all cases, else you will find later that it is hard to figure out why something changed and what were the reasons behind it.
One of the most common reasons for failures or overruns / delays in a complex software project is to have poorly documented requirements specifications. Requirement specifications are the details describing an application’s externally-perceived functionality and properties. The condition for requirements should be clear, complete, reasonably detailed, cohesive, attainable, and testable. A non-testable requirement would be, for example, ‘user-friendly’ (too subjective). A testable requirement would be something like ‘user needs to enter their date of birth while creating their profile’. Determining and organizing requirements details in a useful and efficient way can be a difficult effort; different methods are available depending on the particular project.
Care should be taken to involve ALL of a project’s significant ‘customers’ and important stakeholders in the requirements process. ‘Customers’ could be in-house personnel or out, and could include end-users, customer acceptance testers, customer contract officers, customer management, future software maintenance engineers, salespeople, etc. Anyone who could later derail the project if their expectations aren’t met should be included if possible. This also helps in ensuring that changes later are minimized (can never be eliminated).
Organizations vary considerably in their handling of requirements specifications. Ideally, the requirements are spelled out in a document with statements such as ‘The product shall…..’. ‘Design’ specifications should not be confused with ‘requirements’; design specifications should be traceable back to the requirements.
In some organizations requirements may end up in high level project plans, functional specification documents, in design documents, or in other documents at various levels of detail. No matter what they are called, some type of documentation with detailed requirements will be needed by testers in order to properly plan and execute tests. Without such documentation, there will be no clear-cut way to determine if a software application is performing correctly.
Test plans need to be documented properly with good change control, since a test plan forms the basis of determining the areas of testing, scope, responsibilities, etc. A test plan forms the first level of generation of confidence in the test strategy for the project.