All About Software Testing

Software testing life cycle

2010-10-07T05:13:00.001-07:00

Software testing life cycle identifies what test activities to carry out and when (what is the best time) to accomplish those test activities. Even though testing differs between organizations, there is a testing life cycle.

Software Testing Life Cycle consists of six (generic) phases:

* Test Planning,
* Test Analysis,
* Test Design,
* Construction and verification,
* Testing Cycles,
* Final Testing and Implementation and
* Post Implementation.

Software testing has its own life cycle that intersects with every stage of the SDLC. The basic requirements in software testing life cycle is to control/deal with software testing – Manual, Automated and Performance.
Test Planning

This is the phase where Project Manager has to decide what things need to be tested, do I have the appropriate budget etc. Naturally proper planning at this stage would greatly reduce the risk of low quality software. This planning will be an ongoing process with no end point.

Activities at this stage would include preparation of high level test plan-(according to IEEE test plan template The Software Test Plan (STP) is designed to prescribe the scope, approach, resources, and schedule of all testing activities. The plan must identify the items to be tested, the features to be tested, the types of testing to be performed, the personnel responsible for testing, the resources and schedule required to complete testing, and the risks associated with the plan.). Almost all of the activities done during this stage are included in this software test plan and revolve around a test plan.
Test Analysis

Once test plan is made and decided upon, next step is to delve little more into the project and decide what types of testing should be carried out at different stages of SDLC, do we need or plan to automate, if yes then when the appropriate time to automate is, what type of specific documentation I need for testing.

Proper and regular meetings should be held between testing teams, project managers, development teams, Business Analysts to check the progress of things which will give a fair idea of the movement of the project and ensure the completeness of the test plan created in the planning phase, which will further help in enhancing the right testing strategy created earlier. We will start creating test case formats and test cases itself. In this stage we need to develop Functional validation matrix based on Business Requirements to ensure that all system requirements are covered by one or more test cases, identify which test cases to automate, begin review of documentation, i.e. Functional Design, Business Requirements, Product Specifications, Product Externals etc. We also have to define areas for Stress and Performance testing.
Test Design

Test plans and cases which were developed in the analysis phase are revised. Functional validation matrix is also revised and finalized. In this stage risk assessment criteria is developed. If you have thought of automation then you have to select which test cases to automate and begin writing scripts for them. Test data is prepared. Standards for unit testing and pass / fail criteria are defined here. Schedule for testing is revised (if necessary) & finalized and test environment is prepared.
Construction and verification

In this phase we have to complete all the test plans, test cases, complete the scripting of the automated test cases, Stress and Performance testing plans needs to be completed. We have to support the development team in their unit testing phase. And obviously bug reporting would be done as when the bugs are found. Integration tests are performed and errors (if any) are reported.
Testing Cycles

In this phase we have to complete testing cycles until test cases are executed without errors or a predefined condition is reached. Run test cases --> Report Bugs --> revise test cases (if needed) --> add new test cases (if needed) --> bug fixing --> retesting (test cycle 2, test cycle 3….).
Final Testing and Implementation

In this we have to execute remaining stress and performance test cases, documentation for testing is completed / updated, provide and complete different matrices for testing. Acceptance, load and recovery testing will also be conducted and the application needs to be verified under production conditions.
Post Implementation
In this phase, the testing process is evaluated and lessons learnt from that testing process are documented. Line of attack to prevent similar problems in future projects is identified. Create plans to improve the processes. The recording of new errors and enhancements is an ongoing process. Cleaning up of test environment is done and test machines are restored to base lines in this stage.

MUTATION TESTING

2010-10-07T05:12:00.001-07:00

A kind of testing in which, the application is tested for the code that was modified after fixing a particular bug/defect. It also helps in finding out which code and which strategy of coding can help in developing the functionality effectively.

Proof of Concept & Interoperability Testing

2010-06-09T23:41:00.000-07:00

Interoperability testing has become a requirement for companies that deploy multi-vendor networks. To satisfy this requirement, network and storage providers and managers have three options.

1. Set up an interoperability lab, an expensive and time-consuming project.

2. Use a third-party interoperability lab, such as ISOCORE or the University of New Hampshire.

3. Create a proof-of-concept lab, such as the labs at Cisco or Spirent Communications.

These labs typically connect the devices with a copper or fiber-optic patch cable and run the tests. Such testing reflects a best-case scenario that is useful for base-line interoperability testing but doesn’t represent how the devices will interoperate in an actual network.

It is analogous to testing the auto-pilot system of an airplane to see if it could land the plane in ideal weather conditions. While the test proves the plane can land itself on a perfect day, it is not a predictor of how the system will behave in wide range of weather conditions under which a plane will have to operate.

Sometimes spools of fiber are used to create the delay found in wide area networks. While this is an improvement over patch cables, there are three major limitations to proof of concept and interoperability testing with spools of fiber:

1. Spools of fiber cannot provide dynamic tests. The tests must be manually stopped and restarted to change out the spool of fiber for one with a different length.

2. Spools of fiber are expensive and impractical. Imagine the cost associated with moving an 50,000 km spool of fiber to another lab.

3. Spools of fiber only provide delay. They do not address the various other impairments that exist in a network.

It is as if we improved our auto-pilot system testing to include fixed amounts of wind from a single direction. Factors such as fog, rain, snow and wind sheers are still ignored in the testing. Weather conditions are dynamic and multifaceted. Testing under a single condition is not a realistic test. As with weather, so it is with networks. Impairments in real networks do not limit themselves to a single issue.

2010-06-09T04:31:00.001-07:00

www.qaamitahuja.blogspot.com

Iterative Model

2009-10-14T23:27:00.000-07:00

An iterative lifecycle model does not attempt to start with a full specification of requirements. Instead, development begins by specifying and implementing just part of the software, which can then be reviewed in order to identify further requirements. This process is then repeated, producing a new version of the software for each cycle of the model. Consider an iterative lifecycle model which consists of repeating the following four phases in sequence:

A Requirements phase, in which the requirements for the software are gathered and analyzed. Iteration should eventually result in a requirements phase that produces a complete and final specification of requirements. - A Design phase, in which a software solution to meet the requirements is designed. This may be a new design, or an extension of an earlier design.

An Implementation and Test phase, when the software is coded, integrated and tested.

A Review phase, in which the software is evaluated, the current requirements are reviewed, and changes and additions to requirements proposed.

For each cycle of the model, a decision has to be made as to whether the software produced by the cycle will be discarded, or kept as a starting point for the next cycle (sometimes referred to as incremental prototyping). Eventually a point will be reached where the requirements are complete and the software can be delivered, or it becomes impossible to enhance the software as required, and a fresh start has to be made.

The iterative lifecycle model can be likened to producing software by successive approximation. Drawing an analogy with mathematical methods that use successive approximation to arrive at a final solution, the benefit of such methods depends on how rapidly they converge on a solution.

The key to successful use of an iterative software development lifecycle is rigorous validation of requirements, and verification (including testing) of each version of the software against those requirements within each cycle of the model. The first three phases of the example iterative model is in fact an abbreviated form of a sequential V or waterfall lifecycle model. Each cycle of the model produces software that requires testing at the unit level, for software integration, for system integration and for acceptance. As the software evolves through successive cycles, tests have to be repeated and extended to verify each version of the software.

Amit Ahuja

Test Cases

2009-10-06T03:45:00.000-07:00

Designing Test Cases

A test case is a detailed procedure that fully tests a feature or an aspect of a feature. Whereas the test plan describes what to test, a test case describes how to perform a particular test. You need to develop a test case for each test listed in the test plan. Figure 2.10 illustrates the point at which test case design occurs in the lab development and testing process.

Test case includes:

* The purpose of the test.

* Special hardware requirements, such as a modem.

* Special software requirements, such as a tool.

* Specific setup or configuration requirements.

* A description of how to perform the test.

* The expected results or success criteria for the test.

Test cases should be written by a team member who understands the function or technology being tested, and each test case should be submitted for peer review.

Organizations take a variety of approaches to documenting test cases; these range from developing detailed, recipe-like steps to writing general descriptions. In detailed test cases, the steps describe exactly how to perform the test. In descriptive test cases, the tester decides at the time of the test how to perform the test and what data to use.

Most organizations prefer detailed test cases because determining pass or fail criteria is usually easier with this type of case. In addition, detailed test cases are reproducible and are easier to automate than descriptive test cases. This is particularly important if you plan to compare the results of tests over time, such as when you are optimizing configurations. Detailed test cases are more time-consuming to develop and maintain. On the other hand, test cases that are open to interpretation are not repeatable and can require debugging, consuming time that would be better spent on testing.

Test Case Design

Test Case ID:

It is unique number given to test case in order to be identified.

Test description:

The description if test case you are going to test.

Revision history:

Each test case has to have its revision history in order to know when and by whom it is created or modified.

Function to be tested:

The name of function to be tested.

Environment:

It tells in which environment you are testing.

Test Setup:

Anything you need to set up outside of your application for example printers, network and so on.

Test Execution:

It is detailed description of every step of execution.

Expected Results:

The description of what you expect the function to do.

Actual Results:

pass / failed

If pass - What actually happen when you run the test.

If failed - put in description of what you've observed.

Amit Ahuja

Black Box Testing

2009-10-02T23:13:00.000-07:00

Testing not based on any knowledge of the internal design or code. Tests are based on requirements and functionality.

Amit Ahuja

Who should test? and what? – An Overview

2009-09-30T23:18:00.000-07:00

Who should do the testing?

Software Testing is not the job of one person. It is teamwork. The size of the team depends upon the complexity and size of the software being tested. The software developer should have minimum or no role in the testing process as everyone knows that for a person who has developed the software, it is very difficult to pin point errors in his own creations.

Seven soft skills crucial for a good tester are – that He or She must:

1) Be Cautious
2) Be Curious
3) Have Patience
4) Have Perseverance
5) Be Analytical
6) Be Critical but with an attitude of not jumping to conclusions.
7) Be Good Communicator

Thus “More the soft skills a tester has, better tester He or She is”

Above all, a Most Successful Tester is the one:

1) Who is completely passionate.

2) Who is always eager to learn more & more.

3) Who never gives up.

Various personnel and their roles during development & testing are:

A) Customer: Provides funding, provides requirements, approves changes and some of the test results.

B) Project Manager: Plans and manages the project.

C) Software Developer: Designs the codes and builds the software.

D) Testing coordinator: Creates test plans and test specifications based on the requirements and functional and technical documents.

E) Testing person: Executes the tests and documents the results.

Role based demarcation exclusive to the field of Testing:

1) Junior Software Testers: Are the ones having good theoretical knowledge of testing & would have participated in good number of seminars on testing or would have passed some courses in testing. Such individuals are not expected to have experience on testing; however little experience can be desirable.

2) Software Testers: Are the ones having good ability to understand testing packs, do testing, and doing documentation / defect logging. Such individuals usually do repetitive work & are invariably involved in front-end testing. Usually at least 6 months of testing experience is preferred for this role.

3) Senior Software Testers: Are the ones having responsibility of doing all tasks usually performed by a Tester. In addition to this they are involved in back-end testing. The senior testers also do updating of test cases. Usually 1 to 2 years of experience of testing is preferred for this role.

4) Testing Analysts: Are the ones having expertise in extracting requirements out of documentation & doing thorough verification with business & ascertaining the correctness of all the information. The testing analysts also do writing of test cases, executing the tests & reporting the findings thereafter. Usually 2 to 3 years of experience of testing along with at least 1-year experience of analysis is preferred for this role.

5) Testing Managers: Are the ones having experience of doing all the tasks described above. Apart from this they possess the ability of managing the entire testing process, personnel as well as the testing environment. Usually 3 to 4 years of experience of testing along with at least 1-year of managerial experience & sound expertise of Project Management is ideal for this role.

6) Testing Consultants: Are the senior people having experience of doing all the tasks described above. Consultants are usually good communicators having proven ability of man management, with the help of which they are able to effectively handle the client & the senior executives of the organization. This includes doing systematic analysis of the current testing process of the client & ability to guide the client with their expert comments & recommendations to improve. Consultants have great expertise of manual testing in addition to sound knowledge of specialized fields like Automation, Usability & Security etc.

Generally persons with 5 years or more of Testing & Project Management experience in some senior capacity are appointed as testing consultants.

7) Test Automators: Are the ones having good skills of development plus sound experience of manual testing & have acquired specialization of Automated Testing; but as of date they would not prefer to get involved in manual testing. Automators have thorough knowledge & experience of using different Automation Tools. Usually 1 to 2 years of experience of test automation is preferred for this role.

Next question comes as to what should be tested?

According to Myer (The great Testing Guru) - for most of the programs, it is impractical to attempt to test the program with all possible inputs, due to a combinatorial explosion. For those inputs selected, a testing oracle is needed to determine the correctness of the output for particular test input.

Myer also says that, for most programs, it is impractical to attempt to test all execution paths through the product, due to combinatorial explosion. It is also not possible to develop an algorithm for generating test data for paths in an arbitrary product, due to the inability to determine path feasibility.

The point, which is being emphasized here, is that complete or exhaustive testing is just not possible. This is because exhaustive testing requires every statement in the program and every possible path combination to be executed at once.

Amit Ahuja

Free Download SQL Books

2009-09-24T23:37:00.000-07:00

1) Teach Yourself SQL in 21 Days

Amit Ahuja

Waterfall Model

2009-09-24T03:11:00.000-07:00

There are various software development approaches defined and designed which are used/employed during development process of software, these approaches are also referred as "Software Development Process Models".

Waterfall Model

Waterfall approach was first Process Model to be introduced and followed widely in Software Engineering to ensure success of the project. In "The Waterfall" approach, the whole process of software development is divided into separate process phases.

The phases in Waterfall model are: Requirement Specifications phase, Software Design, Implementation and Testing & Maintenance. All these phases are cascaded to each other so that second phase is started as and when defined set of goals are achieved for first phase and it is signed off, so the name "Waterfall Model". All the methods and processes undertaken in Waterfall Model are more visible.

The stages of "The Waterfall Model" are:

Requirement Analysis & Definition: All possible requirements of the system to be developed are captured in this phase. Requirements are set of functionalities and constraints that the end-user (who will be using the system) expects from the system. The requirements are gathered from the end-user by consultation, these requirements are analyzed for their validity and the possibility of incorporating the requirements in the system to be development is also studied. Finally, a Requirement Specification document is created which serves the purpose of guideline for the next phase of the model.

System & Software Design: Before a starting for actual coding, it is highly important to understand what we are going to create and what it should look like? The requirement specifications from first phase are studied in this phase and system design is prepared. System Design helps in specifying hardware and system requirements and also helps in defining overall system architecture. The system design specifications serve as input for the next phase of the model.

Implementation & Unit Testing: On receiving system design documents, the work is divided in modules/units and actual coding is started. The system is first developed in small programs called units, which are integrated in the next phase. Each unit is developed and tested for its functionality; this is referred to as Unit Testing. Unit testing mainly verifies if the modules/units meet their specifications.

Integration & System Testing: As specified above, the system is first divided in units which are developed and tested for their functionality. These units are integrated into a complete system during Integration phase and tested to check if all modules/units coordinate between each other and the system as a whole behaves as per the specifications. After successfully testing the software, it is delivered to the customer.

Operations & Maintenance: This phase of "The Waterfall Model" is virtually never ending phase (Very long). Generally, problems with the system developed (which are not found during the development life cycle) come up after its practical use starts, so the issues related to the system are solved after deployment of the system. Not all the problems come in picture directly but they arise time to time and needs to be solved; hence this process is referred as Maintenance.

Advantages and Disadvantages

Advantages

The advantage of waterfall development is that it allows for departmentalization and managerial control. A schedule can be set with deadlines for each stage of development and a product can proceed through the development process like a car in a carwash, and theoretically, be delivered on time. Development moves from concept, through design, implementation, testing, installation, troubleshooting, and ends up at operation and maintenance. Each phase of development proceeds in strict order, without any overlapping or iterative steps.

Disadvantages

The disadvantage of waterfall development is that it does not allow for much reflection or revision. Once an application is in the testing stage, it is very difficult to go back and change something that was not well-thought out in the concept stage. Alternatives to the waterfall model include joint application development (JAD), rapid application development (RAD), synch and stabilize, build and fix, and the spiral model.

Common Errors in Requirements Analysis

In the traditional waterfall model of software development, the first phase of requirements analysis is also the most important one. This is the phase which involves gathering information about the customer's needs and defining, in the clearest possible terms, the problem that the product is expected to solve.

This analysis includes understanding the customer's business context and constraints, the functions the product must perform, the performance levels it must adhere to, and the external systems it must be compatible with. Techniques used to obtain this understanding include customer interviews, use cases, and "shopping lists" of software features. The results of the analysis are typically captured in a formal requirements specification, which serves as input to the next step.

Well, at least that's the way it's supposed to work theoretically. In reality, there are a number of problems with this theoretical model, and these can cause delays and knock-on errors in the rest of the process. This article discusses some of the more common problems that project managers experience during this phase, and suggests possible solutions.

Problem 1: Customers don't (really) know what they want

Possibly the most common problem in the requirements analysis phase is that customers have only a vague idea of what they need, and it's up to you to ask the right questions and perform the analysis necessary to turn this amorphous vision into a formally-documented software requirements specification that can, in turn, be used as the basis for both a project plan and an engineering architecture.

To solve this problem, you should:

* Ensure that you spend sufficient time at the start of the project on understanding the objectives, deliverables and scope of the project.
* Make visible any assumptions that the customer is using, and critically evaluate both the likely end-user benefits and risks of the project.
* Attempt to write a concrete vision statement for the project, which encompasses both the specific functions or user benefits it provides and the overall business problem it is expected to solve.
* Get your customer to read, think about and sign off on the completed software requirements specification, to align expectations and ensure that both parties have a clear understanding of the deliverabl

Problem 2: Requirements change during the course of the project

The second most common problem with software projects is that the requirements defined in the first phase change as the project progresses. This may occur because as development progresses and prototypes are developed, customers are able to more clearly see problems with the original plan and make necessary course corrections; it may also occur because changes in the external environment require reshaping of the original business problem and hence necessitates a different solution than the one originally proposed.

Good project managers are aware of these possibilities and typically already have backup plans in place to deal with these changes.

To solve this problem, you should:

* Have a clearly defined process for receiving, analyzing and incorporating change requests, and make your customer aware of his/her entry point into this process.
* Set milestones for each development phase beyond which certain changes are not permissible -- for example, disallowing major changes once a module reaches 75 percent completion.
* Ensure that change requests (and approvals) are clearly communicated to all stakeholders, together with their rationale, and that the master project plan is updated accordingly.

Problem 3: Customers have unreasonable timelines

It's quite common to hear a customer say something like "it's an emergency job and we need this project completed in X weeks". A common mistake is to agree to such timelines before actually performing a detailed analysis and understanding both of the scope of the project and the resources necessary to execute it. In accepting an unreasonable timeline without discussion, you are, in fact, doing your customer a disservice: it's quite likely that the project will either get delayed (because it wasn't possible to execute it in time) or suffer from quality defects (because it was rushed through without proper inspection).

To solve this problem, you should:

* Convert the software requirements specification into a project plan, detailing tasks and resources needed at each stage and modeling best-case, middle-case and worst-case scenarios.

* Ensure that the project plan takes account of available resource constraints and keeps sufficient time for testing and quality inspection.

* Enter into a conversation about deadlines with your customer, using the figures in your draft plan as supporting evidence for your statements. Assuming that your plan is reasonable, it's quite likely that the ensuing negotiation will be both productive and result in a favorable outcome for both parties.

Problem 4: Communication gaps exist between customers, engineers and project managers

Often, customers and engineers fail to communicate clearly with each other because they come from different worlds and do not understand technical terms in the same way. This can lead to confusion and severe miscommunication, and an important task of a project manager, especially during the requirements analysis phase, is to ensure that both parties have a precise understanding of the deliverable and the tasks needed to achieve it.

To solve this problem, you should:

* Take notes at every meeting and disseminate these throughout the project team.

* Be consistent in your use of words. Make yourself a glossary of the terms that you're going to use right at the start, ensure all stakeholders have a copy, and stick to them consistently.

Problem 5: The development team doesn't understand the politics of the customer's organization

The scholars Bolman and Deal suggest that an effective manager is one who views the organization as a "contested arena" and understands the importance of power, conflict, negotiation and coalitions. Such a manager is not only skilled at operational and functional tasks, but he or she also understands the importance of framing agendas for common purposes, building coalitions that are united in their perspective, and persuading resistant managers of the validity of a particular position.

These skills are critical when dealing with large projects in large organizations, as information is often fragmented and requirements analysis is hence stymied by problems of trust, internal conflicts of interest and information inefficiencies.

To solve this problem, you should:

* Review your existing network and identify both the information you need and who is likely to have it.
* Cultivate allies, build relationships and think systematically about your social capital in the organization.
* Persuade opponents within your customer's organization by framing issues in a way that is relevant to their own experience.
* Use initial points of access/leverage to move your agenda forward.

Kinds of Testing that should be considered for Websites.

2009-09-24T02:06:00.001-07:00

Black box testing – Testing not based on any knowledge of the internal design or code. Tests are based on requirements and functionality.

Incremental Integration Testing – Continuous testing of website as new functionality is added; requires that various aspects of an web applications functionality be independent enough to work separately before all parts of the program are completed.

Integration Testing – Testing that is done on combined parts of an application to determine if they function together correctly. The ‘parts’ can be code modules, individual applications, pages in a website etc.

Functional Testing – Black box type testing geared to functional requirements of an application; testers should do this type of testing. This doesn’t mean that the programmers shouldn’t check that their code works before releasing it (which of course applies to any stage of testing.)

System Testing – Black box type testing that is based on overall requirements specifications; covers all combined parts of a web application.

End-to-end Testing – Similar to system testing; the ‘macro’ end of the test scale; involves testing of a complete application environment in a situation that mimics real-world use, such as interacting with a database, using network communications, or interacting with other hardware, applications, or systems if appropriate.

Sanity Testing or Smoke Testing – typically an initial testing effort to determine if a new web application is performing well enough to accept it for a major testing effort. For example, if there are lots of missing links, missing images, missing validations, or corrupting databases, the website may not be in a ’sane’ enough condition to warrant further testing in its current state.

Regression Testing – re-testing after fixes or modifications of the website or its pages. It can be difficult to determine how much re-testing is needed, especially near the end of the development cycle. Automated testing tools can be especially useful for this type of testing.

Acceptance Testing – final testing based on specifications of the end-user or customer, or based on use by end-users/customers over some limited period of time.

Load Testing – testing of a web site under a range of loads to determine at what point the system’s response time degrades or fails.

Stress Testing – term often used interchangeably with ‘load’ and ‘performance’ testing. Also used to describe such tests as system functional testing while under unusually heavy loads, heavy repetition of certain actions or inputs, input of large numerical values, large complex queries to a database system, etc.

Performance Testing – term often used interchangeably with ’stress’ and ‘load’ testing. Ideally ‘performance’ testing (and any other ‘type’ of testing) is defined in requirements documentation or QA or Test Plans.

Usability Testing – testing that is done for ‘user-friendliness’. Clearly this is subjective, and will depend on the targeted end-user or customer. User interviews, surveys, video recording of user sessions, and other techniques can be used. Programmers and testers are usually not appropriate as usability testers.

Security Testing – testing how well the system protects against unauthorized internal or external access, willful damage, etc; may require sophisticated testing techniques.

Compatibility Testing – testing how well website performs in a particular hardware/software/operating system//browser/network etc. environment.

Exploratory Testing – often taken to mean a creative, informal software test that is not based on formal test plans or test cases; testers may be learning the website as they test it.

Ad-hoc Testing – similar to exploratory testing, but often taken to mean that the testers have significant understanding of the website before testing it.

User acceptance Testing – determining if website is satisfactory to an end-user or customer.

Alpha Testing – testing of a web application when development is nearing completion; minor design changes may still be made as a result of such testing. Typically done by end-users or others, not by programmers or testers.

Beta Testing – testing when development and testing are essentially completed and final bugs and problems need to be found before final release. Typically done by end-users or others, not by programmers or testers.

Bug Life Cycle

2009-03-18T05:01:00.001-07:00

The steps in defect life cycle varies from company to company. But the basic flow remains the same. However, below I'm describing a basic flow for Bug Life Cycle:

* A Tester finds a bug. Status --> Open
* Test lead review the bug and authoriz the bug. Stats --> Open
* Development team lead review the defect. Stats --> Open
* The defect can be authorized or unauthorized by the development team. (Here the status of the defect / bug will be Open (For Authorized Defects) & Reject (For Unauthorized Defects).
* Now, the authorized bugs will get fixed or deferred by the development team. Status of the fixed bugs will be Fixed & Status will be Deferred for the bugs which got Deferred.
* The Fixed bugs will be again re-tested by the testing team (Here based on the Closure of the Bug, the status will be made as Closed or if the bug still remains, it will be re-raised and status will be Re-opened.

The above-mentioned cycle continues until all the bugs / defects gets fixed in the application.

Software Testing Bug Report Template

2009-03-18T05:00:00.001-07:00

In continuation to my previous post, here in this post, I'm explaining a simple and effective software bug report.

If you are using any Software Testing Management tool or any Bug reporting tool
like Bugzilla or Test Director or Bughost or any other online bug tracking tool, then; the tool will automatically generate the bug report. If you are not using any tool, you may refer to the following template for your software bug report:

* Name of Reporter:
* Email Id of Reporter:
* Version or Build:
* Module or component:
* Platform / Operating System:
* Type of error:
* Priority:
* Severity:
* Status:
* Assigned to:
* Summary:
* Description:

Software Testing Bug Report

2009-03-18T04:58:00.000-07:00

After you complete your Software Testing, it is good practice to prepare an effective bug report. Fixing a bug depends on how effectively you report it. Below are some tips to write a good software bug report:

* If you are doing manual Software Testing and reporting bugs withour the help of any tool, assign a unique number to each bug report. This will help to identify the bug record.
* Clearly mention the steps to reproduce the bug. Do not assume or skip any reproducing step.
* Be Specific and to the point

Apart from these tips, below are some good practices:

* Report the problem immediately
* Reproduce the bug atleast one more time before you report it
* Test the same bug occurrence on other similar modules of the application
* Read bug report before you submit it or send it.
* Never ever criticize any developer or attack any individual

Validation vs Verification, Reviews, Inspections

2009-03-18T03:46:00.000-07:00

Validation: The process of evaluating software at the end of the software development
process to ensure compliance with software requirements. It is actual testing of the application.

1. Am I building the right product.

2. Determining if the system complies with the requirements and performs functions for which it is intended and meets the organization’s goals and user needs. It is traditional and is performed at the end of the project.

3. Am I accessing the right data (in terms of the data required to satisfy the requirement).

4. High level activity.

5. Performed after a work product is produced against established criteria ensuring that the product integrates correctly into the environment.

6. Determination of correctness of the final software product by a development project with respect to the user needs and requirements.

Verification: The process of determining whether of not the products of a given phase of the software development cycle meet the implementation steps and can be traced to the incoming objectives established during the previous phase.

Verification process helps in detecting defects early, and preventing their leakage downstream. Thus, the higher cost of later detection and rework is eliminated. It includes:

1. Am I building the product right.

2. The review of interim work steps and interim deliverables during a project to ensure they are acceptable. To determine if the system is consistent, adheres to standards, uses reliable techniques and prudent practices, and performs the selected functions in the correct manner.

3. Am I accessing the data right (in the right place; in the right way).

4. Low level activity.

5. Performed during development on key artifacts, like walkthroughs, reviews and inspections, mentor feedback, training, checklists and standards.

6. Demonstration of consistency, completeness, and correctness of the software at each stage and between each stage of the development life cycle.

Review: A process or meeting during which a work product, or set of work products, is presented to project personnel, managers, users, customers, or other interested parties for comment or approval. The main goal of reviews is to find defects. Reviews are a good compliment to testing to help assure quality. A few purposes’ of SQA reviews can be as follows:

Assure the quality of deliverable’s before the project moves to the next stage.
Once a deliverable has been reviewed, revised as required, and approved, it can be used as a basis for the next stage in the life cycle.

Various types of reviews:

Management Reviews: Management reviews are performed by those directly responsible for the system in order to monitor progress, determine status of plans and schedules, confirm requirements and their system allocation. Therefore the main objectives of Management Reviews can be categorized as follows:

- Validate from a management perspective that the project is making progress according to the project plan.

- Ensure that deliverables are ready for management approvals.

- Resolve issues that require management’s attention.

- Identify any project bottlenecks.

- Keeping project in Control.

Requirement Review: A process or meeting during which the requirements for a system, hardware item, or software item are presented to project personnel, managers, users, customers, or other interested parties for comment or approval. Types include system requirements review, software requirements review. Product management leads Requirement Review. Members from every affected department participates in the review.

Input Criteria: Software requirement specification is the essential document for the review. A checklist can be used for the review.

Exit Criteria: Exit criteria include the filled & completed checklist with the reviewers’ comments & suggestions and the re-verification whether they are incorporated in the documents.

Design Review: A process or meeting during which a system, hardware, or software design is presented to project personnel, managers, users, customers, or other interested parties for comment or approval. Types include critical design review, preliminary design review, and system design review.

Inspection: A static analysis technique that relies on visual examination of development products to detect errors, violations of development standards, and other problems. Types include code inspection; design inspection, Architectural inspections, Test ware inspections etc. The participants in Inspections assume one or more of the following roles:

- Inspection leader
- Recorder
- Reader
- Author
- Inspector

All participants in the review are inspectors. The author shall not act as inspection leader and should not act as reader or recorder. Other roles may be shared among the team members. Individual participants may act in more than one role.

Individuals holding management positions over any member of the inspection team shall not participate in the inspection.

Input to the inspection shall include the following:

- A statement of objectives for the inspection
- The software product to be inspected
- Documented inspection procedure
- Inspection reporting forms
- Current anomalies or issues list

Input to the inspection may also include the following:

- Inspection checklists: Any regulations, standards, guidelines, plans, and procedures against which the software product is to be inspected.

- Hardware product specifications

- Hardware performance data

- Anomaly categories

The individuals may make additional reference material available responsible for the software product when requested by the inspection leader.

The purpose of the exit criteria is to bring an unambiguous closure to the inspection meeting. The exit decision shall determine if the software product meets the inspection exit criteria and shall prescribe any appropriate rework and verification. Specifically, the inspection team shall identify the software product disposition as one of the following:

Accept with no or minor rework: The software product is accepted as is or with only minor rework. (For example, that would require no further verification).

Accept with rework verification: The software product is to be accepted after the inspection leader or a designated member of the inspection team (other than the author) verifies rework.

Re-inspect: Schedule a re-inspection to verify rework. At a minimum, a re-inspection shall examine the software product areas changed to resolve anomalies identified in the last inspection, as well as side effects of those changes.

Software Testing Principles

2009-03-18T03:43:00.000-07:00

Below are some basic Software Testing Principles:

- A necessary part of a test case is a definition of the expected output or result.

- A programmer should avoid attempting to test his or her own program.

- A programming organization should not test its own programs.

- Thoroughly inspect the results of each test.

- Test cases must be written for input conditions that are invalid and unexpected, as well as for those that are valid and expected.

- Examining a program to see if it does not do what it is supposed to do is only half the battle; the other half is seeing whether the program does what it is not supposed to do.

- Avoid throwaway test cases unless the program is truly a throwaway program.

- Do not plan a testing effort under the tacit assumption that no errors will be found.

- The probability of the existence of more errors in a section of a program is proportional to the number of errors already found in that section.

- Software Testing is an extremely creative and intellectually challenging task.

Software Testing Process

2009-03-18T03:25:00.000-07:00

Below is a very basic software testing process
. Many companies use this process. Later on I'll post some more testing processes.

1. Understand of business logic and analysis of requirements: In this phase consider the following:

* Are the definitions and descriptions of the required functionalities precise?
* Is there clear delineation between the system and its environment?
* Can the requirements be realized in practice?
* Can the requirements be tested effectively?

2. Test Planning: During this phase Test Strategy is defined and Test Bed is created. The plan should identify:-

* Which aspects of the system should be tested.
* The methods, techniques and tools to be used.
* Personnel responsible for the testing.
* Manual and Automation Testing
* Defect Management and Risk Management etc.

3. Test Environment Setup: A different testing server is prepared where the application will be tested. It is an independent testing environment.

4. Test Design: Identify the test scenarios and prepare the test cases / scripts. Selection of test data is also done in this phase. If required, test designing is done with some automated tools like QTP or LoadRunner or with some other software testing tool. Designing framework, scripting, script integration, Review and approval will be undertaken in this phase.

5. Test Execution: Testers execute the test cases and report any errors found to the development team.

6. Defect Tracking: Raised defects are tracked using some tools like Test Director or Bug Host etc.

7. Test Reports: As soon as testing is completed, Test Lead or Manager generate metrics and make final reports for the whole testing effort.

The Big Picture (Software testing)

2009-03-13T04:32:00.000-07:00

All software problems can be termed as bugs. A software bug usually occurs when the software

does not do what it is intended to do or does something that it is not intended to do. Flaws in specifications, design, code or other reasons can cause these bugs. Identifying and fixing bugs in the early stages of the software is very important as the cost of fixing bugs grows over time. So, the goal of a software tester is to find bugs and find them as early as possible and make sure they are fixed.

Testing is context-based and risk-driven. It requires a methodical and disciplined approach to finding bugs. A good software tester needs to build credibility and possess the attitude to be explorative, troubleshooting, relentless, creative, diplomatic and persuasive.

As against the perception that testing starts only after the completion of coding phase, it actually begins even before the first line of code can be written. In the life cycle of the conventional software product, testing begins at the stage when the specifications are written, i.e. from testing the product specifications or product spec. Finding bugs at this stage can save huge amounts of time and money.

Once the specifications are well understood, you are required to design and execute the test cases. Selecting the appropriate technique that reduces the number of tests that cover a feature is one of the most important things that you need to take into consideration while designing these test cases. Test cases need to be designed to cover all aspects of the software, i.e. security, database, functionality (critical and general) and the user interface. Bugs originate when the test cases are executed.

As a tester you might have to perform testing under different circumstances, i.e. the application could be in the initial stages or undergoing rapid changes, you have less than enough time to test, the product might be developed using a life cycle model that does not support much of formal testing or retesting. Further, testing using different operating systems, browsers and the configurations are to be taken care of.

Reporting a bug may be the most important and sometimes the most difficult task that you as a software tester will perform. By using various tools and clearly communicating to the developer, you can ensure that the bugs you find are fixed.

Using automated tools to execute tests, run scripts and tracking bugs improves efficiency and effectiveness of your tests. Also, keeping pace with the latest developments in the field will augment your career as a software test engineer.

Requirements Testing

2009-02-20T21:03:00.000-08:00

Testing software is an integral part of building a system. However, if the software is based on inaccurate requirements, then despite well written code, the software will be unsatisfactory. Most of the defects in a system can be traced back to wrong, missing, vague or incomplete requirements.

Requirements seem to be ephemeral. They flit in and out of projects, they are capricious, intractable, unpredictable and sometimes invisible. When gathering requirements we are searching for all of the criteria for a system's success. We throw out a net and try to capture all these criteria.

The Quality Gateway

As soon as we have a single requirement in our net we can start testing. The aim is to trap requirements-related defects as early as they can be identified. We prevent incorrect requirements from being incorporated in the design and implementation where they will be more difficult and expensive to find and correct.

To pass through the quality gateway and be included in the requirements specification, a requirement must pass a number of tests. These tests are concerned with ensuring that the requirements are accurate, and do not cause problems by being unsuitable for the design and implementation stages later in the project.

Make The Requirement Measurable

In his work on specifying the requirements for buildings, Christopher Alexander describes setting up a quality measure for each requirement.

"The idea is for each requirement to have a quality measure that makes it possible to divide all solutions to the requirement into two classes: those for which we agree that they fit the requirement and those for which we agree that they do not fit the requirement."

In other words, if we specify a quality measure for a requirement, we mean that any solution that meets this measure will be acceptable. Of course it is also true to say that any solution that does not meet the measure will not be acceptable.

The quality measures will be used to test the new system against the requirements. The remainder of this paper describes how to arrive at a quality measure that is acceptable to all the stakeholders.

Quantifiable Requirements

Consider a requirement that says "The system must respond quickly to customer enquiries". First we need to find a property of this requirement that provides us with a scale for measurement within the context. Let's say that we agree that we will measure the response using minutes. To find the quality measure we ask: "under what circumstances would the system fail to meet this requirement?" The stakeholders review the context of the system and decide that they would consider it a failure if a customer has to wait longer than three minutes for a response to his enquiry. Thus "three minutes" becomes the quality measure for this requirement.

Any solution to the requirement is tested against the quality measure. If the solution makes a customer wait for longer than three minutes then it does not fit the requirement. So far so good: we have defined a quantifiable quality measure. But specifying the quality measure is not always so straightforward. What about requirements that do not have an obvious scale?

Non-quantifiable Requirements

Suppose a requirement is "The automated interfaces of the system must be easy to learn". There is no obvious measurement scale for "easy to learn". However if we investigate the meaning of the requirement within the particular context, we can set communicable limits for measuring the requirement.

Again we can make use of the question: "What is considered a failure to meet this requirement?" Perhaps the stakeholders agree that there will often be novice users, and the stakeholders want novices to be productive within half an hour. We can define the quality measure to say "a novice user must be able to learn to successfully complete a customer order transaction within 30 minutes of first using the system". This becomes a quality measure provided a group of experts within this context is able to test whether the solution does or does not meet the requirement.

An attempt to define the quality measure for a requirement helps to rationalise fuzzy requirements. Something like "the system must provide good value" is an example of a requirement that everyone would agree with, but each person has his own meaning. By investigating the scale that must be used to measure "good value" we identify the diverse meanings.

Sometimes by causing the stakeholders to think about the requirement we can define an agreed quality measure. In other cases we discover that there is no agreement on a quality measure. Then we substitute this vague requirement with several requirements, each with its own quality measure.

Requirements Test 1

Does each requirement have a quality measure that can be used to test whether any solution meets the requirement?

By adding a quality measure to each requirement we have made the requirement visible. This is the first step to defining all the criteria for measuring the goodness of the solution. Now let's look at other aspects of the requirement that we can test before deciding to include it in the requirements specification.

Requirements Test 2

Does the specification contain a definition of the meaning of every essential subject matter term within the specification?

When the allowable values for each of the attributes are defined it provides data that can be used to test the implementation.

Requirements Test 3

Is every reference to a defined term consistent with its definition?

Requirements Test 4

Is the context of the requirements wide enough to cover everything we need to understand?

Requirements Test 5

Have we asked the stakeholders about conscious, unconscious and undreamed of requirements?

Requirements Test 5 (enlarged)

Have we asked the stakeholders about conscious, unconscious and undreamed of requirements? Can you show that a modelling effort has taken place to discover the unconscious requirements? Can you demonstrate that brainstorming or similar efforts taken place to find the undreamed of requirements?

Requirements Test 6

Is every requirement in the specification relevant to this system?

Requirements Test 7

Does the specification contain solutions posturing as requirements?

Requirements Test 8

Is the stakeholder value defined for each requirement?

Requirements Test 9

Is each requirement uniquely identifiable?

Requirements Test 10

Is each requirement tagged to all parts of the system where it is used? For any change to requirements, can you identify all parts of the system where this change has an effect?

Conclusions

The requirements specification must contain all the requirements that are to be solved by our system. The specification should objectively specify everything our system must do and the conditions under which it must perform. Management of the number and complexity of the requirements is one part of the task.

The most challenging aspect of requirements gathering is communicating with the people who are supplying the requirements. If we have a consistent way of recording requirements we make it possible for the stakeholders to participate in the requirements process. As soon as we make a requirement visible we can start testing it. and asking the stakeholders detailed questions. We can apply a variety of tests to ensure that each requirement is relevant, and that everyone has the same understanding of its meaning. We can ask the stakeholders to define the relative value of requirements. We can define a quality measure for each requirement, and we can use that quality measure to test the eventual solutions.

Testing starts at the beginning of the project, not at the end of the coding. We apply tests to assure the quality of the requirements. Then the later stages of the project can concentrate on testing for good design and good code. The advantages of this approach are that we minimise expensive rework by minimising requirements-related defects that could have been discovered, or prevented, early in the project's life.

I18N Testing

2009-02-20T04:35:00.001-08:00

Products developed in one location are used all over the world with different languages and regional standards. This arises the need to test product in different languages and different regional standards. Multilingual and localization testing can increase your products usability and acceptability worldwide.

Internationalization testing is the process, which ensures that product’s functionality is not broken and all the messages are properly externalized when used in different languages and locale. Internationalization testing is also called I18N testing, because there are 18 characters between I and N in Internationalization.

Internationalization, Globalization and Localization, all these words are normally used together. Though the objective of these word is same, that is to make sure that product is ready for the global market, but they serve different purpose and have different meaning.

I hope this article has given you better clarity about I18N testing

Functional Testing The only Answer to Quality

2009-02-20T04:24:00.000-08:00

Functional Testing refers to the type of testing which ensures that all functional requirements are met without any consideration to the final program structure. Functional Testing confirms that the application under development is capable to deliver as per user requirements. Functional Testing emulates the actions performed by the user and ensures that all execution paths are operating perfectly as desired in addition we are able to get the desired for the inputs supplied to the system.

1) Perform Unit Testing & ensure proper execution of each & every line of the code:

The software developers tend to design the code in isolation. When there is absence of pair programming, full code reviews & highly experienced developers, there is bound to be the possibility of inclusion of defects in the new code. Such defects if not detected during early stages of SDLC are quite difficult as well as expensive to detect later on as the project moves.

Hence a strong unit testing process is the backbone of the testing process, upon which the entire reliability of the product depends. Unit testing refers to the process of testing each & every unit of the code going down to the single component level. The developer during the development of the component does the unit testing. It is the responsibility of the developer to ensure that each & every part of the code is logically correct. Unit tests usually provide following type of coverage:

Function coverage: Ensures that every function or method is getting executed by at least one test case.

Statement coverage: Ensures that every line of the code gets covered by a minimum of one test case.

Path coverage: Ensures that every possible path through code gets covered by a minimum of one test case. There can be more number of test cases if need be.

Unit testing helps the developers to regularly ensure that every unit of code performs as expected. Unit tests are modified on continuous basis as the evolution process of the software continues. This helps in maintaining an updated documentation as well.

2) Perform Functional Testing & ensure expected results from every function:

As a part of the testing strategy all expected outcomes must be confirmed by functional testing.
All the function points of various lines of code must yield expected outcome which must be in line with the functional specification described in the specification document.

Functional testing takes care of all concerns revolving around the proper implementation of functional requirements. Commonly known as black box testing, it requires no prior knowledge of the basic code.

Functional test cases are created from requirement use cases wherein every scenario becomes a functional test. As the individual software components get implemented, after successful unit testing the corresponding functional tests are employed on them.

For some software projects, it is not feasible to test every functional aspect. Instead of that appropriate functional testing goals are defined. Critical and commonly used functions are prioritized according to the limitations of resources & time.

“Reporting bugs - a how-to guide”

2009-02-19T23:39:00.000-08:00

When working with a developer or team of developers on an application – whether you are a designer working with developers or an end client hiring developers – you all want the same end result, a slick and bug free application. During the testing process of any application it is likely that some bugs or issues will show up and this article aims to explain how to report bugs and problems effectively so that your developers don’t need to spend time working out what the problem is before being able to fix it. This helps to ensure that projects stay on budget and that developers are spending their time adding features to the application rather than trying to get enough details to be able to reproduce and fix issues.

“It’s just not working!”

When you find a problem, it is very tempting to just fire off an email and presume that the developer will immediately be able to see the problem too. However, by taking a few minutes to describe the problem you have encountered accurately you can prevent any confusion occurring as to what the problem is and save both your time and the developer’s as she won’t need to get back to you to find out what actually happened, or spend a long time trying to reproduce the issue.

A good report

A good bug report tells your developer three vital things:

* What you expected to happen
* What actually happened
* What you did/were doing when it happened

What you expected to happen

There are two kinds of ‘bugs’, the first is where something breaks – you see an error message, your uploaded data disappears, you submit a form and the change isn’t saved. These bugs are generally pretty easy to report and identify as all your developer needs is to know exactly what you were doing or inputting at the time and they should be able to reproduce and fix the issue.

The second kind of bug is where the application doesn’t function as you expected. This might be because the developer has misinterpreted part of the specification or it could be that what you expect just isn’t how something can work. In this case the developer believes that it is working fine – and in fact it is ‘working’ even if it is incorrect. If your bug report is that the feature is broken, the developer may then spend time looking for some error in this part of the application when what they need to realize is that it isn’t working as you expected. By giving the information about what you expected to happen the developer can think ‘ah … you wanted it to do x and it is doing y’ and a resolution can be sorted out quickly.

What actually happened

What actually happened is very rarely ‘nothing’ yet bug reports often contain the phrase, ‘nothing happened’. If what happened was ‘nothing’ in terms of the intended result then explain that in a few more words, for example, if you clicked the submit button on a form and it didn’t submit and go onto the next page you could say,

“The form didn’t submit – it just remained on the same page.”

Or perhaps the form submitted and a blank page displayed,

“After submitting the form a blank page loaded.”

If an error message displays on the screen then include that in the report. Just copy and paste the error message.

If you use Internet Explorer then your browser may not display the error message generated by the server, instead showing a generic error page. You can ensure the IE displays the real error message by going to Tools > Internet Options > Advanced. Then scroll down to the browsing section and uncheck ‘Show Friendly HTTP error messages’.
What you were doing when it happened

Your developer wants to know this information – not because they want to tell you that you were doing something wrong, but because it is highly likely that the bug occurs only when a certain path of actions is followed, or when a certain type of data is entered. The more information you can give your developer the easier it will be for them to reproduce the problem you saw and fix it. Things you should include:

The steps taken

List exactly what you did, in the order you did it if possible. If you can go back and try the same steps again and the problem happens again that is great – note down exactly how you made the problem occur. Your developer will be pleased as you have just saved her time trying to reproduce the issue. Even if you can’t reproduce it, no-one is going to doubt that the problem happened – just describe as much as you can remember how you got to the broken point.

Any data you were entering

If the problem happened after you added some data to a form, include the data with the bug report. If you were uploading something such as an image into the application then include that too.

It may also be helpful to copy and paste the URL out of the address bar of the browser so the developer knows exactly which page you were on at the time.

The browser and operating system you were using at the time

With web applications problems may only be occurring in one browser. Let your developer know exactly what you are using – including the version number - so they can create the same environment to test the problem.

Effective bug reporting can make a huge difference in how quickly problems can be resolved, and prevent frustration on both sides of the process. Including the above information, even if it doesn’t seem relevant, will be appreciated by the developer. You don’t need to write an essay, just a few clear lines explaining the key information of:

* what you expected to happen
* what actually happened and,
* what you did/were doing when it happened.

This will be enough to isolate all but the most complicated of issues, and once an issue can be reproduced it is well on its way to being fixed.

Quality control

2008-11-07T02:51:00.000-08:00

Quality control describes the directed use of testing to measure the achievement of a specified standard. Quality control is a formal (as in structured, not as in wearing-a-tuxedo fancy) use of testing. Quality control is a superset of testing, although it often used synonymously with testing. Roughly, you test to see if something is broken, and with quality control you set limits that say, in effect, if this particular stuff is broken then whatever you're testing fails.

Yet another way of looking at the difference between testing and quality control is to consider the difference between a test as an event and a test as a part of a system. For example, let's say our test is the measurement of your ability to assemble a jigsaw puzzle in one hour. We test you today, and you complete the puzzle in fifty-eight minutes, so you pass. This seems pretty trivial, but say that there is some need in your life requiring you to solve puzzles quickly: we tested you once, but we must verify that you can meet this weird requirement continually over time. The solution is to test you at regular intervals, which will allow us to see if you can still be successful when under stress, when you haven't slept, when your workload is high -- our quality control approach to this issue says "you must finish puzzle in one hour or less" and "we will test this requirement periododically over time". And as a side effect of this quality control testing, you might find that you are more likely to improve in your puzzle solving skills because of the repeated practice; this is the beginning of a shift to quality

Suggestions to Improve Testing.

2008-10-23T02:56:00.001-07:00

1) List down the modules to be tested.This include functional and non-functional testing

2) List down the functionality that you will not test either because of incompleteness of module, data dependency or hardware limitations etc.

3) Pl. get formal approval from your manager for the activities/schedule that you are planning.This will endorse your deliverables and validity.

4) Make sure that you have the defect tracking system in place with severity norms and owners defined.The defect logging format should be approved by your manager.

5) Make sure that you have set of data that will help you in testing various test scenarios.If data is not available, pl. check how you can generate the same.

6) Make sure that you take daily back-up of your test logs and discuss progress and issues with your management team every day.

7) Keep talking with developers. That will help you to increase product understanding.

8) Release notes from Developer team to Testing team will be entry criteria to start testing activity. The release notes will include information like the defects fixed in the build, any work around ,features not addressed.

9) Multiple iterations of testing will bring up new defects. Periodically update the test cases checking the possibility of new scenerios.Once the defect probability starts reducing in further testing iterations, this may be the indication of product getting stable. Introducing multiple testing heads will also help to filter new defects.

10)Try to set the processes with help of your manager. Eventually that will be helpful in long run.

Cross Browser Testing

2008-10-23T02:54:00.000-07:00

It is critical to test across many browser and operating system combinations because the page can look different in each scenario. Another concern is the screen resolution and color depth. A page might look good at a resolution of 600 x 800, but parts of it might get cut off at 640 x 480. Different color depths should be used on the test machines also. Colors might vary unpredictably if a browser-safe palette is not used. As a QA, we should look for:

Color of links
Broken images
Low color contrast
Spacing in tables
Text wrapping issues
Margins
Alignment, formatting, and size of text

Alignment of controls such as radio buttons and check boxes
Switch Javascript off
Switch cookies off
Switch plug-ins off
Switch images off
Printing - Do not forget to test printing of your Web pages by printing on a variety of popular printers. Printing can be unpredictable, particularly with frames. Keep an eye on what is printed, the readability of content, and the speed of the print job.

Be sure to use clean machines when you start testing and make sure no plug-ins are installed. If the plug-ins are already installed,you might miss a defect that has a dependency on the plug-in. There should always be some test cases that involve using a browser as it is first installed,with no extra components.

Netscape Navigator requires a plug-in.Netscape users must be aware that the plug-in is required and should be given instructions on how to install it. (Anyway Netscape is a dying browser)

View in text browser (Incidentally the Opera browser has a built-in text-browser emulator)

Missed a Bug

2008-10-23T02:52:00.000-07:00

Most of the testers/QAs are concern about external issues that they have missed. Carrier wise external issues are not good for testers profile inside their organization. More external issues may destroy the testers reputation within organization. On the other hand it is not fair to put 100% responsibility of external issues to testers. Why testers miss bugs?.

1.Lack of project management.
2.Not enough time to test.
3.Requirement change in last moment before releases.
4.Lack of domain knowledge.
5.Lack of QA/test resources.
6.Poor requirement specifications.
7.Limited time to acquire domain knowledge.
8.Selecting automation testing in the earlier stage of project. ( Where manually testing is required.)
9.High coupling with developers. ( testers who depends of developers to get domain/technical knowledge may influence by developers.
10.Not enough motivation towards testing. ( more open bugs, ignorance of QA team within organization, lost of existing functionalities due to new bugs, less salary.... )
11.Developers/QA/tester attitude towards bugs.

There are more to add for above reasons list, Above points can be categorized in to two.

1.Self factors
2.Organization factors.

Anyway most of the above factors caused due to lots of sub reasons. Head of the organization or middle layer management should take actions to treat the organization factors. Immediate supervisor should care about the testes self factors.

Test Director:

2008-09-05T04:35:00.001-07:00

TestDirector is a single, Web-based application for all essential aspects of test management — Requirements Management, Test Plan, Test Lab, and Defects Management. You can leverage these core modules either as a standalone solution or integrated within a global Quality Centre of Excellence environment. TestDirector supports high levels of communication and collaboration among IT teams. Whether you are coordinating the work of many disparate QA teams, or working with a large, distributed Centre of Excellence, this test management tool helps facilitate information access across geographical and organization boundaries.

Qulity center is the latest vesion of the testdirector. upto 8.0 it is called as testdirector after that it is called as QC. TestDirector for Quality Center has same 4 tabs - that did not change. Quality Center perfectly integrates with Functional Testing that now includes QTP + WinRunner (this

integration was possible before with TestDirector as well). As far as tabs... There is 1 additional tab that appeared in Quality Center that was not there before and it's called Dashboard (for project management).

Test Director:

2008-09-05T04:35:00.000-07:00

Qulity center is the latest vesion of the testdirector.upto 8.0 it is called as testdirector after that it is called as QC. TestDirector for Quality Center has same 4 tabs - that did not change. Quality Center perfectly integrates with Functional Testing that now includes QTP + WinRunner

As far as tabs... There is 1 additional tab that appeared in Quality Center that was not there before and it's called Dashboard (for project management).

WinRunner:

2008-09-05T04:34:00.001-07:00

It is Mercury Interactive Functional Automation Testing Tool.

How many types of Run Modes are available in WinRunner?

WinRunner provide three types of Run Modes.

Verify Mode

Debug Mode

Update Mode

What’s the Verify Mode?

In Verify Mode, WinRunner compare the current result of application to it’s expected result.

What’s the Debug Mode?

In Debug Mode, WinRunner track the defects in a test script.

What’s the Update Mode?

In Update Mode, WinRunner update the expected results of test script.

How many types of recording modes available in WinRunner?

WinRunner provides two types of Recording Mode?

Context Sensitive

Analog

What’s the Context Sensitive recording?

WinRunner captures and records the GUI objects, windows, keyboard inputs, and mouse click activities through Context Sensitive Recording.

What’s the Analog recording?

It captures and records the keyboard inputs, mouse click and mouse movement. It’s not captures the GUI objects and Windows.

Why do we test for performance?

2008-09-01T03:06:00.000-07:00

What a simple question! I even have a simple answer.

"To determine or estimate various performance characteristics under various conditions."

The problem is that answer is virtually useless unless we also know what performance characteristics are interesting to whom and for what purpose. Worse, more often than not, the folks who ask us to do the performance testing fundamentally don't know what they want to know and don't know what we can reasonably provide. They also don't understand that how the results are going to be used significantly impacts which tests we run and how we design them.

In my experience, when I ask stakeholders what the goals of the performance testing effort are, I generally get one of three answers:

You're the performance tester, you tell me.
Tell me how many users/orders/customers we can handle in production.
Make sure it will be fast enough.

Needless to say, these answers aren't only as useless as my response about determining or estimating performance characteristics, but the second two are practically impossible, since we almost never have either the data or the equipment available to accomplish those missions reliably. Somewhere between "virtually useless" and "practically impossible" there must be some reasons for testing performance that are both useful and possible. If there weren't useful and possible reasons for testing performance, we wouldn't still be doing it. (I hope!)

As it turns out, the key to my coming up with a model to explore that middle ground was to stop thinking about performance testing as a "testing effort" and start thinking about it as a "business effort." Once I made that shift, I was quickly able to identify four groups of "for whom" with common "for what purposes." Since then, I've found that using this model to frame conversations about prioritizing performance testing objectives fundamentally changes the discussion and increases the value of the performance testing effort. It also reduces wasted effort from designing tests to collect one class of results only to find out that an entirely different test was needed to provide the class of results that stakeholders wanted, but were unable to articulate until after they were presented with the less valuable results

Whitebox Testing - Is it really white ?

2008-08-19T06:12:00.000-07:00

The popular myths around Blackbox & White Box Testing are by it’s name. It’s black since we can’t see it (don’t have access to the code) & it’s white since you have access to all the code. But then, With in the code there are many black boxes inside and it’s tough to have access to that code base.
We don’t have access to code of a language API. Most of the applications have been built on top of a API & assume that the API works fine
Most of the application do integrate some third party tools over it’s API. We don’t have access to that code base.
We don’t have access to the code of Compiler
We don’t have access to code of rum time engine that executes our application code
We don’t have access to the code of Operating System Services on top of which the application runs

The list goes on and there are many black boxes in side our code too. We are just testing the code written for the application and it’s better to call it as Code Based Testing rather than Whitebox Testing

– Happy Testing..

How to get job in Software Testing quickly?

2008-08-19T05:42:00.000-07:00

I came across this article while surfing the net…..so I thot I shud share it with u all…. In recent days this is the most asked question to me by readers. How to get software testing job? How to come in software testing field? or Can I get job in testing?
All these questions are similar and I want to give also similar answer for this. I have written post on choosing software testing as your career where you can analyze your abilities and know which are the most important skills required for software testing.
I will continue mentioning that “know your interest before going into any career field”. Just going to software testing career or any other hot career is wrong and which may result into loss of your job interest as well as your job.
Now you know your abilities, skills, interests right? and you have made decision to go for software testing career as it is your favorite career and you suit most in this career. So here is a guideline for how to get a good job in software testing field.
If you are a fresher and just passed out from your college or passing out in coming months then you need to prepare well for some software testing methodologies. Prepare all the manual testing concepts. If possible have some hands-on on some automation and bug tracking tools like winrunner and test director. It is always a good idea to join any software testing institute or class which will provide you a good start and direction of preparation. You can join any 4 months duration software testing course or can do diploma in software testing which is typically of 6 months to 1 year. Keep the preparation going on in your course duration. This will help you to start giving interviews right after getting over your course.
If you have some sort of previous IT experience and want to switch to software testing then it’s somewhat simple for you. Show your previous IT experience in your resume while applying for software testing jobs. If possible do some crash course to get idea of software testing concepts like I mentioned for freshers above. Keep in mind that you have some kind of IT experience so be prepared for some tough interview questions here.
As companies always prefer some kind of relevant experience for any software job, its better if you have relevant experience in software testing and QA. It may be any kind of software testing tools hands-on or some testing course from reputed institutes.
Please always keep in mind- Do not add fake experience of any kind. This can ruin your career forever. Wait and try for some more days to get the job by your abilities instead of getting into trap of fake experience.
Last important words, Software testing is not ‘anyone can do career!’ Remove this attitude from your mind if someone has told such kind of foolish thing to you. Testing requires in depth knowledge of SDLC, out of box thinking, analytical skill and some programming language skill apart from software testing basics.

So best luck and start preparation for your rocking career!

What Do You Mean By SRS

2008-08-19T00:52:00.000-07:00

a set of precisely stated properties or constraints which a software system must satisfy.

a software requirements document establishes boundaries on the solution space of the problem of developing a useful software system.

A software requirements document allows a design to be validated - if the constraints and properties specified in the document are satisfied by the software design, then that design is an acceptable solution to the problem.

Six requirements which a software requirements document should satisfy

it should specify only external system behaviour,

it should specify constraints on the implementation,

it should be easy to change,

it should serve as a reference tool for system maintainers,

it should record forethought about the life cycle of the system, and

it should characterize acceptable responses to undesired events

Characteristics of a Software Requirements Specification

A good SRS is

unambiguous,

complete,

verifiable,

consistent,

modifiable,

traceable, and

usable during the operation and maintenance phase.

Unambiguous

Every requirement has only one interpretation.

Each characteristic of the final product is described using a single unique term.

A glossary should be used when a term used in a particular context could have multiple meanings.

Complete

A complete SRS must possess the following qualities:

inclusion of all significant requirements,

definition of the responses of the software to all realizable classes of input,

conformity to any standard that applies to it,

full labelling and referencing of all tables and diagrams and the definition of all terms.

Verifiable

Every requirement must be verifiable.

There must exist some finite cost-effective process with which a person or machine can check that the software meets the requirement.

Consistent

No set of individual requirements described in the SRS can be in conflict.

Types of likely conflicts:

Two or more requirements describe the same real world object in different terms.
The specified characteristics of real world objects might conflict.
There may be a logical or temporal conflict between two specified actions.

Modifiable

The structure and style of the SRS are such that any necessary changes to the requirements can be made easily, completely and consistently.

Requirements:

a coherent and easy-to-use organization (including a table of contents, index and cross-referencing),
not be redundant - this can lead to errors.

Traceable

The origin of each requirement must be clear.

The SRS should facilitate the referencing of each requirement in future development or enhancement documentation.

Types:

Backward traceability
- Each requirement must explicitly reference its source in previous documents.
Forward traceability
- Each requirement must have an unique name or reference number.

Usable during the operation and maintenance phase

The SRS must address the needs of the operation and maintenance phase, including the eventual replacement of the software.

Feasibility and Requirement Analysis

2008-08-13T02:32:00.000-07:00

The purpose of the feasibility study is to determine that whether the project can go ahead or not. If the project can go ahead the feasibility study produce the project plan and budget for the future stages of Development. The main goal of this phase is to identify the exact requirement for the System. This study sometimes performed by the market organizations, developer and Client. If the system already exist the goal of this phase is to identify the part of the System which have to be automated. Developing a system which does not exist is more Difficult task because it requires more creative thinking. Once the requirement’s are Identified, they are recorded in the document called SRS. For this pupose some specific language has been selected. The document at the end of this phase is known as SRS.

Amit Ahuja

Difference between Testing Types and Testing Techniques

2008-08-12T06:08:00.000-07:00

Testing types deal with what aspect of the computer software would be tested, while testing techniques deal with how a specific part of the software would be tested.

That is, testing types mean whether we are testing the function or the structure of the software. In other words, we may test each function of the software to see if it is operational or we may test the internal components of the software to check if its internal workings are according to specification.

On the other hand, ‘Testing technique’ means what methods or ways would be applied or calculations would be done to test a particular feature of a software (Sometimes we test the interfaces, sometimes we test the segments, sometimes loops etc.)

Amit Ahuja

Inspection

2008-08-12T06:07:00.000-07:00

A static analysis technique that relies on visual examination of development products to detect errors, violations of development standards, and other problems. Types include code inspection; design inspection, Architectural inspections, Test ware inspections etc.
The participants in Inspections assume one or more of the following roles:
a) Inspection leader
b) Recorder
c) Reader
d) Author
e) Inspector

All participants in the review are inspectors. The author shall not act as inspection leader and should not act as reader or recorder. Other roles may be shared among the team members. Individual participants may act in more than one role.
Individuals holding management positions over any member of the inspection team shall not participate in the inspection.

Input to the inspection shall include the following:
a) A statement of objectives for the inspection
b) The software product to be inspected
c) Documented inspection procedure
d) Inspection reporting forms
e) Current anomalies or issues list
Input to the inspection may also include the following:
f) Inspection checklists
g) Any regulations, standards, guidelines, plans, and procedures against which the software product is to be inspected
h) Hardware product specifications
i) Hardware performance data
j) Anomaly categories
The individuals may make additional reference material available responsible for the software product when requested by the inspection leader.
The purpose of the exit criteria is to bring an unambiguous closure to the inspection meeting. The exit decision shall determine if the software product meets the inspection exit criteria and shall prescribe any appropriate rework and verification. Specifically, the inspection team shall identify the software product disposition as one of the following:
a) Accept with no or minor rework. The software product is accepted as is or with only minor rework. (For example, that would require no further verification).
b) Accept with rework verification. The software product is to be accepted after the inspection leader or
a designated member of the inspection team (other than the author) verifies rework.
c) Re-inspect. Schedule a re-inspection to verify rework. At a minimum, a re-inspection shall examine the software product areas changed to resolve anomalies identified in the last inspection, as well as side effects of those changes.

Amit Ahuja

Walkthrough

2008-08-12T06:06:00.000-07:00

A static analysis technique in which a designer or programmer leads members of the development team and other interested parties through a segment of documentation or code, and the participants ask questions and make comments about possible errors, violation of development standards, and other problems.
The objectives of Walkthrough can be summarized as follows:
Detect errors early.
· Ensure (re)established standards are followed:
· Train and exchange technical information among project teams which participate in the walkthrough.
· Increase the quality of the project, thereby improving morale of the team members.
The participants in Walkthroughs assume one or more of the following roles:
a) Walk-through leader
b) Recorder
c) Author
d) Team member
To consider a review as a systematic walk-through, a team of at least two members shall be assembled. Roles may be shared among the team members. The walk-through leader or the author may serve as the recorder. The walk-through leader may be the author.
Individuals holding management positions over any member of the walk-through team shall not participate in the walk-through.

Input to the walk-through shall include the following:
a) A statement of objectives for the walk-through
b) The software product being examined
c) Standards that are in effect for the acquisition, supply, development, operation, and/or maintenance of the software product
Input to the walk-through may also include the following:
d) Any regulations, standards, guidelines, plans, and procedures against which the software product is to be inspected
e) Anomaly categories

The walk-through shall be considered complete when
a) The entire software product has been examined
b) Recommendations and required actions have been recorded
c) The walk-through output has been completed

Amit Ahuja

Objective Testing

2008-08-12T06:01:00.002-07:00

Objective testing is mainly used in systems where the data can be recorded while the simulation is running. This testing technique relies on the application of statistical and automated methods to the data collected.
Statistical methods are used to provide an insight into the accuracy of the simulation. These methods include hypothesis testing, data plots, principle component analysis and cluster analysis.
Automated testing requires a knowledge base of valid outcomes for various runs of simulation. This knowledge base is created by domain experts of the simulation system being tested. The data collected in various test runs is compared against this knowledge base to automatically validate the system under test. An advantage of this kind of testing is that the system can continually be regression tested as it is being developed.
Statistical Methods
Statistical methods are used to provide an insight into the accuracy of the simulation. These methods include hypothesis testing, data plots, principle component analysis and cluster analysis.
Automated Testing
Automated testing requires a knowledge base of valid outcomes for various runs of simulation. This knowledge base is created by domain experts of the simulation system being tested. The data collected in various test runs is compared against this knowledge base to automatically validate the system under test. An advantage of this kind of testing is that the system can continually be regression tested as it is being developed.

Amit Ahuja

Subjective Testing

2008-08-12T06:01:00.001-07:00

Subjective testing mainly depends on an expert's opinion. An expert is a person who is proficient and experienced in the system under test. Conducting the test involves test runs of the simulation by the expert and then the expert evaluates and validates the results based on some criteria.
One advantage of this approach over objective testing is that it can test those conditions which cannot be tested objectively. For example, an expert can determine whether the joystick handling of the flight feels "right".

One disadvantage is that the evaluation of the system is based on the "expert's" opinion, which may differ from expert to expert. Also, if the system is very large then it is bound to have many experts. Each expert may view it differently and can give conflicting opinions. This makes it difficult to determine the validity of the system. Despite all these disadvantages, subjective testing is necessary for testing systems with human interaction.

Amit Ahuja

Black Box and Gray Box Testing

2008-08-11T00:13:00.001-07:00

Black box testing is based on the Software's specifications or requirements, without reference to its internal workings. Gray box testing combines white box techniques with black box input testing [Hoglund 04]. This method of testing explores paths that are directly accessible from user inputs or external interfaces to the software. In a typical case, white box analysis is used to find vulnerable areas, and black box testing is then used to develop working attacks against these areas. The use of gray box techniques combines both white box and black box testing methods in a powerful way.

Amit Ahuja

White Box Testing

2008-08-11T00:10:00.000-07:00

The purpose of any Security testing method is to ensure the robustness of a system in the face of malicious attacks or regular Software failures. White box testing is performed based on the knowledge of how the system is implemented. White box testing includes analyzing data flow, control flow, information flow, coding practices, and exception and error handling within the system, to test the intended and unintended software behavior. White box testing can be performed to validate whether code implementation follows intended design, to validate implemented security functionality, and to uncover exploitable vulnerabilities.

White box testing requires access to the source code. Though white box testing can be performed any time in the life cycle after the code is developed, it is a good practice to perform white box testing during the unit testing phase.

White box testing requires knowing what makes software secure or insecure, how to think like an attacker, and how to use different testing tools and techniques. The first step in white box testing is to comprehend and analyze source code, so knowing what makes software secure is a fundamental requirement. Second, to create tests that exploit software, a tester must think like an attacker. Third, to perform testing effectively, testers need to know the different tools and techniques available for white box testing. The three requirements do not work in isolation, but together.

Amit Ahuja

Kinds of testing that should be considered for Websites.

2008-08-09T03:44:00.001-07:00

Black box testing - testing not based on any knowledge of the internal design or code. Tests are based on requirements and functionality.

Incremental integration testing - continuous testing of website as new functionality is added; requires that various aspects of an web applications functionality be independent enough to work separately before all parts of the program are completed.

Integration testing - testing that is done on combined parts of an application to determine if they function together correctly. The ‘parts’ can be code modules, individual applications, pages in a website etc.

Functional testing - black box type testing geared to functional requirements of an application; testers should do this type of testing. This doesn’t mean that the programmers shouldn’t check that their code works before releasing it (which of course applies to any stage of testing.)

System testing - black box type testing that is based on overall requirements specifications; covers all combined parts of a web application.

End-to-end testing - similar to system testing; the ‘macro’ end of the test scale; involves testing of a complete application environment in a situation that mimics real-world use, such as interacting with a database, using network communications, or interacting with other hardware, applications, or systems if appropriate.

Sanity testing or smoke testing - typically an initial testing effort to determine if a new web application is performing well enough to accept it for a major testing effort. For example, if there are lots of missing links, missing images, missing validations, or corrupting databases, the website may not be in a ’sane’ enough condition to warrant further testing in its current state.

Regression testing - re-testing after fixes or modifications of the website or its pages. It can be difficult to determine how much re-testing is needed, especially near the end of the development cycle. Automated testing tools can be especially useful for this type of testing.

Acceptance testing - final testing based on specifications of the end-user or customer, or based on use by end-users/customers over some limited period of time.

Load testing - testing of a web site under a range of loads to determine at what point the system’s response time degrades or fails.

Stress testing - term often used interchangeably with ‘load’ and ‘performance’ testing. Also used to describe such tests as system functional testing while under unusually heavy loads, heavy repetition of certain actions or inputs, input of large numerical values, large complex queries to a database system, etc.

Performance testing - term often used interchangeably with ’stress’ and ‘load’ testing. Ideally ‘performance’ testing (and any other ‘type’ of testing) is defined in requirements documentation or QA or Test Plans.

Usability testing - testing that is done for ‘user-friendliness’. Clearly this is subjective, and will depend on the targeted end-user or customer. User interviews, surveys, video recording of user sessions, and other techniques can be used. Programmers and testers are usually not appropriate as usability testers.

Security testing - testing how well the system protects against unauthorized internal or external access, willful damage, etc; may require sophisticated testing techniques.

Compatibility testing - testing how well website performs in a particular hardware/software/operating system//browser/network etc. environment.

Exploratory testing - often taken to mean a creative, informal software test that is not based on formal test plans or test cases; testers may be learning the website as they test it.

Ad-hoc testing - similar to exploratory testing, but often taken to mean that the testers have significant understanding of the website before testing it.

User acceptance testing - determining if website is satisfactory to an end-user or customer.

Alpha testing - testing of a web application when development is nearing completion; minor design changes may still be made as a result of such testing. Typically done by end-users or others, not by programmers or testers.

Beta testing - testing when development and testing are essentially completed and final bugs and problems need to be found before final release. Typically done by end-users or others, not by programmers or testers.

Amit Ahuja

Testing Policy VS. Quality Policy

2008-08-09T03:41:00.000-07:00

Testing Policy:

A Testing policy is management‘s definition of testing a department. A testing policy involves the following four criteria

Definition of testing: a clear, brief and unambiguous definition of testing

Standards: The standards against which testing will be measured

Testing system: The method through which testing will be achieved and enforced

Evaluation: How testing team will measure and evaluate testing.

Quality Policy:

A quality policy is again a management definition of providing customer satisfaction for the first time and every time. Understanding the definition of excellence and quality is important because it is starting point of any management team contemplating the implementation of a quality policy.

Amit Ahuja

What if Automation finds bugs ....? Good thing or bad thing?

2008-08-09T03:36:00.000-07:00

Now, let us track that trail of what happens a bug is discovered -
In automation - situation could be bit tricky especially when the automation tests, logs are bigger. An error/bug reported by an automation bug needs to be checked to see if it is a bug in automation code or a bug in application or bug in data setup or some timing or synchronization related problem (in GUI automation scenario). Let us say you have 5-7 pages of log file - you will have to scan/read through the log file an locate the bug. You might have to do execute failed automated test manually (and corresponding data setup etc).
In manual testing, human tester can easily trace and follow the bug trail and document the bug. At a high level, bug investigation and isolation tasks tend to become relatively low.

Hence, when automation discovers a bug - things get really problematic.

If one were to cut down cycle time by automation or otherwise, they HAVE to make sure either "no bugs are discovered" or "any discovered bugs are IGNORED" or "bugs that are discovered, if fixed, not tested again and other regression testing is done ....

Can automation control or influence any of above events - prevents bugs being discovered or igonore the bugs if accidently discovered or mandate that bugs fixes will not be subsequently tested?

For the sake of argument, let us suppose that both human test cycle and Automation find same number of bugs ... and take out "bugs" portion of test cycle, how automation can save test cycle time? On what parameters this cycle time reduction by Automation depends ?

Type of test - nature of interactions between "test execution agent" (human or an automated script) and nature of verifications (during and post execution).

GUI Forms with lots of data input fields - can result in quick form fill tests when automated (zero think time).
Tests that require longer processing time can not gain from automation as automation can not speed up the processing time.
Tests that require visual inspection - window titles, error messages, color, tool tip and other GUI centric elements - are better tested manunally as programmatic tests would mean lots of investment. Human testes are quicker and cheaper in such cases.
Result verification that requires detailed analysis, text file processing, database check etc are good candidates for gaining cycle time.

Thus, there are parameters that are beyond the reach of automation ... hence the notion of cycle time reduction has to be really, really taken with "caution".

Amit Ahuja

Are all best practices "worthless"? Testing Best Practices

2008-08-09T03:25:00.000-07:00

Other day I was quoting following from Jerry’s new book on testing to one of my colleague who is a “best practice” proponent.

…..The risks in these two situations are vastly different, so do you think I recommended the same testing process I used for finding a personal web-writing application? Do you imagine I recommended that my client install random freeware pacemakers into live patients until he found something he liked, or didn't dislike? Why not?

I took above sentences as reference and told him.. “Can you use software testing strategy that one uses for web application writing to that of an embedded software in a heart pace maker? Hence best practices are such a junk thing ...”

To that he was silent for a while answered --- I agree with your point that test strategy or approach used for web application cannot be applied for embedded software in pace maker … How about picking the practice from a same field/domain – will that not save the time, energy and effort for my client ? Let us say I develop a list of practices for a given field (embedded software used in human bodies) and keep “selling” them as best practices (jump start kit) for those clients who deal with such software? What is your opinion? Would you still say … best practices (in a context) are junk?

I did not have a good answer for him …. Then we discussed about “universal best practices” (I am not sure if such phrase exists as all best practices are universal in nature by default and context less??) such as “walking is good for health”,”Test considering end user scenarios”, “Do unit testing” “Do code review”, “Aspirin is good for heart”, “Drunken driving leads to accidents”, “Do meditation to calm your mind” etc. I told him about at least 3 contexts for each of these best practices where following best practices can lead to harmful effects.

After listening to me … he said … Shrini … you appear to be "making up" all these contexts to prove your point …I want you to answer my question – Are all generic best practices recommendations are worthless or fake? When customers want something readymade that will help them to jumpstart the work, they would like to see if I, as a consultant, can bring some “best practices” from my previous similar experiments. Is that expectation unreasonable?

I am thinking ... I don’t have a good answer for him … do you? I hope Jerry would have some answer …

Are there any "universal best practices" or by default all best practices are universal and context free? Will a best practice cease to remain as bet practice once it comes with a context?

[update] Quoting from Jerry's book again - "As humans - we are not perfect thinkers, we are affected by emotions and we are not clones. We are Imperfect, irrational, value driven,diverse humans - hence we test software and test our
testing AND hence test "best practices" that sales and marketing folks associate with software testing.

Amit Ahuja

Software - A game of questions and answers

2008-08-09T03:23:00.000-07:00

The most serious mistakes are not being made as a result of wrong answers. The truly dangerous thing is asking the wrong question."
— Peter Drucker

"Testing is a questioning process in order to evaluate software" - James Bach

"Computers are useless. They can only give you answers." - Pablo Picasso

"One who asks a question is a fool for five minutes; one who does not ask a question remains a fool forever." - A Chinese proverb

Other day I was discussing with one of my colleague … somehow our discussion went about interpreting in simple terms the whole “game” of software development and testing. Here is what and how ended up in agreeing on the “simple” model to describe software and software lifecycle …(not SDLC but SLC)

Software development is about coming up with answers (and demonstrating those answers with an example) to the questions raised by testers, end users and other stakeholders.

Software testers ask questions about claims and capabilities of what software is supposed to do, take the questions to developers and project mangers and ask for answers.

Project manager or project sponsors scan these questions and pick up those that they think are worth “answering”, prioritize them and pass on the developers for providing answers and ways to demonstrate the answers. Before releasing the software to testers, developer do some question-answer session with buddy developers and leads (peer testing, unit testing and code reviews)

Developers then get on mission to analyze questions and develop/construct answers in the form of capabilities in the software and “release” to testers to check to see the answers are “satisfactory”. When developers do not get answers or feel that it takes relatively long time to find the answers – they turn to project manager with their analysis as why answer can not be made available immediately. Project manager then takes the decision of “deferring” those “unanswered” question to be taken up in future releases.
At times Testers, act on behalf of end users and other

Testers verify those answers and check to see they are OK … some times there will be follow-up questions or new questions (regression bugs/issues) and they are routed to developers via the project manager. This cycle repeats until there are new questions to be answered by the developers.

So … as long as there are questions to be answered about software … there will be the need of developers (who will provide answers) and there will be need of Project managers (to prioritize and check which questions need to be answered) and hence a software development project …

Guess what … it is software testers who drive the whole thing by asking relevant and important questions about software – about it’s claims and capabilities …

So … as important trait of a tester is to practice asking “good” questions …

Amit Ahuja