Selenium Vs QTP

1. Actual end user simulation, Is the test conducted using this tool equivalent to an end user action?
Selenium performs actions in the background on the browser. It modifies the DOM structure of the HTML page in order to perform actions on the page. To be more precise it executes javascript on UI objects within the webpage to perform actions like click, type, select etc. This is the reason why you can execute tests with the browser minimized.
QTP claims to perform end user simulation, in other words executing QTP scripts is equivalent to a person performing those steps manually on the application.
2. Support for most UI Components
Selenium Certain events, methods and Object properties are not supported by selenium. But broadly most UI components are supported.
QTP requires extra add-ons (plug-in, not free) to work with .Net components.
3. UI-Object management & storage
QTP comes built-in with object repository. Object repository management is quite easy in QTP. Selenium has no such built in feature but objects can be managed using UI-Element user extension. Other custom solutions like properties files can also be used in selenium. But for all such solutions the map file in selenium has to be hand-coded i.e unlike QTP the objects won't be recorded/added automatically .
4. Support for Dialog Boxes
QTP supports all kinds of IE dialog boxes. There is partial support for dialog boxes in Selenium. Some actions like retrieving the title of he dialog box can't be performed in selenium.
5. Support for web browsers
QTP supports IE & Firefox. Selenium supports IE, Firefox, Safari and Opera and a few more browsers. But either tools are far far away from full cross-browser support. Don't expect scripts created using browser to run flawlessly in another browser.
6. Object Recognition Parameters, Recognition on the basis of WYSWYG (what you see is what you get).
Selenium recognizes objects on the basis of the DOM structure of the HTML Page. The UI objects in selenium have vague descriptions and don't comply with WYSWYG policy.
QTP recognises and names objects based on properties which are more visible and obvious and are hence the objects have user friendly names.
7. Object Oriented Language support & Scalability (as in Integration with External tools utilities and libraries). Selenium Supports JAVA, dot net and many other industry standard programming languages. QTP supports vb script only.
8. Integration with test management tool.
QTP integrates seamlessly with QC and TD. Test management and mapping the manual testing process with automation becomes a lot easier with this integration. I have not yet heard of any test management tool that can integrate seamlessly with Selenium. keep an eye out for Bromide though.
9. Types of application supported
QTP wins this one hands down. This is one of the main reason why selenium can't even be considered in many cases. Imagine real-time applications like trading terminals, risk management applications built in TCL/TK and PowerBuilder. QTP supports most of these interfaces.
Selenium on the other hand can work only on applications that open up inside a browser. But aren't most applications moving to the browser based platform? Smile
10. Support for operating system/platforms
Selenium Supports JAVA and hence can be used in Windows PC or MAC or UNIX. Using selenium you can test your web application in all the above platforms. QTP supports Windows only.
11. Ease of creation of Scripts
Selenium IDE Recorder is not as powerful as QTP but is good for a free tool, many actions are not recorded by the IDE and have to be manually entered.
12. Technical Support
QTP offers technical support by phone and mail, HP also have a web-forum. QTP user community is vast and questions posted on online forums get answered quickly. Selenium being an open source tool has no official tech support, the user community is small, less-active and questions on forums seldom get answered. But the community is growing day by day as the tool gains acceptance.
13. Cost
QTP - Very Costly, in some thousand dollars per seat license. Many people want to switch to selenium because it's free. But cost isn't really a factor when your client's a investment bank Smile
14. Test Development Environment
When you are working on selenium you have the option of using wide range of IDEs like Eclipse, Netbeans, Visual Studio etc depending on your choice of development language. If you are a developer then you must have developed a taste for rich IDEs and switching to a environment given by a test tool may be hard for you.
QTP tests can only be developed in QTP.
15. Integration with development process
Tests developed using selenium can be easily part of the development project. Using tools like cruise control Continuous Integration is easier with Selenium. But don't get too caught up with this feature, it's really not that important to be integrated with the development process but it's nice to have.
16. Future in terms of usability and acceptance
The future bodes well for selenium because it's free, supports all programming languages/platforms and is immensely scalable and expandable due to it being free and open source. Many pundits have predicted that it will completely conquer the web testing market in the next 5 years. It being free makes a huge difference especially when times are hard like what we have now.
QTP on the other hand is the current market leader and I think that it will have it's presence for long due to it's user friendliness and support for interfaces other than web.

Software testing life cycle

Software testing life cycle identifies what test activities to carry out and when (what is the best time) to accomplish those test activities. Even though testing differs between organizations, there is a testing life cycle.

Software Testing Life Cycle consists of six (generic) phases:

* Test Planning,
* Test Analysis,
* Test Design,
* Construction and verification,
* Testing Cycles,
* Final Testing and Implementation and
* Post Implementation.


Software testing has its own life cycle that intersects with every stage of the SDLC. The basic requirements in software testing life cycle is to control/deal with software testing – Manual, Automated and Performance.
Test Planning

This is the phase where Project Manager has to decide what things need to be tested, do I have the appropriate budget etc. Naturally proper planning at this stage would greatly reduce the risk of low quality software. This planning will be an ongoing process with no end point.

Activities at this stage would include preparation of high level test plan-(according to IEEE test plan template The Software Test Plan (STP) is designed to prescribe the scope, approach, resources, and schedule of all testing activities. The plan must identify the items to be tested, the features to be tested, the types of testing to be performed, the personnel responsible for testing, the resources and schedule required to complete testing, and the risks associated with the plan.). Almost all of the activities done during this stage are included in this software test plan and revolve around a test plan.
Test Analysis

Once test plan is made and decided upon, next step is to delve little more into the project and decide what types of testing should be carried out at different stages of SDLC, do we need or plan to automate, if yes then when the appropriate time to automate is, what type of specific documentation I need for testing.

Proper and regular meetings should be held between testing teams, project managers, development teams, Business Analysts to check the progress of things which will give a fair idea of the movement of the project and ensure the completeness of the test plan created in the planning phase, which will further help in enhancing the right testing strategy created earlier. We will start creating test case formats and test cases itself. In this stage we need to develop Functional validation matrix based on Business Requirements to ensure that all system requirements are covered by one or more test cases, identify which test cases to automate, begin review of documentation, i.e. Functional Design, Business Requirements, Product Specifications, Product Externals etc. We also have to define areas for Stress and Performance testing.
Test Design

Test plans and cases which were developed in the analysis phase are revised. Functional validation matrix is also revised and finalized. In this stage risk assessment criteria is developed. If you have thought of automation then you have to select which test cases to automate and begin writing scripts for them. Test data is prepared. Standards for unit testing and pass / fail criteria are defined here. Schedule for testing is revised (if necessary) & finalized and test environment is prepared.
Construction and verification

In this phase we have to complete all the test plans, test cases, complete the scripting of the automated test cases, Stress and Performance testing plans needs to be completed. We have to support the development team in their unit testing phase. And obviously bug reporting would be done as when the bugs are found. Integration tests are performed and errors (if any) are reported.
Testing Cycles

In this phase we have to complete testing cycles until test cases are executed without errors or a predefined condition is reached. Run test cases --> Report Bugs --> revise test cases (if needed) --> add new test cases (if needed) --> bug fixing --> retesting (test cycle 2, test cycle 3….).
Final Testing and Implementation

In this we have to execute remaining stress and performance test cases, documentation for testing is completed / updated, provide and complete different matrices for testing. Acceptance, load and recovery testing will also be conducted and the application needs to be verified under production conditions.
Post Implementation
In this phase, the testing process is evaluated and lessons learnt from that testing process are documented. Line of attack to prevent similar problems in future projects is identified. Create plans to improve the processes. The recording of new errors and enhancements is an ongoing process. Cleaning up of test environment is done and test machines are restored to base lines in this stage.

MUTATION TESTING

A kind of testing in which, the application is tested for the code that was modified after fixing a particular bug/defect. It also helps in finding out which code and which strategy of coding can help in developing the functionality effectively.

Proof of Concept & Interoperability Testing

Interoperability testing has become a requirement for companies that deploy multi-vendor networks. To satisfy this requirement, network and storage providers and managers have three options.

1. Set up an interoperability lab, an expensive and time-consuming project.

2. Use a third-party interoperability lab, such as ISOCORE or the University of New Hampshire.

3. Create a proof-of-concept lab, such as the labs at Cisco or Spirent Communications.

These labs typically connect the devices with a copper or fiber-optic patch cable and run the tests. Such testing reflects a best-case scenario that is useful for base-line interoperability testing but doesn’t represent how the devices will interoperate in an actual network.

It is analogous to testing the auto-pilot system of an airplane to see if it could land the plane in ideal weather conditions. While the test proves the plane can land itself on a perfect day, it is not a predictor of how the system will behave in wide range of weather conditions under which a plane will have to operate.

Sometimes spools of fiber are used to create the delay found in wide area networks. While this is an improvement over patch cables, there are three major limitations to proof of concept and interoperability testing with spools of fiber:

1. Spools of fiber cannot provide dynamic tests. The tests must be manually stopped and restarted to change out the spool of fiber for one with a different length.

2. Spools of fiber are expensive and impractical. Imagine the cost associated with moving an 50,000 km spool of fiber to another lab.

3. Spools of fiber only provide delay. They do not address the various other impairments that exist in a network.

It is as if we improved our auto-pilot system testing to include fixed amounts of wind from a single direction. Factors such as fog, rain, snow and wind sheers are still ignored in the testing. Weather conditions are dynamic and multifaceted. Testing under a single condition is not a realistic test. As with weather, so it is with networks. Impairments in real networks do not limit themselves to a single issue.

www.qaamitahuja.blogspot.com

Iterative Model

An iterative lifecycle model does not attempt to start with a full specification of requirements. Instead, development begins by specifying and implementing just part of the software, which can then be reviewed in order to identify further requirements. This process is then repeated, producing a new version of the software for each cycle of the model. Consider an iterative lifecycle model which consists of repeating the following four phases in sequence:

A Requirements phase, in which the requirements for the software are gathered and analyzed. Iteration should eventually result in a requirements phase that produces a complete and final specification of requirements. - A Design phase, in which a software solution to meet the requirements is designed. This may be a new design, or an extension of an earlier design.

An Implementation and Test phase, when the software is coded, integrated and tested.

A Review phase, in which the software is evaluated, the current requirements are reviewed, and changes and additions to requirements proposed.

For each cycle of the model, a decision has to be made as to whether the software produced by the cycle will be discarded, or kept as a starting point for the next cycle (sometimes referred to as incremental prototyping). Eventually a point will be reached where the requirements are complete and the software can be delivered, or it becomes impossible to enhance the software as required, and a fresh start has to be made.

The iterative lifecycle model can be likened to producing software by successive approximation. Drawing an analogy with mathematical methods that use successive approximation to arrive at a final solution, the benefit of such methods depends on how rapidly they converge on a solution.

The key to successful use of an iterative software development lifecycle is rigorous validation of requirements, and verification (including testing) of each version of the software against those requirements within each cycle of the model. The first three phases of the example iterative model is in fact an abbreviated form of a sequential V or waterfall lifecycle model. Each cycle of the model produces software that requires testing at the unit level, for software integration, for system integration and for acceptance. As the software evolves through successive cycles, tests have to be repeated and extended to verify each version of the software.

Amit Ahuja

Test Cases

Designing Test Cases


A test case is a detailed procedure that fully tests a feature or an aspect of a feature. Whereas the test plan describes what to test, a test case describes how to perform a particular test. You need to develop a test case for each test listed in the test plan. Figure 2.10 illustrates the point at which test case design occurs in the lab development and testing process.


Test case includes:

* The purpose of the test.

* Special hardware requirements, such as a modem.

* Special software requirements, such as a tool.

* Specific setup or configuration requirements.

* A description of how to perform the test.

* The expected results or success criteria for the test.

Test cases should be written by a team member who understands the function or technology being tested, and each test case should be submitted for peer review.

Organizations take a variety of approaches to documenting test cases; these range from developing detailed, recipe-like steps to writing general descriptions. In detailed test cases, the steps describe exactly how to perform the test. In descriptive test cases, the tester decides at the time of the test how to perform the test and what data to use.

Most organizations prefer detailed test cases because determining pass or fail criteria is usually easier with this type of case. In addition, detailed test cases are reproducible and are easier to automate than descriptive test cases. This is particularly important if you plan to compare the results of tests over time, such as when you are optimizing configurations. Detailed test cases are more time-consuming to develop and maintain. On the other hand, test cases that are open to interpretation are not repeatable and can require debugging, consuming time that would be better spent on testing.

Test Case Design

Test Case ID:

It is unique number given to test case in order to be identified.

Test description:

The description if test case you are going to test.

Revision history:

Each test case has to have its revision history in order to know when and by whom it is created or modified.

Function to be tested:

The name of function to be tested.

Environment:

It tells in which environment you are testing.

Test Setup:

Anything you need to set up outside of your application for example printers, network and so on.

Test Execution:

It is detailed description of every step of execution.

Expected Results:

The description of what you expect the function to do.

Actual Results:

pass / failed

If pass - What actually happen when you run the test.

If failed - put in description of what you've observed.


Amit Ahuja

Black Box Testing

Testing not based on any knowledge of the internal design or code. Tests are based on requirements and functionality.


Amit Ahuja

Who should test? and what? – An Overview

Who should do the testing?

Software Testing is not the job of one person. It is teamwork. The size of the team depends upon the complexity and size of the software being tested. The software developer should have minimum or no role in the testing process as everyone knows that for a person who has developed the software, it is very difficult to pin point errors in his own creations.


Seven soft skills crucial for a good tester are – that He or She must:

1) Be Cautious
2) Be Curious
3) Have Patience
4) Have Perseverance
5) Be Analytical
6) Be Critical but with an attitude of not jumping to conclusions.
7) Be Good Communicator

Thus “More the soft skills a tester has, better tester He or She is”

Above all, a Most Successful Tester is the one:

1) Who is completely passionate.

2) Who is always eager to learn more & more.

3) Who never gives up.

Various personnel and their roles during development & testing are:

A) Customer: Provides funding, provides requirements, approves changes and some of the test results.

B) Project Manager: Plans and manages the project.

C) Software Developer: Designs the codes and builds the software.

D) Testing coordinator: Creates test plans and test specifications based on the requirements and functional and technical documents.

E) Testing person: Executes the tests and documents the results.


Role based demarcation exclusive to the field of Testing:

1) Junior Software Testers: Are the ones having good theoretical knowledge of testing & would have participated in good number of seminars on testing or would have passed some courses in testing. Such individuals are not expected to have experience on testing; however little experience can be desirable.

2) Software Testers: Are the ones having good ability to understand testing packs, do testing, and doing documentation / defect logging. Such individuals usually do repetitive work & are invariably involved in front-end testing. Usually at least 6 months of testing experience is preferred for this role.

3) Senior Software Testers: Are the ones having responsibility of doing all tasks usually performed by a Tester. In addition to this they are involved in back-end testing. The senior testers also do updating of test cases. Usually 1 to 2 years of experience of testing is preferred for this role.

4) Testing Analysts: Are the ones having expertise in extracting requirements out of documentation & doing thorough verification with business & ascertaining the correctness of all the information. The testing analysts also do writing of test cases, executing the tests & reporting the findings thereafter. Usually 2 to 3 years of experience of testing along with at least 1-year experience of analysis is preferred for this role.

5) Testing Managers: Are the ones having experience of doing all the tasks described above. Apart from this they possess the ability of managing the entire testing process, personnel as well as the testing environment. Usually 3 to 4 years of experience of testing along with at least 1-year of managerial experience & sound expertise of Project Management is ideal for this role.

6) Testing Consultants: Are the senior people having experience of doing all the tasks described above. Consultants are usually good communicators having proven ability of man management, with the help of which they are able to effectively handle the client & the senior executives of the organization. This includes doing systematic analysis of the current testing process of the client & ability to guide the client with their expert comments & recommendations to improve. Consultants have great expertise of manual testing in addition to sound knowledge of specialized fields like Automation, Usability & Security etc.

Generally persons with 5 years or more of Testing & Project Management experience in some senior capacity are appointed as testing consultants.

7) Test Automators: Are the ones having good skills of development plus sound experience of manual testing & have acquired specialization of Automated Testing; but as of date they would not prefer to get involved in manual testing. Automators have thorough knowledge & experience of using different Automation Tools. Usually 1 to 2 years of experience of test automation is preferred for this role.

Next question comes as to what should be tested?

According to Myer (The great Testing Guru) - for most of the programs, it is impractical to attempt to test the program with all possible inputs, due to a combinatorial explosion. For those inputs selected, a testing oracle is needed to determine the correctness of the output for particular test input.

Myer also says that, for most programs, it is impractical to attempt to test all execution paths through the product, due to combinatorial explosion. It is also not possible to develop an algorithm for generating test data for paths in an arbitrary product, due to the inability to determine path feasibility.

The point, which is being emphasized here, is that complete or exhaustive testing is just not possible. This is because exhaustive testing requires every statement in the program and every possible path combination to be executed at once.



Amit Ahuja

Free Download SQL Books

1) Teach Yourself SQL in 21 Days




Amit Ahuja

Waterfall Model

There are various software development approaches defined and designed which are used/employed during development process of software, these approaches are also referred as "Software Development Process Models".

Waterfall Model

Waterfall approach was first Process Model to be introduced and followed widely in Software Engineering to ensure success of the project. In "The Waterfall" approach, the whole process of software development is divided into separate process phases.

The phases in Waterfall model are: Requirement Specifications phase, Software Design, Implementation and Testing & Maintenance. All these phases are cascaded to each other so that second phase is started as and when defined set of goals are achieved for first phase and it is signed off, so the name "Waterfall Model". All the methods and processes undertaken in Waterfall Model are more visible.

The stages of "The Waterfall Model" are:

Requirement Analysis & Definition: All possible requirements of the system to be developed are captured in this phase. Requirements are set of functionalities and constraints that the end-user (who will be using the system) expects from the system. The requirements are gathered from the end-user by consultation, these requirements are analyzed for their validity and the possibility of incorporating the requirements in the system to be development is also studied. Finally, a Requirement Specification document is created which serves the purpose of guideline for the next phase of the model.

System & Software Design: Before a starting for actual coding, it is highly important to understand what we are going to create and what it should look like? The requirement specifications from first phase are studied in this phase and system design is prepared. System Design helps in specifying hardware and system requirements and also helps in defining overall system architecture. The system design specifications serve as input for the next phase of the model.

Implementation & Unit Testing: On receiving system design documents, the work is divided in modules/units and actual coding is started. The system is first developed in small programs called units, which are integrated in the next phase. Each unit is developed and tested for its functionality; this is referred to as Unit Testing. Unit testing mainly verifies if the modules/units meet their specifications.

Integration & System Testing: As specified above, the system is first divided in units which are developed and tested for their functionality. These units are integrated into a complete system during Integration phase and tested to check if all modules/units coordinate between each other and the system as a whole behaves as per the specifications. After successfully testing the software, it is delivered to the customer.

Operations & Maintenance: This phase of "The Waterfall Model" is virtually never ending phase (Very long). Generally, problems with the system developed (which are not found during the development life cycle) come up after its practical use starts, so the issues related to the system are solved after deployment of the system. Not all the problems come in picture directly but they arise time to time and needs to be solved; hence this process is referred as Maintenance.

Advantages and Disadvantages

Advantages

The advantage of waterfall development is that it allows for departmentalization and managerial control. A schedule can be set with deadlines for each stage of development and a product can proceed through the development process like a car in a carwash, and theoretically, be delivered on time. Development moves from concept, through design, implementation, testing, installation, troubleshooting, and ends up at operation and maintenance. Each phase of development proceeds in strict order, without any overlapping or iterative steps.

Disadvantages

The disadvantage of waterfall development is that it does not allow for much reflection or revision. Once an application is in the testing stage, it is very difficult to go back and change something that was not well-thought out in the concept stage. Alternatives to the waterfall model include joint application development (JAD), rapid application development (RAD), synch and stabilize, build and fix, and the spiral model.

Common Errors in Requirements Analysis

In the traditional waterfall model of software development, the first phase of requirements analysis is also the most important one. This is the phase which involves gathering information about the customer's needs and defining, in the clearest possible terms, the problem that the product is expected to solve.

This analysis includes understanding the customer's business context and constraints, the functions the product must perform, the performance levels it must adhere to, and the external systems it must be compatible with. Techniques used to obtain this understanding include customer interviews, use cases, and "shopping lists" of software features. The results of the analysis are typically captured in a formal requirements specification, which serves as input to the next step.

Well, at least that's the way it's supposed to work theoretically. In reality, there are a number of problems with this theoretical model, and these can cause delays and knock-on errors in the rest of the process. This article discusses some of the more common problems that project managers experience during this phase, and suggests possible solutions.

Problem 1: Customers don't (really) know what they want

Possibly the most common problem in the requirements analysis phase is that customers have only a vague idea of what they need, and it's up to you to ask the right questions and perform the analysis necessary to turn this amorphous vision into a formally-documented software requirements specification that can, in turn, be used as the basis for both a project plan and an engineering architecture.

To solve this problem, you should:

* Ensure that you spend sufficient time at the start of the project on understanding the objectives, deliverables and scope of the project.
* Make visible any assumptions that the customer is using, and critically evaluate both the likely end-user benefits and risks of the project.
* Attempt to write a concrete vision statement for the project, which encompasses both the specific functions or user benefits it provides and the overall business problem it is expected to solve.
* Get your customer to read, think about and sign off on the completed software requirements specification, to align expectations and ensure that both parties have a clear understanding of the deliverabl

Problem 2: Requirements change during the course of the project

The second most common problem with software projects is that the requirements defined in the first phase change as the project progresses. This may occur because as development progresses and prototypes are developed, customers are able to more clearly see problems with the original plan and make necessary course corrections; it may also occur because changes in the external environment require reshaping of the original business problem and hence necessitates a different solution than the one originally proposed.

Good project managers are aware of these possibilities and typically already have backup plans in place to deal with these changes.

To solve this problem, you should:

* Have a clearly defined process for receiving, analyzing and incorporating change requests, and make your customer aware of his/her entry point into this process.
* Set milestones for each development phase beyond which certain changes are not permissible -- for example, disallowing major changes once a module reaches 75 percent completion.
* Ensure that change requests (and approvals) are clearly communicated to all stakeholders, together with their rationale, and that the master project plan is updated accordingly.


Problem 3: Customers have unreasonable timelines

It's quite common to hear a customer say something like "it's an emergency job and we need this project completed in X weeks". A common mistake is to agree to such timelines before actually performing a detailed analysis and understanding both of the scope of the project and the resources necessary to execute it. In accepting an unreasonable timeline without discussion, you are, in fact, doing your customer a disservice: it's quite likely that the project will either get delayed (because it wasn't possible to execute it in time) or suffer from quality defects (because it was rushed through without proper inspection).

To solve this problem, you should:

* Convert the software requirements specification into a project plan, detailing tasks and resources needed at each stage and modeling best-case, middle-case and worst-case scenarios.

* Ensure that the project plan takes account of available resource constraints and keeps sufficient time for testing and quality inspection.

* Enter into a conversation about deadlines with your customer, using the figures in your draft plan as supporting evidence for your statements. Assuming that your plan is reasonable, it's quite likely that the ensuing negotiation will be both productive and result in a favorable outcome for both parties.


Problem 4: Communication gaps exist between customers, engineers and project managers

Often, customers and engineers fail to communicate clearly with each other because they come from different worlds and do not understand technical terms in the same way. This can lead to confusion and severe miscommunication, and an important task of a project manager, especially during the requirements analysis phase, is to ensure that both parties have a precise understanding of the deliverable and the tasks needed to achieve it.

To solve this problem, you should:

* Take notes at every meeting and disseminate these throughout the project team.

* Be consistent in your use of words. Make yourself a glossary of the terms that you're going to use right at the start, ensure all stakeholders have a copy, and stick to them consistently.


Problem 5: The development team doesn't understand the politics of the customer's organization

The scholars Bolman and Deal suggest that an effective manager is one who views the organization as a "contested arena" and understands the importance of power, conflict, negotiation and coalitions. Such a manager is not only skilled at operational and functional tasks, but he or she also understands the importance of framing agendas for common purposes, building coalitions that are united in their perspective, and persuading resistant managers of the validity of a particular position.

These skills are critical when dealing with large projects in large organizations, as information is often fragmented and requirements analysis is hence stymied by problems of trust, internal conflicts of interest and information inefficiencies.

To solve this problem, you should:

* Review your existing network and identify both the information you need and who is likely to have it.
* Cultivate allies, build relationships and think systematically about your social capital in the organization.
* Persuade opponents within your customer's organization by framing issues in a way that is relevant to their own experience.
* Use initial points of access/leverage to move your agenda forward.

Kinds of Testing that should be considered for Websites.

Black box testing – Testing not based on any knowledge of the internal design or code. Tests are based on requirements and functionality.


Incremental Integration Testing – Continuous testing of website as new functionality is added; requires that various aspects of an web applications functionality be independent enough to work separately before all parts of the program are completed.


Integration Testing – Testing that is done on combined parts of an application to determine if they function together correctly. The ‘parts’ can be code modules, individual applications, pages in a website etc.


Functional Testing – Black box type testing geared to functional requirements of an application; testers should do this type of testing. This doesn’t mean that the programmers shouldn’t check that their code works before releasing it (which of course applies to any stage of testing.)

System Testing – Black box type testing that is based on overall requirements specifications; covers all combined parts of a web application.

End-to-end Testing – Similar to system testing; the ‘macro’ end of the test scale; involves testing of a complete application environment in a situation that mimics real-world use, such as interacting with a database, using network communications, or interacting with other hardware, applications, or systems if appropriate.

Sanity Testing or Smoke Testing – typically an initial testing effort to determine if a new web application is performing well enough to accept it for a major testing effort. For example, if there are lots of missing links, missing images, missing validations, or corrupting databases, the website may not be in a ’sane’ enough condition to warrant further testing in its current state.


Regression Testing – re-testing after fixes or modifications of the website or its pages. It can be difficult to determine how much re-testing is needed, especially near the end of the development cycle. Automated testing tools can be especially useful for this type of testing.


Acceptance Testing – final testing based on specifications of the end-user or customer, or based on use by end-users/customers over some limited period of time.


Load Testing – testing of a web site under a range of loads to determine at what point the system’s response time degrades or fails.


Stress Testing – term often used interchangeably with ‘load’ and ‘performance’ testing. Also used to describe such tests as system functional testing while under unusually heavy loads, heavy repetition of certain actions or inputs, input of large numerical values, large complex queries to a database system, etc.

Performance Testing – term often used interchangeably with ’stress’ and ‘load’ testing. Ideally ‘performance’ testing (and any other ‘type’ of testing) is defined in requirements documentation or QA or Test Plans.


Usability Testing – testing that is done for ‘user-friendliness’. Clearly this is subjective, and will depend on the targeted end-user or customer. User interviews, surveys, video recording of user sessions, and other techniques can be used. Programmers and testers are usually not appropriate as usability testers.


Security Testing – testing how well the system protects against unauthorized internal or external access, willful damage, etc; may require sophisticated testing techniques.


Compatibility Testing – testing how well website performs in a particular hardware/software/operating system//browser/network etc. environment.


Exploratory Testing – often taken to mean a creative, informal software test that is not based on formal test plans or test cases; testers may be learning the website as they test it.


Ad-hoc Testing – similar to exploratory testing, but often taken to mean that the testers have significant understanding of the website before testing it.

User acceptance Testing – determining if website is satisfactory to an end-user or customer.


Alpha Testing – testing of a web application when development is nearing completion; minor design changes may still be made as a result of such testing. Typically done by end-users or others, not by programmers or testers.


Beta Testing – testing when development and testing are essentially completed and final bugs and problems need to be found before final release. Typically done by end-users or others, not by programmers or testers.

Bug Life Cycle

The steps in defect life cycle varies from company to company. But the basic flow remains the same. However, below I'm describing a basic flow for Bug Life Cycle:

* A Tester finds a bug. Status --> Open
* Test lead review the bug and authoriz the bug. Stats --> Open
* Development team lead review the defect. Stats --> Open
* The defect can be authorized or unauthorized by the development team. (Here the status of the defect / bug will be Open (For Authorized Defects) & Reject (For Unauthorized Defects).
* Now, the authorized bugs will get fixed or deferred by the development team. Status of the fixed bugs will be Fixed & Status will be Deferred for the bugs which got Deferred.
* The Fixed bugs will be again re-tested by the testing team (Here based on the Closure of the Bug, the status will be made as Closed or if the bug still remains, it will be re-raised and status will be Re-opened.

The above-mentioned cycle continues until all the bugs / defects gets fixed in the application.

Software Testing Bug Report Template

In continuation to my previous post, here in this post, I'm explaining a simple and effective software bug report.

If you are using any Software Testing Management tool or any Bug reporting tool
like Bugzilla or Test Director or Bughost or any other online bug tracking tool, then; the tool will automatically generate the bug report. If you are not using any tool, you may refer to the following template for your software bug report:

* Name of Reporter:
* Email Id of Reporter:
* Version or Build:
* Module or component:
* Platform / Operating System:
* Type of error:
* Priority:
* Severity:
* Status:
* Assigned to:
* Summary:
* Description:

Software Testing Bug Report

After you complete your Software Testing, it is good practice to prepare an effective bug report. Fixing a bug depends on how effectively you report it. Below are some tips to write a good software bug report:

* If you are doing manual Software Testing and reporting bugs withour the help of any tool, assign a unique number to each bug report. This will help to identify the bug record.
* Clearly mention the steps to reproduce the bug. Do not assume or skip any reproducing step.
* Be Specific and to the point

Apart from these tips, below are some good practices:

* Report the problem immediately
* Reproduce the bug atleast one more time before you report it
* Test the same bug occurrence on other similar modules of the application
* Read bug report before you submit it or send it.
* Never ever criticize any developer or attack any individual

Validation vs Verification, Reviews, Inspections

Validation: The process of evaluating software at the end of the software development
process to ensure compliance with software requirements. It is actual testing of the application.

1. Am I building the right product.

2. Determining if the system complies with the requirements and performs functions for which it is intended and meets the organization’s goals and user needs. It is traditional and is performed at the end of the project.

3. Am I accessing the right data (in terms of the data required to satisfy the requirement).

4. High level activity.

5. Performed after a work product is produced against established criteria ensuring that the product integrates correctly into the environment.

6. Determination of correctness of the final software product by a development project with respect to the user needs and requirements.


Verification: The process of determining whether of not the products of a given phase of the software development cycle meet the implementation steps and can be traced to the incoming objectives established during the previous phase.

Verification process helps in detecting defects early, and preventing their leakage downstream. Thus, the higher cost of later detection and rework is eliminated. It includes:

1. Am I building the product right.

2. The review of interim work steps and interim deliverables during a project to ensure they are acceptable. To determine if the system is consistent, adheres to standards, uses reliable techniques and prudent practices, and performs the selected functions in the correct manner.

3. Am I accessing the data right (in the right place; in the right way).

4. Low level activity.

5. Performed during development on key artifacts, like walkthroughs, reviews and inspections, mentor feedback, training, checklists and standards.

6. Demonstration of consistency, completeness, and correctness of the software at each stage and between each stage of the development life cycle.


Review: A process or meeting during which a work product, or set of work products, is presented to project personnel, managers, users, customers, or other interested parties for comment or approval. The main goal of reviews is to find defects. Reviews are a good compliment to testing to help assure quality. A few purposes’ of SQA reviews can be as follows:

Assure the quality of deliverable’s before the project moves to the next stage.
Once a deliverable has been reviewed, revised as required, and approved, it can be used as a basis for the next stage in the life cycle.

Various types of reviews:

Management Reviews: Management reviews are performed by those directly responsible for the system in order to monitor progress, determine status of plans and schedules, confirm requirements and their system allocation. Therefore the main objectives of Management Reviews can be categorized as follows:

- Validate from a management perspective that the project is making progress according to the project plan.

- Ensure that deliverables are ready for management approvals.

- Resolve issues that require management’s attention.

- Identify any project bottlenecks.

- Keeping project in Control.

Requirement Review: A process or meeting during which the requirements for a system, hardware item, or software item are presented to project personnel, managers, users, customers, or other interested parties for comment or approval. Types include system requirements review, software requirements review. Product management leads Requirement Review. Members from every affected department participates in the review.

Input Criteria: Software requirement specification is the essential document for the review. A checklist can be used for the review.

Exit Criteria: Exit criteria include the filled & completed checklist with the reviewers’ comments & suggestions and the re-verification whether they are incorporated in the documents.

Design Review: A process or meeting during which a system, hardware, or software design is presented to project personnel, managers, users, customers, or other interested parties for comment or approval. Types include critical design review, preliminary design review, and system design review.


Inspection: A static analysis technique that relies on visual examination of development products to detect errors, violations of development standards, and other problems. Types include code inspection; design inspection, Architectural inspections, Test ware inspections etc. The participants in Inspections assume one or more of the following roles:

- Inspection leader
- Recorder
- Reader
- Author
- Inspector

All participants in the review are inspectors. The author shall not act as inspection leader and should not act as reader or recorder. Other roles may be shared among the team members. Individual participants may act in more than one role.

Individuals holding management positions over any member of the inspection team shall not participate in the inspection.

Input to the inspection shall include the following:

- A statement of objectives for the inspection
- The software product to be inspected
- Documented inspection procedure
- Inspection reporting forms
- Current anomalies or issues list

Input to the inspection may also include the following:

- Inspection checklists: Any regulations, standards, guidelines, plans, and procedures against which the software product is to be inspected.

- Hardware product specifications

- Hardware performance data

- Anomaly categories

The individuals may make additional reference material available responsible for the software product when requested by the inspection leader.

The purpose of the exit criteria is to bring an unambiguous closure to the inspection meeting. The exit decision shall determine if the software product meets the inspection exit criteria and shall prescribe any appropriate rework and verification. Specifically, the inspection team shall identify the software product disposition as one of the following:

Accept with no or minor rework: The software product is accepted as is or with only minor rework. (For example, that would require no further verification).

Accept with rework verification: The software product is to be accepted after the inspection leader or a designated member of the inspection team (other than the author) verifies rework.

Re-inspect: Schedule a re-inspection to verify rework. At a minimum, a re-inspection shall examine the software product areas changed to resolve anomalies identified in the last inspection, as well as side effects of those changes.

Software Testing Principles

Below are some basic Software Testing Principles:

- A necessary part of a test case is a definition of the expected output or result.

- A programmer should avoid attempting to test his or her own program.

- A programming organization should not test its own programs.

- Thoroughly inspect the results of each test.

- Test cases must be written for input conditions that are invalid and unexpected, as well as for those that are valid and expected.

- Examining a program to see if it does not do what it is supposed to do is only half the battle; the other half is seeing whether the program does what it is not supposed to do.

- Avoid throwaway test cases unless the program is truly a throwaway program.

- Do not plan a testing effort under the tacit assumption that no errors will be found.

- The probability of the existence of more errors in a section of a program is proportional to the number of errors already found in that section.

- Software Testing is an extremely creative and intellectually challenging task.

Software Testing Process

Below is a very basic software testing process
. Many companies use this process. Later on I'll post some more testing processes.

1. Understand of business logic and analysis of requirements: In this phase consider the following:

* Are the definitions and descriptions of the required functionalities precise?
* Is there clear delineation between the system and its environment?
* Can the requirements be realized in practice?
* Can the requirements be tested effectively?

2. Test Planning: During this phase Test Strategy is defined and Test Bed is created. The plan should identify:-

* Which aspects of the system should be tested.
* The methods, techniques and tools to be used.
* Personnel responsible for the testing.
* Manual and Automation Testing
* Defect Management and Risk Management etc.


3. Test Environment Setup: A different testing server is prepared where the application will be tested. It is an independent testing environment.



4. Test Design: Identify the test scenarios and prepare the test cases / scripts. Selection of test data is also done in this phase. If required, test designing is done with some automated tools like QTP or LoadRunner or with some other software testing tool. Designing framework, scripting, script integration, Review and approval will be undertaken in this phase.



5. Test Execution: Testers execute the test cases and report any errors found to the development team.



6. Defect Tracking: Raised defects are tracked using some tools like Test Director or Bug Host etc.



7. Test Reports: As soon as testing is completed, Test Lead or Manager generate metrics and make final reports for the whole testing effort.

The Big Picture (Software testing)

All software problems can be termed as bugs. A software bug usually occurs when the software

does not do what it is intended to do or does something that it is not intended to do. Flaws in specifications, design, code or other reasons can cause these bugs. Identifying and fixing bugs in the early stages of the software is very important as the cost of fixing bugs grows over time. So, the goal of a software tester is to find bugs and find them as early as possible and make sure they are fixed.

Testing is context-based and risk-driven. It requires a methodical and disciplined approach to finding bugs. A good software tester needs to build credibility and possess the attitude to be explorative, troubleshooting, relentless, creative, diplomatic and persuasive.

As against the perception that testing starts only after the completion of coding phase, it actually begins even before the first line of code can be written. In the life cycle of the conventional software product, testing begins at the stage when the specifications are written, i.e. from testing the product specifications or product spec. Finding bugs at this stage can save huge amounts of time and money.

Once the specifications are well understood, you are required to design and execute the test cases. Selecting the appropriate technique that reduces the number of tests that cover a feature is one of the most important things that you need to take into consideration while designing these test cases. Test cases need to be designed to cover all aspects of the software, i.e. security, database, functionality (critical and general) and the user interface. Bugs originate when the test cases are executed.

As a tester you might have to perform testing under different circumstances, i.e. the application could be in the initial stages or undergoing rapid changes, you have less than enough time to test, the product might be developed using a life cycle model that does not support much of formal testing or retesting. Further, testing using different operating systems, browsers and the configurations are to be taken care of.

Reporting a bug may be the most important and sometimes the most difficult task that you as a software tester will perform. By using various tools and clearly communicating to the developer, you can ensure that the bugs you find are fixed.

Using automated tools to execute tests, run scripts and tracking bugs improves efficiency and effectiveness of your tests. Also, keeping pace with the latest developments in the field will augment your career as a software test engineer.