Why Performance Test?
- Speed – Does the application respond quickly enough for the intended users?
- Scalability – Will the application handle the expected user load and beyond? (AKA Capacity)
- Stability – Is the application stable under expected and unexpected user loads? (AKA Robustness)
- Confidence – Are you sure that users will have a positive experience on go-live day?
If you know what the performance is…
- you can assess risk.
- you can make informed decisions.
- you can plan for the future.
- you can sleep the night before go-live day.
The peace of mind that it will work on go-live day alone justifies the cost of performance testing.
Performance Testing Types:
1) Load Testing:
Load Testing is type of performance testing to check system with constantly increasing the load on the system until the time load is reaches to its threshold value. Here Increasing load means increasing number of concurrent users, transactions & check the behavior of application under test. It is normally carried out underneath controlled environment in order to distinguish between two different systems. It is also called as “Endurance testing” and “Volume testing”. The main purpose of load testing is to monitor the response time and staying power of application when system is performing well under heavy load. Load testing comes under the Non Functional Testing & it is designed to test the non-functional requirements of a software application.
Load testing is perform to make sure that what amount of load can be withstand the application under test. The successfully executed load testing is only if the specified test cases are executed without any error in allocated time.
Simple examples of load testing:
Testing printer by sending large job.
Editing a very large document for testing of word processor
Continuously reading and writing data into hard disk.
Running multiple applications simultaneously on server.
Testing of mail server by accessing thousands of mailboxes
In case of zero-volume testing & system fed with zero load.
2) Stress Testing:
Stress Testing is performance testing type to check the stability of software when hardware resources are not sufficient like CPU, memory, disk space etc.
Stress testing is Negative testing where we load the software with large number of concurrent users/processes which cannot be handled by the systems hardware resources. This testing is also known as Fatigue testing, this testing should capture the stability of the application by testing it beyond its bandwidth capacity.
The main idea behind stress testing is to determine the failure of system and to keep an eye on how the system gracefully get recover back, this quality is known as recoverability. Stress testing comes under the Non Functional Testing & it is designed to test the non-functional requirements of a software application. This testing is to be carried out under controlled environment before launch, so that we can accurately capture the system behavior under most erratic scenarios
3) Spike testing:
Spike testing is subset of Stress Testing. A spike test is carried out to validate the performance characteristics when the system under test subjected to workload models and load volumes that repeatedly increase beyond anticipated production operations for short periods of time.
4) Endurance testing:
Endurance testing is a non functional type of testing. Endurance testing involves testing a system with a expected amount of load over a long period of time to find the behavior of system. Let’s take a example where system is designed to work for 3 hrs of time but same system endure for 6 hrs of time to check the staying power of system. Most commonly test cases are executed to check the behavior of system like memory leaks or system fails or random behavior. Sometimes endurance testing is also referred as Soak testing.
5) Scalability Testing:
Scalability Testing is type of non-functional tests and it is the testing of a software application for determine its capability to scale up in terms of any of its non-functional capability like the user load supported, the number of transactions, the data volume etc. The main aim if this testing is to understand at what peak the system prevent more scaling.
6) Volume testing:
Volume testing is non-functional testing which refers to testing a software application with a large amount of data to be processed to check the efficiency of the application. The main goal of this testing is to monitor the performance of application under varying database volumes.
Process of Performance Testing:
1) Identify your testing environment –
Do proper requirement study & analysing test goals and its objectives. Also determine the testing scope along with test Initiation Checklist. Identify the logical and physical production architecture for performance testing, identify the software, hardware and networks configurations required for kick off the performance testing. Compare the both test and production environments while identifying the testing environment. Get resolve the environment-related concerns if any, analyze that whether additional tools are required for performance testing. This step also helps to identify the probable challenges tester may face while performance testing.
2) Identify the performance acceptance criteria –
Identify the desired performance characteristics of the application like Response time, Throughput and Resource utilisation.
3) Plan & design performance tests –
Planning and designing performance tests involves identifying key usage scenarios, determining appropriate variability across users, identifying and generating test data, and specifying the metrics to be collected. Ultimately, these items will provide the foundation for workloads and workload profiles. The output of this stage is prerequisites for Test execution are ready, all required resources, tools & test data are ready.
4) Configuring the test environment –
Prepare with conceptual strategy, available tools, designed tests along with testing environment before execution. The output of this stage is configured load-generation environment and resource-monitoring tools.
5) Implement test design –
According to test planning and design create your performance tests.
6) Execute the tests –
Collect and analyse the data.
Problem Investigation like bottlenecks (memory, disk, processor, process, cache, network, etc.) resource usage like (memory, CPU, network, etc.,)
Generate the Performance analysis reports containing all performance attributes of the application.
Based on the analysis prepare recommendation report.
Repeat the above test for the new build received from client after fixing the bugs and implementing the recommendations
7) Analyse Results, Report, and Retest
Consolidate, analyse and share test results.
Based on the test report re-prioritise the test & re-execute the same. If any specific test result within the specified metric limit & all results are between the thresholds limits then testing of same scenario on particular configuration is completed.